Showing preview only (877K chars total). Download the full file or copy to clipboard to get everything.
Repository: BlockScience/cadCAD
Branch: master
Commit: dff463842775
Files: 101
Total size: 840.0 KB
Directory structure:
gitextract_5t5ec52q/
├── .github/
│ ├── FUNDING.yml
│ └── workflows/
│ ├── cadcad-ci.yml
│ └── cadcad-publish.yml
├── .gitignore
├── AUTHORS.txt
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.txt
├── README.md
├── ascii_art.py
├── cadCAD/
│ ├── __init__.py
│ ├── configuration/
│ │ ├── __init__.py
│ │ └── utils/
│ │ ├── __init__.py
│ │ ├── depreciationHandler.py
│ │ ├── policyAggregation.py
│ │ └── userDefinedObject.py
│ ├── diagram/
│ │ ├── __init__.py
│ │ └── config_diagram.py
│ ├── engine/
│ │ ├── __init__.py
│ │ ├── execution.py
│ │ ├── simulation.py
│ │ └── utils.py
│ ├── tools/
│ │ ├── __init__.py
│ │ ├── execution/
│ │ │ ├── __init__.py
│ │ │ └── easy_run.py
│ │ ├── preparation.py
│ │ ├── profiling/
│ │ │ ├── __init__.py
│ │ │ ├── profile_run.py
│ │ │ └── visualizations.py
│ │ ├── types.py
│ │ └── utils.py
│ ├── types.py
│ └── utils/
│ ├── __init__.py
│ ├── execution.py
│ ├── jupyter.py
│ └── sys_config.py
├── documentation/
│ ├── Historically_State_Access.md
│ ├── Policy_Aggregation.md
│ ├── README.md
│ ├── Simulation_Execution.md
│ ├── System_Configuration.md
│ ├── System_Model_Parameter_Sweep.md
│ ├── __init__.py
│ ├── cadCAD-v0.4.23-Model-Upgrade-Guide.md
│ ├── cadCAD-v0.4.27-Model-Upgrade-Guide.md
│ ├── cadCAD-v0.4.28-Model-Upgrade-Guide.md
│ └── examples/
│ ├── __init__.py
│ ├── cadCAD_diagram.ipynb
│ ├── cadCAD_tools_example.ipynb
│ ├── example_1.py
│ ├── historical_state_access.py
│ ├── param_sweep.py
│ ├── policy_aggregation.py
│ ├── sys_model_A.py
│ ├── sys_model_AB_exec.py
│ ├── sys_model_A_exec.py
│ ├── sys_model_B.py
│ └── sys_model_B_exec.py
├── expected_results/
│ ├── param_sweep_4.pkl
│ ├── param_sweep_5.pkl
│ ├── param_sweep_psub0_4.pkl
│ ├── param_sweep_timestep1_4.pkl
│ ├── policy_agg_4.pkl
│ └── policy_agg_5.pkl
├── requirements.txt
├── setup.py
└── testing/
├── __init__.py
├── experiments/
│ └── __init__.py
├── generic_test.py
├── models/
│ ├── __init__.py
│ ├── param_sweep.py
│ └── policy_aggregation.py
├── results_comparison.py
├── test_additional_objs.py
├── test_arg_count.py
├── test_param_count.py
├── test_print.py
├── test_results_signature.py
├── test_row_count.py
├── test_runs.py
├── tests/
│ ├── __init__.py
│ ├── a_b_tests/
│ │ ├── 0_4_23_record_count.json
│ │ └── multi_model_row_count_0_4_23.py
│ ├── append_mod_test.py
│ ├── expected_results/
│ │ ├── param_sweep_4.pkl
│ │ ├── param_sweep_5.pkl
│ │ ├── param_sweep_psub0_4.pkl
│ │ ├── param_sweep_timestep1_4.pkl
│ │ ├── policy_agg_4.pkl
│ │ └── policy_agg_5.pkl
│ ├── import_cadCAD.ipynb
│ ├── test_cadCAD_exp.py
│ ├── test_import_cadCAD_test.py
│ ├── test_multi_model_row_count.py
│ ├── test_param_sweep.py
│ ├── test_policy_aggregation.py
│ ├── test_run1psub0.py
│ ├── test_runs_not_zero.py
│ └── test_timestep1psub0.py
├── tools/
│ └── test_tools.py
└── utils.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .github/FUNDING.yml
================================================
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: cadcad1
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
================================================
FILE: .github/workflows/cadcad-ci.yml
================================================
# This workflow will install Python dependencies and run tests with multiple versions of Python
name: cadCAD CI
on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
workflow_dispatch:
permissions:
contents: read
jobs:
windows-build:
if: false
continue-on-error: true
runs-on: windows-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Install test and build dependencies
run: |
python -m pip install --upgrade pip
python -m pip install jupyter
pip install -r requirements.txt
- name: Build cadCAD
shell: cmd
run: |
python setup.py bdist_wheel
for %%x in ("dist\*.whl") do python -m pip install %%x --force-reinstall
- name: Run tests
run: |
python -m pytest
build:
continue-on-error: true
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
os: [ubuntu-latest, macos-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Install test and build dependencies
run: |
python -m pip install --upgrade pip
python -m pip install jupyter
pip install -r requirements.txt
- name: Build cadCAD
run: |
python setup.py bdist_wheel
python -m pip install dist/*.whl --force-reinstall
- name: Run tests
run: |
python -m pytest
================================================
FILE: .github/workflows/cadcad-publish.yml
================================================
# This workflow will publish the cadCAD library to PyPI when a release is created
name: cadCAD Publish
on:
release:
types: [published]
jobs:
pypi-publish:
name: Upload release to PyPI
if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/cadCAD
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: Build cadCAD
run: |
python setup.py bdist_wheel
- name: Publish package
uses: pypa/gh-action-pypi-publish@release/v1
================================================
FILE: .gitignore
================================================
*.idea
*.vscode
*.ipynb_checkpoints
*.DS_Store
*.eggs
*.pytest_cache
*.mypy_cache
*.csv
*.egg-info
*.sqlite3
*.pyc
*venv
*external_data
*sys_exec.py
*/__pycache__
*.DS_Store
*notes.*
*announcement.md
*dist.py
*/poc
__pycache__
Pipfile
Pipfile.lock
build
results
result
notebooks
client_work
monkeytype
venv
notes.*
announcement.md
testing/tests/a_b_tests/venv_0_4_23
simulations/tickets
simulations/validation
simulations/poc/local/
simulations/regression_tests/poc
simulations/regression_tests/poc_configs
dist/*
testing/tests/import_cadCAD.nbconvert.ipynb
testing/tests/cadCAD_memory_address.json
================================================
FILE: AUTHORS.txt
================================================
Authors
=======
cadCAD was originally implemented by Joshua E. Jodesty and designed by Michael Zargham, Markus B. Koch, and
Matthew V. Barlin in 2018. Since then, upgrades and improvements were further committed by Danilo L. Bernardineli, Tyler D. Mace
and Emanuel Lima.
Current Project Maintainers:
- Emanuel Lima <emanuel@block.science> @emanuellima1
- Michael Zargham <zargham@block.science> @mzargham
Active and Past Contributors:
- Joshua E. Jodesty @JEJodesty
- Markus B. Koch @markusbkoch
- Matthew V. Barlin @matttyb80
- Michael Zargham @mzargham
- Danilo L. Bernardineli @danlessa
- Tyler D. Mace @tylerdmace
- Emanuel Lima @emanuellima1
- Zixuan Zhang @zixuanzh
- Charles Rice @charles-rice
- Benjamin Scholz @BenSchZA
We’d also like to thank:
- Andrew Clark @aclarkData
- Nick Hirannet @nick-phl-7
- Jonathan Gabler
- Harry Goodnight @
- Charlie Hoppes
- Nikhil Jamdade
- Chris Frazier
- Griff Green @GriffGreen
- Isaac @eenti
- Jonny Dubowsky @jonnydubowsky
- Sem Brestels @sembrestels
- @fjribi
- Andrea Maria Piana @cammellos
================================================
FILE: CHANGELOG.md
================================================
# Changelog:
## 0.5.3 - April 19, 2024
### Bugs fixed
- Solve issue on which nested initial states are being copied from one run to another ([#357](https://github.com/cadCAD-org/cadCAD/pull/357));
- Created add_parameter_labels ([#360](https://github.com/cadCAD-org/cadCAD/pull/360));
- Add supress_print attribute to the Executor class for silencing prints and tqdm ([#361](https://github.com/cadCAD-org/cadCAD/pull/361));
## 0.5.2 - April 11, 2024
### Bugs fixed
- fix: easy supports list/array/sequence type as parameter ([#348](https://github.com/cadCAD-org/cadCAD/pull/348));
- Update setup.py to include tqdm package ([#342](https://github.com/cadCAD-org/cadCAD/pull/342));
- Debug M dictionairy unpacking in utils ([#343](https://github.com/cadCAD-org/cadCAD/pull/343));
- Add test for counting the parameters argument inside policies and SUFs ([#345](https://github.com/cadCAD-org/cadCAD/pull/345));
- Fix sweep_cartesian_product such that it works nicely with easy_run ([#355](https://github.com/cadCAD-org/cadCAD/pull/355));
- Update easy_run.py ([#356](https://github.com/cadCAD-org/cadCAD/pull/356));
## 0.5.1 - December 21, 2023
### Bugs fixed
- Fixed a bug where the single_proc mode would result in a missing simulation key ([#332](https://github.com/cadCAD-org/cadCAD/issues/332));
- Fixed a bug where the single_proc mode would not return results ([#335](https://github.com/cadCAD-org/cadCAD/issues/335));
- Fixed a wrong parameter order when doing single runs ([#337](https://github.com/cadCAD-org/cadCAD/issues/337)).
- Added a Windows CI pipeline;
## 0.5.0 - December 20, 2023
### New User Features
- Added toggle for enabling users to deactivate deepcopying. This is done by passing an additional object on the `ExecutionContext`, eg. `ExecutionContext(mode, additional_objs={'deepcopy_off': True})`
### New Submodules
- A collection of type annotations for encapsuling `cadCAD` projects is now implemented through the `cadCAD.types` submodules
- Added `cadCAD.tools` as a submodule, which is originated from the `cadCAD_tools` Python package. This submodule contains several helper functions for making the simulation experience more straightforward as well as a collection of performance profiling tools.
- Added `cadCAD.diagram` as a submodule, which is originated from the `cadCAD_diagram` Python package. This submodule contains functions for programatically generating block diagrams from existing models.
- More informative error messages when policies and SUFs are wrongly implemented. (Issues #288 and #258)
### Backend Improvements
- Merged repo with the `cadCAD_legacy_devel`, which includes performance improvements. In particular, simulations will start up faster due to code optimizations.
- `cadCAD` now uses `pytest` as the testing framework. This was made possible by isolating the existing tests and wrapping them into functions.
### Fixes
- cadCAD is now Python 3.10+ compatible (Issue #306 and #301)
- Proper support for `ExecutionMode.single_mode` (Issue #253 and #254)
## September 28, 2021
### New Features:
* **ver. ≥ `0.4.28`:**
* #### [Experiments](documentation#experiments)
* #### System Model Configuration
* Configurations (`cadCAD.utils.Configuration`'s) are now accessed via the `configs` member of `cadCAD.configuration.Experiment`.
[Example:](documentation#experiments) `cadCAD.configuration.Experiment().configs`
* `cadCAD.configs` has been re-included for backwards compatibility and has been assigned `cadCAD.experiment.configs`
* #### Experiments
* `cadCAD.configuration.Experiment()` is unique representation of an experiment of one or more configured System
Models.
* The `cadCAD` module now contains a default Experiment object `cadCAD.experiment` as an instantiation of
`cadCAD.configuration.Experiment()`
* An `Experiment`'s `append_model` method stores multiple system model `Configuration`'s for simulation
execution within `cadCAD.configuration.Experiment().configs`.
`cadCAD.configuration.Experiment().model_ids` contains system model labels and/or indexes for
`cadCAD.configuration.Experiment().configs`
* The `Experiment`'s `append_model` method is defined with `model_id` parameter that accepts a system model
label.
* If duplicate `model_id`'s are specified, an index is appended to the label after the `@` symbol.
(Example: `cadCAD.configuration.Experiment().model_ids = ['sys_model', 'sys_model@1', 'sys_model@2', ...]`)
* If `model_id`'s are not specified or duplicate, the label is auto-generated as a string indicating the
system model index within `cadCAD.configuration.Experiment().configs`.
(Example of unspecified system models at indexes 1, 3, and 4:
`cadCAD.configuration.Experiment().model_ids = ['sys_model', '1', 'sys_model@2', '3', '4', ...]`)
* #### [Upgrade Guide:](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/cadCAD-v0.4.28-Model-Upgrade-Guide.md) specific to feature changes / additions
## August 25, 2021
### New Features:
* **ver. ≥ `0.4.27`:**
* #### [Experiments](documentation#experiments)
* #### System Model Configurations
* Configurations (`cadCAD.utils.Configuration`'s) as are no longer a part of the `cadCAD` module
(as `cadCAD.configs`) and are now accessed via the `configs` member of `cadCAD.configuration.Experiment`.
[Example:](documentation#experiments) `cadCAD.configuration.Experiment().configs`
* #### Experiments
* `cadCAD.configuration.Experiment` is unique representation of an experiment of one or more configured System
Models. An `Experiment`'s `append_model` method stores multiple system model `Configuration`'s for simulation
execution.
* The `Experiment`'s `append_model` method requires a `model_id` parameter that is auto-created as `'sys_model_#'`.
* **Requirements:**
* Users must use different `model_id`'s when appending multiple System Model Configurations.
* Users can no longer use the `config_list` method of `cadCAD.configuration.Experiment`
* **Backwards Compatibility:** The `append_model` method of `cadCAD.configuration.Experiment` can also be used as
the `append_configs` method.
* Removed [Nix](https://nixos.org/)
* #### [Upgrade Guide:](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/cadCAD-v0.4.27-Model-Upgrade-Guide.md) specific to feature changes / additions
* **Fixes:**
* [#248](https://github.com/cadCAD-org/cadCAD/issues/248)
* The previous release was returning partial results. An A/B test for this has been included and will be for
future releases
* [#242](https://github.com/cadCAD-org/cadCAD/issues/242)
* Parallelized simulations enabled with the re-inclusion of `ProcessPool`.
* [#257](https://github.com/cadCAD-org/cadCAD/issues/257)
* ValueError for runs accepted by the `cadCAD.configuration.Experiment().append_model` via the `sim_configs` no
longer gives mis-leading error message if catching a non-related ValueError
* [#252](https://github.com/cadCAD-org/cadCAD/issues/252)
* Jupyter lab and Jupyter notebook recognises cadCAD module
## September 22, 2020
#### [Multi - System Model Execution](https://github.com/cadCAD-org/cadCAD/blob/master/documentation/Simulation_Execution.md#multiple-simulation-execution)
* **ver. ≥ `0.4.23`:**
* **Hot-Fix:** [#203](https://github.com/cadCAD-org/cadCAD/pull/203) (**No Breaking Changes**)
* Multi - System Model simulation results will no longer return truncated results (exclude the results of the last
`cadCAD.configuration.Configuration` appended to `cadCAD.configs`).
* Issue: [#195](https://github.com/cadCAD-org/cadCAD/issues/195)
* Parameter Sweep value `M` (Params) requires up to a maximum of 2 distinct lengths
## August 5, 2020
#### [Experiments](documentation#experiments)
* `cadCAD.configuration.Experiment` (Alpha) is in development and needed to be released to support the implementation of
web applications and proprietary feature extensions. It is intended to represent a unique identifier of an experiment of
one or more configured System Models. For this reason, `append_configs` is a method of
`cadCAD.configuration.Experiment`. As of now it does not support multi - system model simulation because configurations
are still appended globally despite `append_config` being a method of Experiment.
##### Examples:
* **ver. ≥ `0.4.22`:**
```python
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_configs(...)
```
* **ver. `0.3.1`:** *Deprecated*
```python
from cadCAD.configuration import append_configs
append_configs(...)
```
## June 22, 2020
* Bug Fix: Multiprocessing error for Windows
## June 19, 2020
## New Features:
#### [Local Execution Mode (Default)](documentation/Simulation_Execution.md#simulation-execution-modes)
* Local Execution Mode (Default): Implicit parallelization of Monte-Carlo / Stochastic simulations (Automatically
selects Multi-Threaded Mode if simulations are configured for more than a single run)
* **Backwards Compatibility:** `cadCAD.engine.ExecutionMode` accepts legacy execution modes from ver. `0.3.1`
##### Examples:
* **ver. ≥ `0.4.22`:**
```python
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
```
* **ver. `0.3.1`:**
Multi-Threaded:
```python
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
single_ctx = ExecutionContext(context=exec_mode.multi_proc)
```
Single-Thread:
```python
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
multi_ctx = ExecutionContext(context=exec_mode.single_proc)
```
#### cadCAD Post-Processing Enhancements / Modifications
* [**Single Result Dataset**](documentation/Simulation_Execution.md#4-execute-simulation--produce-system-event-dataset)
as a 2 dimensional `list`
* Returns a single dataset instead of multiple datasets per Monte Carlo simulation as in `0.3.1`:
* New System Metrics as dataset attributes:
* **Simulation** (Alpha) is a unique identifier being developed to represent Experiments as stated above and
will be renamed accordingly
* **Subset** is a unique identifier of Monte-Carlo simulations produced by parameter sweeps
* Note: Returning a single dataset was originally specified during the project’s inception instead of multiple per
simulation
##### Examples:
* **ver. ≥ `0.4.22`:**
```python
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
simulation = Executor(exec_context=local_ctx, configs=configs)
raw_result, sys_model, _ = simulation.execute()
result = pd.DataFrame(raw_result)
print(tabulate(result, headers='keys', tablefmt='psql'))
```
Results:
```
+----+------------+-----------+----+---------------------+------------+--------+-----+---------+----------+
| | s1 | s2 | s3 | timestamp | simulation | subset | run | substep | timestep |
|----+------------+-----------+----+---------------------+------------+--------+-----+---------+----------|
| 0 | 0.0 | 0.0 | 1 | 2018-10-01 15:16:24 | 0 | 0 | 1 | 0 | 0 |
| 1 | 1.0 | 4 | 5 | 2018-10-01 15:16:25 | 0 | 0 | 1 | 1 | 1 |
| 2 | 2.0 | 6 | 5 | 2018-10-01 15:16:25 | 0 | 0 | 1 | 2 | 1 |
| 3 | 3.0 | [ 30 300] | 5 | 2018-10-01 15:16:25 | 0 | 0 | 1 | 3 | 1 |
| 4 | 0 | 0 | 1 | 2018-10-01 15:16:24 | 1 | 0 | 1 | 0 | 0 |
| 5 | 1 | 0 | 5 | 2018-10-01 15:16:25 | 1 | 0 | 1 | 1 | 1 |
| 6 | a | 0 | 5 | 2018-10-01 15:16:25 | 1 | 0 | 1 | 2 | 1 |
| 7 | ['c', 'd'] | [ 30 300] | 5 | 2018-10-01 15:16:25 | 1 | 0 | 1 | 3 | 1 |
+----+------------+-----------+----+---------------------+------------+--------+-----+---------+----------+
```
* **ver. `0.3.1`:**
```python
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
multi_ctx = ExecutionContext(context=exec_mode.multi_proc)
simulation = Executor(exec_context=multi_ctx, configs=configs)
i = 0
config_names = ['sys_model_A', 'sys_model_B']
for raw_result, _ in simulation.execute():
result = pd.DataFrame(raw_result)
print()
print(f"{config_names[i]} Result: System Events DataFrame:")
print(tabulate(result, headers='keys', tablefmt='psql'))
print()
i += 1
```
Results:
```
+----+------------+-----------+----+---------------------+-----+---------+----------+
| | s1 | s2 | s3 | timestamp | run | substep | timestep |
|----+------------+-----------+----+---------------------+-----+---------+----------|
| 0 | 0.0 | 0.0 | 1 | 2018-10-01 15:16:24 | 1 | 0 | 0 |
| 1 | 1.0 | 4 | 5 | 2018-10-01 15:16:25 | 1 | 1 | 1 |
| 2 | 2.0 | 6 | 5 | 2018-10-01 15:16:25 | 1 | 2 | 1 |
| 3 | 3.0 | [ 30 300] | 5 | 2018-10-01 15:16:25 | 1 | 3 | 1 |
| 4 | 0 | 0 | 1 | 2018-10-01 15:16:24 | 1 | 0 | 0 |
| 5 | 1 | 0 | 5 | 2018-10-01 15:16:25 | 1 | 1 | 1 |
| 6 | a | 0 | 5 | 2018-10-01 15:16:25 | 1 | 2 | 1 |
| 7 | ['c', 'd'] | [ 30 300] | 5 | 2018-10-01 15:16:25 | 1 | 3 | 1 |
+----+------------+-----------+----+---------------------+-----+---------+----------+
```
* **Flattened Configuration list:** The `cadCAD.configs` (System Model Configuration) `list` has been **temporarily**
flattened to contain single run `cadCAD.configuration.Configuration` objects to both fault-tolerant simulation and
elastic workloads. This functionality will be restored in a subsequent release by a class that returns
`cadCAD.configs`'s original representation in ver. `0.3.1`.
* The conversion utilities have been provided to restore its original representation of configurations with
runs >= 1
* [System Configuration Conversions](documentation/System_Configuration.md)
* Configuration as list of Configuration Objects (as in ver. `0.3.1`)
* New: System Configuration as a Pandas DataFrame
* New: System Configuration as list of Dictionaries
##### Examples:
* Notes:
* `configs` is temporarily returned in a flattened format and reformatted into its intended format
* `Configuration` objects at `0x10790e470` and `0x1143dd630` are reconstituted into objects at `0x10790e7b8`
and `0x116268908` respectively.
* **ver. ≥ `0.4.22`:**
```python
from pprint import pprint
from documentation.examples import sys_model_A, sys_model_B
from cadCAD.configuration.utils import configs_as_objs, configs_as_dataframe, configs_as_dicts
from cadCAD import configs
flattened_configs = configs
print('Flattened Format: Temporary')
pprint(flattened_configs)
print()
print('Intended Format:')
intended_configs = configs_as_objs(flattened_configs)
pprint(intended_configs)
print()
```
Result:
```bash
Flattened Format: Temporary
[<cadCAD.configuration.Configuration object at 0x10790e470>,
<cadCAD.configuration.Configuration object at 0x10790e7b8>,
<cadCAD.configuration.Configuration object at 0x1143dd630>,
<cadCAD.configuration.Configuration object at 0x116268908>]
Intended Format:
[<cadCAD.configuration.Configuration object at 0x10790e7b8>,
<cadCAD.configuration.Configuration object at 0x116268908>]
```
#### Expandable state and policy update parameter space:
* Enables the development of feature enhancements that involve the use of additional parameters without requiring users
to modify their update parameters spaces when upgrading to newer versions. For this reason state / policy update
examples in documentation include an additional `**kwargs` parameter.
* [State Updates](documentation/README.md#state-update-functions)
* [Policy Updates](documentation/README.md#state-update-functions)
##### Examples:
* **ver. ≥ `0.4.22`:**
```python
def state_update(_params, substep, sH, s, _input, **kwargs):
...
return 'state_variable_name', new_value
def policy(_params, substep, sH, s, **kwargs):
...
return {'signal_1': value_1, ..., 'signal_N': value_N}
```
* **ver. `0.3.1`:**
```python
def state_update(_params, substep, sH, s, _input):
...
return 'state_variable_name', new_value
def policy(_params, substep, sH, s):
...
return {'signal_1': value_1, ..., 'signal_N': value_N}
```
## May 29, 2020
* Packaging: Add [Nix](https://nixos.org/) derivation and shell for local development and distribution of cadCAD package
using Nix. Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and
reproducible, allowing you to share your development and build environments across different machines.
================================================
FILE: CONTRIBUTING.md
================================================
# Contributing to cadCAD
:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:
The following is a set of guidelines for contributing to cadCAD.
Use your best judgment, and feel free to propose changes to this document in a pull request.
### Pull Requests:
Pull Request (PR) presented as "->".
General Template:
`user:branch -> org:staging`
Contributing a new feature:
`user:feature -> org:staging`
Contributing to an existing feature:
`user:feature -> org:feature`
### General Advise for Forked Repositories:
1. `git pull fork staging`
2. `git checkout -b feature` (new feature)
3. Apply your awesomeness! (Commit Often)
4. `git push fork feature`
5. Apply a rebase/merge strategy you're comfortable with (Recommended Below).
6. Submit PR from `user:staging` into `org:staging`
7. PR is queued for review
8. PR Reviewed (Update necessary if rejected)
9. PR Approved (There may be circumstances delaying the merge.)
10. Your contribution merged into next feature release on `org:master`
### Recommended Strategy: [Rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing)
1. Add cadCAD-org/cadCAD as remote within you forked project locally.
2. `git checkout remote/master`
3. `git pull remote master`
4. `git checkout your_branch`
5. `git rebase master`
6. Resolve merge conflicts (while leveraging rebase commands)
7. `git push fork your_branch`
Thanks! :heart:
================================================
FILE: LICENSE.txt
================================================
MIT License
Copyright (c) 2018-2020 BlockScience
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
```
___________ ____
________ __ ___/ / ____/ | / __ \
/ ___/ __` / __ / / / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/ |_/_____/
by cadCAD ver. 0.5.3
======================================
Complex Adaptive Dynamics
o i e
m d s
p e i
u d g
t n
e
r
```
***cadCAD*** is a Python package that assists in the processes of designing, testing and validating complex systems
through simulation, with support for Monte Carlo methods, A/B testing and parameter sweeping.
# Getting Started
#### Change Log: [ver. 0.5.3](CHANGELOG.md)
## 0. Pre-installation Virtual Environments with [`venv`](https://docs.python.org/3/library/venv.html) (Optional):
If you wish to create an easy to use virtual environment to install cadCAD within, please use python's built in `venv` package.
***Create** a virtual environment:*
```bash
$ python3 -m venv ~/cadcad
```
***Activate** an existing virtual environment:*
```bash
$ source ~/cadcad/bin/activate
(cadcad) $
```
***Deactivate** virtual environment:*
```bash
(cadcad) $ deactivate
$
```
## 1. Installation:
Requires [>= Python 3.9.0](https://www.python.org/downloads/)
**Option A:** Install Using **[pip](https://pypi.org/project/cadCAD/)**
```bash
pip3 install cadCAD
```
**Option B:** Build From Source
```
pip3 install -r requirements.txt
python3 setup.py sdist bdist_wheel
pip3 install dist/*.whl
```
## 2. Documentation:
* [Simulation Configuration](documentation/README.md)
* [Simulation Execution](documentation/Simulation_Execution.md)
* [Policy Aggregation](documentation/Policy_Aggregation.md)
* [Parameter Sweep](documentation/System_Model_Parameter_Sweep.md)
* [Display System Model Configurations](documentation/System_Configuration.md)
## 3. Connect:
* Website: https://www.cadcad.org
* Discord: https://discord.gg/DX9uH8m4qY
* Twitter: https://twitter.com/cadcad_org
* Forum: https://community.cadcad.org
* Github: https://github.com/cadCAD-org
* Telegram: https://t.me/cadcad_org
## 4. Contribute to this repository:
Follow [this document](CONTRIBUTING.md) to start contributing!
================================================
FILE: ascii_art.py
================================================
production = r'''
__________ ____
________ __ _____/ ____/ | / __ \
/ ___/ __` / __ / / / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/ |_/_____/
by BlockScience
======================================
Complex Adaptive Dynamics
o i e
m d s
p e i
u d g
t n
e
r
'''
text = r'''
Complex Adaptive Dynamics
o i e
m d s
p e i
u d g
t n
e
r
'''
block_letters = r'''
__________ ____
________ __ _____/ ____/ | / __ \
/ ___/ __` / __ / / / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/ |_/_____/
by BlockScience
'''
================================================
FILE: cadCAD/__init__.py
================================================
import os
import dill
from cadCAD.configuration import Experiment
name = "cadCAD"
version = "0.5.3"
experiment = Experiment()
configs = experiment.configs
if os.name == "nt":
dill.settings["recurse"] = True
logo = r"""
___________ ____
________ __ ___/ / ____/ | / __ \
/ ___/ __` / __ / / / /| | / / / /
/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /
\___/\__,_/\__,_/\____/_/ |_/_____/
by cadCAD
"""
================================================
FILE: cadCAD/configuration/__init__.py
================================================
from typing import Dict, Callable, List, Tuple
from pandas.core.frame import DataFrame # type: ignore
from datetime import datetime
from collections import deque
from copy import deepcopy
import pandas as pd # type: ignore
from cadCAD.utils import key_filter
from cadCAD.configuration.utils import exo_update_per_ts, configs_as_objs
from cadCAD.configuration.utils.depreciationHandler import sanitize_partial_state_updates, sanitize_config
class Configuration():
def __init__(self, user_id, model_id, subset_id, subset_window, sim_config={}, initial_state={}, seeds={}, env_processes={},
exogenous_states={}, partial_state_update_blocks={}, policy_ops=[lambda a, b: a + b],
session_id=0, simulation_id=0, run_id=1, experiment_id=0, exp_window=deque([0, None], 2),
exp_creation_ts=None, **kwargs
) -> None:
self.sim_config = sim_config
self.initial_state = initial_state
self.seeds = seeds
self.env_processes = env_processes
self.exogenous_states = exogenous_states
self.partial_state_update_blocks = partial_state_update_blocks
self.policy_ops = policy_ops
self.kwargs = kwargs
self.session_id = session_id # essentially config id
self.experiment_id = experiment_id
self.user_id = user_id
self.model_id = model_id
self.exp_creation_ts = exp_creation_ts
self.labeled_jobs = {}
self.simulation_id = simulation_id
self.subset_id = subset_id
self.run_id = run_id
self.exp_window = exp_window
self.subset_window = subset_window
sanitize_config(self)
class Experiment(object):
def __init__(self):
self.exp_creation_ts = str(datetime.utcnow())
self.configs = []
self.sys_configs = []
self.model_job_map, self.model_job_counts = {}, {}
self.model_ids = list(self.model_job_map.keys())
self.model_id_queue = []
self.exp_id = 0
self.simulation_id = -1
self.subset_id = 0
self.exp_window = deque([self.exp_id, None], 2)
self.subset_window = deque([self.subset_id, None], 2)
def append_model(
self,
user_id='cadCAD_user',
model_id=None,
sim_configs={}, initial_state={}, seeds={}, raw_exogenous_states={}, env_processes={},
partial_state_update_blocks={}, policy_ops=[lambda a, b: a + b], _exo_update_per_ts: bool = True, **kwargs
# config_list=deepcopy(global_configs)
) -> None:
_sim_configs = deepcopy(sim_configs)
# self.configs = config_list
self.simulation_id += 1
try:
max_runs = _sim_configs[0]['N']
except KeyError:
max_runs = _sim_configs['N']
if _exo_update_per_ts is True:
exogenous_states = exo_update_per_ts(raw_exogenous_states)
else:
exogenous_states = raw_exogenous_states
if isinstance(_sim_configs, dict):
_sim_configs = [_sim_configs]
sim_cnt_local = 0
new_sim_configs = []
for subset_id, t in enumerate(list(zip(_sim_configs, list(range(len(_sim_configs)))))):
sim_config = t[0]
sim_config['subset_id'] = subset_id
sim_config['subset_window'] = self.subset_window
N: int = sim_config['N']
for n in range(0, N):
sim_config['simulation_id'] = self.simulation_id
sim_config['run_id'] = n
sim_config['N'] = 1
new_sim_configs.append((sim_config.copy()))
sim_cnt_local += 1
if model_id == None:
new_model_id = str(len(self.model_ids))
if new_model_id in self.model_ids:
model_id = f"model@{len(self.model_ids)}"
else:
model_id = str(new_model_id)
elif model_id != None:
if model_id in self.model_ids:
model_id = f"{model_id}@{len(self.model_ids)}"
else:
model_id = str(model_id)
run_id = 0
new_model_ids, new_configs = [], []
for sim_config in new_sim_configs:
subset_id = sim_config['subset_id']
sim_config['N'] = run_id + 1
if max_runs == 1:
sim_config['run_id'] = run_id
elif max_runs >= 1:
if run_id >= max_runs:
sim_config['N'] = run_id - (max_runs - 1)
self.exp_window = self.exp_window.copy()
config = Configuration(
exp_creation_ts=self.exp_creation_ts,
sim_config=sim_config,
initial_state=deepcopy(initial_state),
seeds=seeds,
exogenous_states=exogenous_states,
env_processes=env_processes,
partial_state_update_blocks=partial_state_update_blocks,
policy_ops=policy_ops,
# session_id=session_id,
user_id=user_id,
model_id=model_id,
session_id=f"{user_id}={sim_config['simulation_id']}_{sim_config['run_id']}",
experiment_id=self.exp_id,
simulation_id=self.simulation_id,
subset_id=subset_id,
run_id=sim_config['run_id'],
exp_window=self.exp_window,
subset_window=self.subset_window
)
# self.configs.append(config)
new_configs.append(config)
new_model_ids.append(model_id)
run_id += 1
self.configs += new_configs
self.model_id_queue += new_model_ids
self.exp_id += 1
self.exp_window.appendleft(self.exp_id)
self.sys_configs += configs_as_objs(new_configs)
unique_new_model_ids = list(set(new_model_ids))
new_model_job_list = [(model_id, []) for model_id in unique_new_model_ids]
for model_id, v in new_model_job_list:
if model_id not in self.model_ids:
self.model_job_map[model_id] = v
self.model_ids.append(model_id)
else:
except_str = f"""
Error: Duplicate model_id in Experiment - \'{model_id}\' in {self.model_ids}
-- Specify unique model_id for each use of `.append_model` per `Experiment()`
"""
raise Exception(except_str)
for model_id, job in list(zip(new_model_ids, new_configs)):
self.model_job_map[model_id].append(job)
self.model_job_counts = dict([(k, len(v)) for k, v in self.model_job_map.items()])
append_configs = append_model
class Identity:
def __init__(self, policy_id: Dict[str, int] = {'identity': 0}) -> None:
self.beh_id_return_val = policy_id
def p_identity(self, var_dict, sub_step, sL, s, **kwargs):
return self.beh_id_return_val
def policy_identity(self, k: str) -> callable:
return self.p_identity
def no_state_identity(self, var_dict, sub_step, sL, s, _input, **kwargs):
return None
def state_identity(self, k: str) -> callable:
return lambda var_dict, sub_step, sL, s, _input, **kwargs: (k, s[k])
# state_identity = cloudpickle.dumps(state_identity)
def apply_identity_funcs(self,
identity: callable,
df: DataFrame,
cols: List[str]) -> DataFrame:
"""
Apply the identity on each df column, using its self value as the
argument.
"""
fill_values = {col: identity(col) for col in cols}
filled_df = df.fillna(fill_values)
return filled_df
class Processor:
def __init__(self, id: Identity = Identity()) -> None:
self.id = id
self.p_identity = id.p_identity
self.policy_identity = id.policy_identity
self.no_state_identity = id.no_state_identity
self.state_identity = id.state_identity
self.apply_identity_funcs = id.apply_identity_funcs
def create_matrix_field(self, partial_state_updates, key: str) -> DataFrame:
if key == 'variables':
identity = self.state_identity
elif key == 'policies':
identity = self.policy_identity
df = pd.DataFrame(key_filter(partial_state_updates, key))
filled_df = self.apply_identity_funcs(identity, df, list(df.columns))
if len(filled_df) > 0:
return filled_df
else:
return pd.DataFrame({'empty': []})
def generate_config(self, initial_state, partial_state_updates, exo_proc
) -> List[tuple[list[callable], List[callable]]]:
def no_update_handler(bdf, sdf):
if (bdf.empty == False) and (sdf.empty == True):
bdf_values = bdf.values.tolist()
sdf_values = [[self.no_state_identity] * len(bdf_values) for m in range(len(partial_state_updates))]
return sdf_values, bdf_values
elif (bdf.empty == True) and (sdf.empty == False):
sdf_values = sdf.values.tolist()
bdf_values = [[self.p_identity] * len(sdf_values) for m in range(len(partial_state_updates))]
return sdf_values, bdf_values
else:
sdf_values = sdf.values.tolist()
bdf_values = bdf.values.tolist()
return sdf_values, bdf_values
def only_ep_handler(state_dict):
sdf_functions = [
lambda var_dict, sub_step, sL, s, _input, **kwargs: (k, v) for k, v in zip(state_dict.keys(), state_dict.values())
]
sdf_values = [sdf_functions]
bdf_values = [[self.p_identity] * len(sdf_values)]
return sdf_values, bdf_values
if len(partial_state_updates) != 0:
# backwards compatibility
partial_state_updates = sanitize_partial_state_updates(partial_state_updates)
bdf = self.create_matrix_field(partial_state_updates, 'policies')
sdf = self.create_matrix_field(partial_state_updates, 'variables')
sdf_values, bdf_values = no_update_handler(bdf, sdf)
zipped_list = list(zip(sdf_values, bdf_values))
else:
sdf_values, bdf_values = only_ep_handler(initial_state)
zipped_list = list(zip(sdf_values, bdf_values))
return list(map(lambda x: (x[0] + exo_proc, x[1]), zipped_list))
================================================
FILE: cadCAD/configuration/utils/__init__.py
================================================
import pandas as pd # type: ignore
from datetime import datetime, timedelta
from collections import Counter
from functools import reduce
from funcy import curry # type: ignore
from cadCAD.types import *
from typing import Union, Dict, List
from cadCAD.configuration.utils.depreciationHandler import sanitize_partial_state_updates
from cadCAD.utils import dict_filter, contains_type, flatten_tabulated_dict, tabulate_dict
class TensorFieldReport:
def __init__(self, config_proc):
self.config_proc = config_proc
def create_tensor_field(self, partial_state_updates, exo_proc, keys=['policies', 'variables']):
partial_state_updates = sanitize_partial_state_updates(partial_state_updates) # Temporary
dfs = [self.config_proc.create_matrix_field(partial_state_updates, k) for k in keys]
df = pd.concat(dfs, axis=1)
for es, i in zip(exo_proc, range(len(exo_proc))):
df['es' + str(i + 1)] = es
df['m'] = df.index + 1
return df
def configs_as_spec(configs):
sim_ids = list(map(lambda x: x.simulation_id, configs))
# sim_ids = [config.simulation_id for config in configs]
sim_id_counts = list(Counter(sim_ids).values())
IDed_configs = list(zip(sim_ids, configs))
del sim_ids
selected_IDed_configs = dict(IDed_configs)
del IDed_configs
counted_IDs_configs = list(zip(sim_id_counts, selected_IDed_configs.values()))
del sim_id_counts, selected_IDed_configs
for runs, config in counted_IDs_configs:
config.sim_config['N'] = runs
return counted_IDs_configs
def configs_as_objs(configs):
counted_IDs_configs = configs_as_spec(configs)
new_configs = list(map(lambda x: x[1], counted_IDs_configs))
del counted_IDs_configs
return new_configs
def configs_as_dicts(configs):
counted_IDs_configs = configs_as_spec(configs)
new_configs = list(map(lambda x: x[1].__dict__, counted_IDs_configs))
del counted_IDs_configs
return new_configs
def configs_as_dataframe(configs):
new_configs = configs_as_dicts(configs)
configs_df = pd.DataFrame(new_configs)
del new_configs
configs_df_columns = list(configs_df.columns)
header_cols = ['session_id', 'user_id', 'simulation_id', 'run_id']
for col in header_cols:
configs_df_columns.remove(col)
return configs_df[header_cols + configs_df_columns]
## System Model
def state_update(y, x):
return lambda var_dict, sub_step, sL, s, _input, **kwargs: (y, x)
def policy(y, x):
return lambda _g, step, sL, s, **kwargs: {y: x}
def bound_norm_random(rng, low, high):
res = rng.normal((high+low)/2, (high-low)/6)
if res < low or res > high:
res = bound_norm_random(rng, low, high)
# return Decimal(res)
return float(res)
tstep_delta = timedelta(days=0, minutes=0, seconds=30)
def time_step(dt_str, dt_format='%Y-%m-%d %H:%M:%S', _timedelta = tstep_delta):
dt = datetime.strptime(dt_str, dt_format)
t = dt + _timedelta
return t.strftime(dt_format)
ep_t_delta = timedelta(days=0, minutes=0, seconds=1)
def ep_time_step(s_condition, dt_str, fromat_str='%Y-%m-%d %H:%M:%S', _timedelta = ep_t_delta):
if s_condition:
return time_step(dt_str, fromat_str, _timedelta)
else:
return dt_str
def exo_update_per_ts(ep):
# @curried
def ep_decorator(f, y, var_dict, sub_step, sL, s, _input, **kwargs):
if s['substep'] + 1 == 1:
return f(var_dict, sub_step, sL, s, _input, **kwargs)
else:
return y, s[y]
return {es: ep_decorator(f, es) for es, f in ep.items()} # type: ignore
def trigger_condition(s, pre_conditions, cond_opp):
condition_bools = [s[field] in precondition_values for field, precondition_values in pre_conditions.items()]
return reduce(cond_opp, condition_bools)
def apply_state_condition(pre_conditions, cond_opp, y, f, _g, step, sL, s, _input, **kwargs):
def state_scope_tuner(f):
lenf = f.__code__.co_argcount
if lenf == 5:
return f(_g, step, sL, s, _input)
elif lenf == 6:
return f(_g, step, sL, s, _input, **kwargs)
if trigger_condition(s, pre_conditions, cond_opp):
return state_scope_tuner(f)
else:
return y, s[y]
def var_trigger(y, f, pre_conditions, cond_op):
return lambda _g, step, sL, s, _input, **kwargs: \
apply_state_condition(pre_conditions, cond_op, y, f, _g, step, sL, s, _input, **kwargs)
def var_substep_trigger(substeps):
def trigger(end_substep, y, f):
pre_conditions = {'substep': substeps}
cond_opp = lambda a, b: a and b
return var_trigger(y, f, pre_conditions, cond_opp)
return lambda y, f: curry(trigger)(substeps)(y)(f)
def env_trigger(end_substep):
def trigger(end_substep, trigger_field, trigger_vals, funct_list):
def env_update(state_dict, sweep_dict, target_value):
state_dict_copy = state_dict.copy()
# Use supstep to simulate current sysMetrics
if state_dict_copy['substep'] == end_substep:
state_dict_copy['timestep'] = state_dict_copy['timestep'] + 1
if state_dict_copy[trigger_field] in trigger_vals:
for g in funct_list:
target_value = g(sweep_dict, target_value)
del state_dict_copy
return target_value
return env_update
return lambda trigger_field, trigger_vals, funct_list: \
curry(trigger)(end_substep)(trigger_field)(trigger_vals)(funct_list)
def config_sim(config_dict: ConfigurationDict):
if "N" in config_dict:
if config_dict["N"] <= 0:
raise ValueError("'N' must be > 0")
else:
pass
else:
raise KeyError("The 'sim_configs' dictionary must contain the key 'N' (# of Monte Carlo Runs)")
if "T" not in config_dict:
raise KeyError("The 'sim_configs' dictionary must contain the key 'T' (Timestep Iterator)")
else:
if "M" in config_dict:
params = config_dict['M']
param_values_length = {key: len(value) if type(value) == list else 0
for key, value in params.items()}
param_values_length_set = set(param_values_length.values())
distinct_param_value_lengths = len(param_values_length_set)
if distinct_param_value_lengths > 2:
raise Exception('When sweeping, `M` list lengths should either be 1 and/or equal. More than two distinct lengths are not allowed')
elif (distinct_param_value_lengths == 1) and (0 in param_values_length_set):
return config_dict
elif (distinct_param_value_lengths == 1) or (1 in param_values_length_set):
return [{**config_dict, "M": M}
for M in flatten_tabulated_dict(tabulate_dict(params))]
else:
raise Exception('When sweeping, `M` list lengths should either be 1 and/or equal. ')
else:
config_dict["M"] = [{}]
return config_dict
def psub_list(psu_block, psu_steps):
return [psu_block[psu] for psu in psu_steps]
def psub(policies, state_updates):
return {
'policies': policies,
'states': state_updates
}
def genereate_psubs(policy_grid, states_grid, policies, state_updates):
PSUBS = []
for policy_ids, state_list in zip(policy_grid, states_grid):
filtered_policies = {k: v for (k, v) in policies.items() if k in policy_ids}
filtered_state_updates = {k: v for (k, v) in state_updates.items() if k in state_list}
PSUBS.append(psub(filtered_policies, filtered_state_updates))
return PSUBS
def access_block(state_history, target_field, psu_block_offset, exculsion_list=[]):
exculsion_list += [target_field]
def filter_history(key_list, block):
filter = lambda key_list: \
lambda d: {k: v for k, v in d.items() if k not in key_list}
return list(map(filter(key_list), block))
if psu_block_offset < -1:
if len(state_history) >= abs(psu_block_offset):
return filter_history(exculsion_list, state_history[psu_block_offset])
else:
return []
elif psu_block_offset == -1:
return filter_history(exculsion_list, state_history[psu_block_offset])
else:
return []
## Parameter Sweep
def partial_state_sweep_filter(state_field, partial_state_updates):
partial_state_dict = dict([(k, v[state_field]) for k, v in partial_state_updates.items()])
return dict([
(k, dict_filter(v, lambda v: isinstance(v, list))) for k, v in partial_state_dict.items()
if contains_type(list(v.values()), list)
])
def state_sweep_filter(raw_exogenous_states):
return dict([(k, v) for k, v in raw_exogenous_states.items() if isinstance(v, list)])
# @curried
def sweep_partial_states(_type, in_config):
configs = []
# filtered_mech_states
filtered_partial_states = partial_state_sweep_filter(_type, in_config.partial_state_update_blocks)
if len(filtered_partial_states) > 0:
for partial_state, state_dict in filtered_partial_states.items():
for state, state_funcs in state_dict.items():
for f in state_funcs:
config = in_config.copy()
config.partial_state_updates[partial_state][_type][state] = f
configs.append(config)
del config
else:
configs = [in_config]
return configs
# @curried
def sweep_states(state_type, states, in_config):
configs = []
filtered_states = state_sweep_filter(states)
if len(filtered_states) > 0:
for state, state_funcs in filtered_states.items():
for f in state_funcs:
config = in_config.copy()
exploded_states = states.copy()
exploded_states[state] = f
if state_type == 'exogenous':
config.exogenous_states = exploded_states
elif state_type == 'environmental':
config.env_processes = exploded_states
configs.append(config)
del config, exploded_states
else:
configs = [in_config]
return configs
================================================
FILE: cadCAD/configuration/utils/depreciationHandler.py
================================================
from copy import deepcopy
def sanitize_config(config):
for key, value in config.kwargs.items():
if key == 'state_dict':
config.initial_state = value
elif key == 'seed':
config.seeds = value
elif key == 'mechanisms':
config.partial_state_update_blocks = value
if config.initial_state == {}:
raise Exception('The initial conditions of the system have not been set')
def sanitize_partial_state_updates(partial_state_updates):
new_partial_state_updates = partial_state_updates.copy()
def rename_keys(d):
if 'behaviors' in d:
d['policies'] = d.pop('behaviors')
if 'states' in d:
d['variables'] = d.pop('states')
if isinstance(new_partial_state_updates, list):
for v in new_partial_state_updates:
rename_keys(v)
elif isinstance(new_partial_state_updates, dict):
for k, v in new_partial_state_updates.items():
rename_keys(v)
del partial_state_updates
return new_partial_state_updates
================================================
FILE: cadCAD/configuration/utils/policyAggregation.py
================================================
def get_base_value(x):
if isinstance(x, str):
return ''
elif isinstance(x, int):
return 0
elif isinstance(x, list):
return []
else:
return 0
def policy_to_dict(v):
return dict(list(zip(map(lambda n: 'p' + str(n + 1), list(range(len(v)))), v)))
add = lambda a, b: a + b
# @curried
# def foldr_dict_vals(f, d):
# return foldr(f)(list(d.values()))
#
#
# def sum_dict_values():
# return foldr_dict_vals(add)
# @curried
# def dict_op(f, d1, d2):
# def set_base_value(target_dict, source_dict, key):
# if key not in target_dict:
# return get_base_value(source_dict[key])
# else:
# return target_dict[key]
#
# key_set = set(list(d1.keys()) + list(d2.keys()))
#
# return {k: f(set_base_value(d1, d2, k), set_base_value(d2, d1, k)) for k in key_set}
#
# def dict_elemwise_sum():
# return dict_op(add)
================================================
FILE: cadCAD/configuration/utils/userDefinedObject.py
================================================
from collections import namedtuple
from inspect import getmembers, ismethod
from pandas.core.frame import DataFrame
from cadCAD.utils import SilentDF
def val_switch(v):
if isinstance(v, DataFrame) is True:
return SilentDF(v)
else:
return v
class udcView(object):
def __init__(self, d, masked_members):
self.__dict__ = d
self.masked_members = masked_members
def __repr__(self):
members = {}
variables = {
k: val_switch(v) for k, v in self.__dict__.items()
if str(type(v)) != "<class 'method'>" and k not in self.masked_members # and isinstance(v, DataFrame) is not True
}
members['methods'] = [k for k, v in self.__dict__.items() if str(type(v)) == "<class 'method'>"]
members.update(variables)
return f"{members}"
class udcBroker(object):
def __init__(self, obj, function_filter=['__init__']):
d = {}
funcs = dict(getmembers(obj, ismethod))
filtered_functions = {k: v for k, v in funcs.items() if k not in function_filter}
d['obj'] = obj
d.update(vars(obj)) # somehow is enough
d.update(filtered_functions)
self.members_dict = d
def get_members(self):
return self.members_dict
def get_view(self, masked_members):
return udcView(self.members_dict, masked_members)
def get_namedtuple(self):
return namedtuple("Hydra", self.members_dict.keys())(*self.members_dict.values())
def UDO(udo, masked_members=['obj']):
return udcBroker(udo).get_view(masked_members)
def udoPipe(obj_view):
return UDO(obj_view.obj, obj_view.masked_members)
================================================
FILE: cadCAD/diagram/__init__.py
================================================
from cadCAD.diagram.config_diagram import diagram, diagram_from_config
================================================
FILE: cadCAD/diagram/config_diagram.py
================================================
from graphviz import Digraph
import inspect
import re
### Inspect functions
def extract_var_key(raw_line: str, var_id: str) -> str:
"""
Extract the key from an line in the form "dict['key']" or
"dict.get('key', *args)".
"""
line = raw_line.strip()[len(var_id) :]
state_var = ""
if line[0] == "[":
state_var = line[2:-2]
elif line[0:4] == ".get":
call = line.split("(")[1]
call = call.split(")")[0]
call = call.strip()
sep = "'"
if call[0] == '"':
sep = '"'
state_var = [el for el in call.split(sep) if len(el) > 0][0]
return state_var
def extract_vars_from_source(source: str, var_id: str) -> set:
"""
Extract keys from an source code that consumes an dict with
var_id name.
"""
regex = (
r"(("
+ var_id
+ r"\[(\'|\")\w+(\'|\")\])|("
+ var_id
+ r".get\((\'|\")\w+(\'|\")[A-z,\s\"\']*\)))"
)
matches = re.findall(regex, source)
lines = [match[0] for match in matches]
state_vars = set([extract_var_key(line, var_id) for line in lines])
return state_vars
def extract_keys(f: callable) -> dict:
"""
"""
src = inspect.getsource(f)
params = inspect.signature(f)
params_key = list(params.parameters)[0]
state_key = list(params.parameters)[3]
output = {
"state": extract_vars_from_source(src, state_key),
"params": extract_vars_from_source(src, params_key),
}
return output
def relate_psub(psub: dict) -> dict:
"""
Given an dict describing an Partial State Update block, this functions
generates an dict with three keys: 'params' and 'state' which are sets
containing all unique parameters and state variables for the PSUB,
and 'map' for doing an more detailed map.
"""
psub_relation = {"map": {}, "params": set(), "state": set()}
unique_params = set()
unique_vars = set()
keys = ["policies", "variables"]
for key in keys:
type_functions = psub.get(key, {})
type_keys = {k: extract_keys(v) for k, v in type_functions.items()}
params_list = [v.get("params", set()) for v in type_keys.values()]
vars_list = [v.get("state", set()) for v in type_keys.values()]
if len(params_list) > 0:
params = set.union(*params_list)
else:
params = set()
if len(vars_list) > 0:
vars = set.union(*vars_list)
else:
vars = set()
psub_relation["params"] = psub_relation["params"].union(params)
psub_relation["state"] = psub_relation["state"].union(vars)
psub_relation["map"][key] = type_keys
return psub_relation
def generate_relations(psubs) -> list:
"""
Generates an list of dicts,
"""
psub_relations = [relate_psub(psub) for psub in psubs]
return psub_relations
### Diagram functions
def generate_time_graph() -> Digraph:
time_graph = Digraph("cluster_timestep", engine="dot")
time_graph.attr(style="filled", bgcolor="pink", dpi="50", rankdir="LR")
return time_graph
def generate_variables_cluster(variables: dict, i: int, suffix="") -> Digraph:
state_graph = Digraph("cluster_variables_{}{}".format(i, suffix))
state_graph.attr(style="filled, dashed", label="State", fillcolor="skyblue")
for key, value in variables.items():
name = "variable_{}_{}{}".format(key, i, suffix)
description = "{} ({})".format(key, type(value).__name__)
state_graph.node(
name,
description,
shape="cylinder",
style="filled, solid",
fillcolor="honeydew",
),
return state_graph
def generate_params_cluster(params: dict, i: int) -> Digraph:
params_graph = Digraph("cluster_params_{}".format(i))
params_graph.attr(style="filled, dashed", label="Parameters", fillcolor="skyblue")
for key, value in params.items():
name = "param_{}_{}".format(key, i)
description = "{} ({})".format(key, type(value).__name__)
params_graph.node(
name,
description,
shape="cylinder",
style="filled, solid",
fillcolor="honeydew",
),
return params_graph
def generate_psub_graph(i: int):
psub_graph = Digraph("cluster_psub_{}".format(i))
psub_graph.attr(
style="filled, dashed",
label=f"Partial State Update Block #{i}",
fillcolor="thistle",
center="true",
)
return psub_graph
def relate_params(graph: Digraph, params, i, origin=-1) -> Digraph:
for param in params:
dst = "param_{}_{}".format(param, i)
src = "param_{}_{}".format(param, origin)
graph.edge(src, dst)
return graph
def generate_policies_cluster(policies: dict, i: int, psub_graph) -> Digraph:
policy_graph = Digraph("cluster_policy_{}".format(i))
policy_graph.attr(label="Policies")
policy_graph.node(
"agg_{}".format(i),
"Aggregation",
shape="circle",
style="filled,bold",
fillcolor="greenyellow",
width="1",
)
for key, value in policies.items():
name = "policy_{}_{}".format(key, i)
description = "{} ({})".format(key, value.__name__)
policy_graph.node(
name,
description,
style="filled, bold",
fillcolor="palegreen",
shape="cds",
height="1",
width="1",
)
psub_graph.edge(name, "agg_{}".format(i))
return psub_graph.subgraph(policy_graph)
def relate(
graph, relations, i, src_prefix, dst_prefix, suffix="", reverse=False
) -> Digraph:
for key, value in relations.items():
dst = "{}_{}_{}".format(dst_prefix, key, i)
for param in value:
src = "{}_{}_{}{}".format(src_prefix, param, i, suffix)
if reverse:
graph.edge(dst, src)
else:
graph.edge(src, dst)
return graph
def generate_sufs_cluster(sufs: dict, i: int, psub_graph, agg=False) -> Digraph:
suf_graph = Digraph("cluster_suf_{}".format(i))
suf_graph.attr(label="State Update Functions")
for key, value in sufs.items():
name = "suf_{}_{}".format(key, i)
description = "{} ({})".format(key, value.__name__)
suf_graph.node(
name,
description,
style="filled, bold",
fillcolor="red",
shape="cds",
height="1",
width="1",
)
if agg:
psub_graph.edge("agg_{}".format(i), name)
return psub_graph.subgraph(suf_graph)
def relate_params_to_sufs(graph, sufs, i) -> Digraph:
for key, value in sufs.items():
dst = "suf_{}_{}".format(key, i)
for param in value:
src = "param_{}_{}".format(param, i)
graph.edge(src, dst)
return graph
def diagram(initial_state, params, psubs):
"""
Generates an diagram for an cadCAD configuration object.
"""
relations = generate_relations(psubs)
time_graph = generate_time_graph()
for i_psub, psub in enumerate(psubs):
psub_graph = generate_psub_graph(i_psub)
# Parameters
psub_params = relations[i_psub].get("params", set())
psub_params = {k: params.get(k, None) for k in psub_params}
psub_vars = relations[i_psub].get("state", set())
psub_vars = {k: initial_state.get(k, None) for k in psub_vars}
psub_graph.subgraph(generate_params_cluster(psub_params, i_psub))
psub_graph.subgraph(generate_variables_cluster(psub_vars, i_psub))
# psub_graph = relate_params(psub_graph, psub_params, i)
# Policies
policies = psub.get("policies", {})
psub_map = relations[i_psub].get("map", {})
policy_map = psub_map.get("policies", {})
policy_inputs = {
policy: relation["state"] for policy, relation in policy_map.items()
}
policy_params = {
policy: relation["params"] for policy, relation in policy_map.items()
}
list_of_inputs = list(policy_inputs.values())
if len(list_of_inputs) > 0:
agg = True
inputs = set.union(*list_of_inputs)
inputs = {k: initial_state.get(k, None) for k in inputs}
psub_graph.subgraph(generate_policies_cluster(policies, i_psub, psub_graph))
psub_graph = relate(psub_graph, policy_params, i_psub, "param", "policy")
psub_graph = relate(psub_graph, policy_inputs, i_psub, "variable", "policy")
else:
agg = False
# SUFs
sufs = psub.get("variables", {})
suf_map = psub_map.get("variables", {})
sufs_inputs = {
policy: relation["state"] for policy, relation in suf_map.items()
}
sufs_params = {
policy: relation["params"] for policy, relation in suf_map.items()
}
list_of_inputs = list(sufs_inputs.values())
if len(list_of_inputs) > 0:
inputs = set.union(*list_of_inputs)
inputs = {k: initial_state.get(k, None) for k in inputs}
generate_sufs_cluster(sufs, i_psub, psub_graph, agg=agg)
psub_graph = relate(psub_graph, sufs_params, i_psub, "param", "suf")
psub_graph = relate(psub_graph, sufs_inputs, i_psub, "variable", "suf")
time_graph.subgraph(psub_graph)
return time_graph
def diagram_from_config(config):
initial_state = config.initial_state
params = config.sim_config["M"]
psubs = config.partial_state_update_blocks
return diagram(initial_state, params, psubs)
================================================
FILE: cadCAD/engine/__init__.py
================================================
from time import time
from typing import Callable, Dict, List, Any, Tuple, Union, Sequence, Mapping
from tqdm.auto import tqdm
from cadCAD.utils import flatten
from cadCAD.utils.execution import print_exec_info
from cadCAD.configuration import Configuration, Processor
from cadCAD.configuration.utils import TensorFieldReport, configs_as_objs, configs_as_dicts
from cadCAD.engine.simulation import Executor as SimExecutor
from cadCAD.engine.execution import single_proc_exec, parallelize_simulations, local_simulations
from cadCAD.types import *
VarDictType = Dict[str, List[object]]
StatesListsType = List[dict[str, object]]
ConfigsType = List[tuple[list[callable], List[callable]]]
EnvProcessesType = Dict[str, callable]
class ExecutionMode:
local_mode = 'local_proc'
multi_mode = 'multi_proc'
distributed = 'dist_proc'
single_mode = 'single_proc'
# Backwards compatible modes below
single_proc = 'single_proc'
multi_proc = 'multi_proc'
def auto_mode_switcher(config_amt: int):
try:
if config_amt == 1:
return ExecutionMode.single_mode, single_proc_exec
elif (config_amt > 1):
return ExecutionMode.multi_mode, parallelize_simulations
except AttributeError:
if config_amt < 1:
raise ValueError('N must be >= 1!')
class ExecutionContext:
def __init__(self, context=ExecutionMode.local_mode, method=None, additional_objs=None) -> None:
self.name = context
self.additional_objs = additional_objs
if context == 'local_proc':
self.method = local_simulations
elif context == 'single_proc':
self.method = single_proc_exec
elif context == 'multi_proc':
self.method = parallelize_simulations
elif context == 'dist_proc':
def distroduce_proc(
simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, RunIDs,
ExpIDs,
SubsetIDs,
SubsetWindows,
configured_n, # exec_method,
sc, additional_objs=additional_objs
):
return method(
simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, RunIDs,
ExpIDs,
SubsetIDs,
SubsetWindows,
configured_n, # exec_method,
sc, additional_objs
)
self.method = distroduce_proc
class Executor:
def __init__(self,
exec_context: ExecutionContext, configs: List[Configuration], sc=None, empty_return=False, supress_print=False
) -> None:
self.sc = sc
self.SimExecutor = SimExecutor
self.exec_method = exec_context.method
self.exec_context = exec_context.name
self.additional_objs = exec_context.additional_objs
self.configs = configs
self.empty_return = empty_return
self.supress_print = supress_print
def execute(self) -> Tuple[object, object, Dict[str, object]]:
if self.empty_return is True:
return [], [], []
config_proc = Processor()
create_tensor_field = TensorFieldReport(
config_proc).create_tensor_field
sessions = []
var_dict_list, states_lists = [], []
Ts, Ns, SimIDs, RunIDs = [], [], [], []
ExpIDs, ExpWindows, SubsetIDs, SubsetWindows = [], [], [], []
eps, configs_structs, env_processes_list = [], [], []
partial_state_updates, sim_executors = [], []
config_idx = 0
# Execution Info
if self.supress_print is False:
print_exec_info(self.exec_context, configs_as_objs(self.configs))
t1 = time()
for x in tqdm(self.configs,
total=len(self.configs),
desc="Initializing configurations",
disable=self.supress_print):
sessions.append(
{
'user_id': x.user_id, 'experiment_id': x.experiment_id, 'session_id': x.session_id,
'simulation_id': x.simulation_id, 'run_id': x.run_id,
'subset_id': x.subset_id, 'subset_window': x.subset_window
}
)
Ts.append(x.sim_config['T'])
Ns.append(x.sim_config['N'])
ExpIDs.append(x.experiment_id)
ExpWindows.append(x.exp_window)
SimIDs.append(x.simulation_id)
SubsetIDs.append(x.subset_id)
RunIDs.append(x.run_id)
SubsetWindows.append(x.subset_window)
var_dict_list.append(x.sim_config['M'])
states_lists.append([x.initial_state])
eps.append(list(x.exogenous_states.values()))
configs_structs.append(config_proc.generate_config(
x.initial_state, x.partial_state_update_blocks, eps[config_idx]))
env_processes_list.append(x.env_processes)
partial_state_updates.append(x.partial_state_update_blocks)
sim_executors.append(SimExecutor(x.policy_ops).simulation)
config_idx += 1
remote_threshold = 100
config_amt = len(self.configs)
def get_final_dist_results(simulations: List[StateHistory],
psus: List[StateUpdateBlocks],
eps,
sessions: List[SessionDict]):
tensor_fields = [create_tensor_field(
psu, ep) for psu, ep in list(zip(psus, eps))]
return simulations, tensor_fields, sessions
def get_final_results(simulations: List[StateHistory],
psus: List[StateUpdateBlocks],
eps,
sessions: List[SessionDict],
remote_threshold: int):
# if list of lists of lists of dicts: do flatten
# if list of dicts: do not flatetn
# else raise error
init: bool = isinstance(simulations, Sequence)
failed_1 = False
failed_2 = False
try:
init: bool = isinstance(simulations, Sequence)
dont_flatten = init & isinstance(simulations[0], Mapping)
do_flatten = not dont_flatten
except:
failed_1 = True
do_flatten = True
try:
do_flatten = init & isinstance(simulations[0], Sequence)
do_flatten &= isinstance(simulations[0][0], Sequence)
do_flatten &= isinstance(simulations[0][0][0], Mapping)
except:
failed_2 = True
do_flatten = False
if failed_1 and failed_2:
raise ValueError('Invalid simulation results (Executor output is not list[dict] or list[list[list[dict]]])')
flat_timesteps, tensor_fields = [], []
for sim_result, psu, ep in tqdm(list(zip(simulations, psus, eps)),
total=len(simulations),
desc='Flattening results',
disable=self.supress_print):
if do_flatten:
flat_timesteps.append(flatten(sim_result))
tensor_fields.append(create_tensor_field(psu, ep))
if do_flatten:
flat_simulations = flatten(flat_timesteps)
else:
flat_simulations = simulations
return flat_simulations, tensor_fields, sessions
final_result = None
original_N = len(configs_as_dicts(self.configs))
if self.exec_context != ExecutionMode.distributed:
# Consider Legacy Support
if self.exec_context == ExecutionMode.local_mode:
self.exec_context, self.exec_method = auto_mode_switcher(
config_amt)
elif self.exec_context == ExecutionMode.single_mode or self.exec_context == ExecutionMode.single_proc:
self.exec_context, self.exec_method = ExecutionMode.single_mode, single_proc_exec
elif self.exec_context == ExecutionMode.multi_mode or self.exec_context == ExecutionMode.multi_proc:
if config_amt == 1:
raise ValueError("Multi mode must have at least 2 configs")
else:
self.exec_context, self.exec_method = ExecutionMode.multi_mode, parallelize_simulations
else:
raise ValueError("Invalid execution mode specified")
if self.supress_print is False:
print("Execution Method: " + self.exec_method.__name__)
simulations_results = self.exec_method(
sim_executors, var_dict_list, states_lists, configs_structs, env_processes_list, Ts, SimIDs, RunIDs,
ExpIDs, SubsetIDs, SubsetWindows, original_N, self.additional_objs
)
final_result = get_final_results(
simulations_results, partial_state_updates, eps, sessions, remote_threshold)
elif self.exec_context == ExecutionMode.distributed:
if self.supress_print is False:
print("Execution Method: " + self.exec_method.__name__)
simulations_results = self.exec_method(
sim_executors, var_dict_list, states_lists, configs_structs, env_processes_list, Ts,
SimIDs, RunIDs, ExpIDs, SubsetIDs, SubsetWindows, original_N, self.sc
)
final_result = get_final_dist_results(
simulations_results, partial_state_updates, eps, sessions)
t2 = time()
if self.supress_print is False:
print(f"Total execution time: {t2 - t1 :.2f}s")
return final_result
================================================
FILE: cadCAD/engine/execution.py
================================================
from typing import Callable, Dict, List, Any, Tuple, Sequence
from pathos.multiprocessing import ProcessPool # type: ignore
from collections import Counter
from cadCAD.types import *
from cadCAD.utils import flatten
VarDictType = Dict[str, List[object]]
StatesListsType = List[dict[str, object]]
ConfigsType = List[tuple[list[callable], List[callable]]]
EnvProcessesType = Dict[str, callable]
def single_proc_exec(
simulation_execs: Sequence[ExecutorFunction],
var_dict_list: Union[Sequence[Parameters], Parameters],
states_lists: Sequence[StateHistory],
configs_structs: Sequence[StateUpdateBlocks],
env_processes_list: Sequence[EnvProcesses],
Ts: Sequence[TimeSeq],
SimIDs: Sequence[SimulationID],
Ns: Sequence[Run],
ExpIDs: Sequence[int],
SubsetIDs: Sequence[SubsetID],
SubsetWindows: Sequence[SubsetWindow],
configured_n: Sequence[N_Runs],
additional_objs=None
) -> List:
if not isinstance(var_dict_list, Sequence):
var_dict_list = list([var_dict_list])
raw_params = (
simulation_execs, states_lists, configs_structs, env_processes_list,
Ts, SimIDs, Ns, SubsetIDs, SubsetWindows, var_dict_list)
results: List = []
for raw_param in zip(*raw_params):
simulation_exec, states_list, config, env_processes, T, sim_id, N, subset_id, subset_window, var_dict = raw_param
result = simulation_exec(
var_dict, states_list, config, env_processes, T, sim_id, N, subset_id, subset_window, configured_n, additional_objs
)
results.append(flatten(result))
return flatten(results)
def parallelize_simulations(
simulation_execs: List[ExecutorFunction],
var_dict_list: List[Parameters],
states_lists: List[StateHistory],
configs_structs: List[StateUpdateBlocks],
env_processes_list: List[EnvProcesses],
Ts: List[TimeSeq],
SimIDs: List[SimulationID],
Ns: List[Run],
ExpIDs: List[int],
SubsetIDs: List[SubsetID],
SubsetWindows: List[SubsetWindow],
configured_n: List[N_Runs],
additional_objs=None
):
params = list(
zip(
simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list,
Ts, SimIDs, Ns, SubsetIDs, SubsetWindows
)
)
len_configs_structs = len(configs_structs)
unique_runs = Counter(SimIDs)
sim_count = max(unique_runs.values())
highest_divisor = int(len_configs_structs / sim_count)
new_configs_structs, new_params = [], []
for count in range(len(params)):
if count == 0:
new_params.append(
params[count: highest_divisor]
)
new_configs_structs.append(
configs_structs[count: highest_divisor]
)
elif count > 0:
new_params.append(
params[count * highest_divisor: (count + 1) * highest_divisor]
)
new_configs_structs.append(
configs_structs[count * highest_divisor: (count + 1) * highest_divisor]
)
def process_executor(params):
if len_configs_structs > 1:
with ProcessPool(processes=len_configs_structs) as pp:
results = pp.map(
lambda t: t[0](t[1], t[2], t[3], t[4], t[5], t[6], t[7], t[8], t[9], configured_n), params
)
else:
t = params[0]
results = t[0](t[1], t[2], t[3], t[4], t[5], t[6], t[7], t[8], t[9], configured_n)
return results
results = flatten(list(map(lambda params: process_executor(params), new_params)))
return results
def local_simulations(
simulation_execs: List[ExecutorFunction],
var_dict_list: List[Parameters],
states_lists: List[StateHistory],
configs_structs: List[StateUpdateBlocks],
env_processes_list: List[EnvProcesses],
Ts: List[TimeSeq],
SimIDs: List[SimulationID],
Ns: List[Run],
ExpIDs: List[int],
SubsetIDs: List[SubsetID],
SubsetWindows: List[SubsetWindow],
configured_n: List[N_Runs],
additional_objs=None
):
config_amt = len(configs_structs)
if config_amt == 1: # and configured_n != 1
return single_proc_exec(
simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list,
Ts, SimIDs, Ns, ExpIDs, SubsetIDs, SubsetWindows, configured_n, additional_objs
)
elif config_amt > 1: # and configured_n != 1
return parallelize_simulations(
simulation_execs, var_dict_list, states_lists, configs_structs, env_processes_list,
Ts, SimIDs, Ns, ExpIDs, SubsetIDs, SubsetWindows, configured_n, additional_objs
)
# elif config_amt > 1 and configured_n == 1:
================================================
FILE: cadCAD/engine/simulation.py
================================================
from typing import Any, Callable, Dict, List, Tuple
from copy import deepcopy, copy
from functools import reduce
from funcy import curry # type: ignore
from cadCAD.utils import flatten
from cadCAD.engine.utils import engine_exception
from cadCAD.types import *
id_exception: callable = curry(engine_exception)(KeyError)(KeyError)(None)
Aggregator = Callable[[object, object], object]
def policy_scope_tuner(args: tuple,
additional_objs: object,
f: PolicyFunction) -> dict:
"""
Execute cadCAD policy function.
"""
(sweep_dict, sub_step, sL, s) = args
if additional_objs is None:
return f(sweep_dict, sub_step, sL, s)
else:
return f(sweep_dict, sub_step, sL, s, additional_objs)
def compose(init_reduction_funct: Aggregator,
funct_list: List[Aggregator],
val_list: dict) -> object:
"""
Reduce the nested policy input dict into a simple one.
"""
result, i = None, 0
def composition(x): return [reduce(
init_reduction_funct, x.values())] + funct_list
for g in composition(val_list):
if i == 0:
result = g
i = 1
else:
result = g(result)
return result
class Executor:
def __init__(
self,
policy_ops,
policy_update_exception: callable = id_exception,
state_update_exception: callable = id_exception
) -> None:
self.policy_ops = policy_ops
self.state_update_exception = state_update_exception
self.policy_update_exception = policy_update_exception
def get_policy_input(
self,
sweep_dict: Dict[str, List[object]],
sub_step: int,
sL: List[dict[str, object]],
s: Dict[str, object],
funcs: List[callable],
additional_objs
) -> Dict[str, object]:
"""
Retrieves the Policy Input for usage on State Update Functions
Arguments:
sweep_dict - System Parameters
sub_step - Execution order in regards to PSUBs
sL - History of the variables state
s - Current variables state
funcs - list of cadCAD Policies to be executed
"""
ops = self.policy_ops
args = (sweep_dict, sub_step, sL, s)
def execute_policy(f: PolicyFunction) -> dict:
return policy_scope_tuner(args, additional_objs, f)
col_results: List[PolicyOutput] = map(execute_policy, funcs)
# Create a nested dict containing all results combination
# new_dict[policy_input][policy_ordinal] = policy_input_value
new_dict: dict = {}
for i, col_result in enumerate(col_results):
for label, value in col_result.items():
if label not in new_dict.keys():
new_dict[label] = {}
else:
pass
new_dict[label][i] = value
# Aggregator functions
ops_head, *ops_tail = ops
# Function for aggregating a combination of policy inputs
# for the same signal
def f(val_list):
return compose(init_reduction_funct=ops_head,
funct_list=ops_tail,
val_list=val_list)
# Generate dict to be consumed by SUFs
policy_input = {
label: f(val_list)
for label, val_list
in new_dict.items()}
return policy_input
def apply_env_proc(
self,
sweep_dict,
env_processes: Dict[str, callable],
state_dict: Dict[str, object]
) -> Dict[str, object]:
def env_composition(target_field, state_dict, target_value):
function_type = type(lambda x: x)
env_update = env_processes[target_field]
if isinstance(env_update, list):
for f in env_update:
target_value = f(sweep_dict, target_value)
elif isinstance(env_update, function_type):
target_value = env_update(state_dict, sweep_dict, target_value)
else:
target_value = env_update
return target_value
filtered_state_dict = {
k: v for k, v in state_dict.items() if k in env_processes.keys()}
env_proc_dict = {
target_field: env_composition(
target_field, state_dict, target_value)
for target_field, target_value in filtered_state_dict.items()
}
for k, v in env_proc_dict.items():
state_dict[k] = v
return state_dict
# mech_step
def partial_state_update(
self,
sweep_dict: Parameters,
sub_step: Substep,
sL: List[State],
sH: StateHistory,
state_funcs: List[StateUpdateFunction],
policy_funcs: List[PolicyFunction],
env_processes: EnvProcesses,
time_step: int,
run: int,
additional_objs
) -> List[dict[str, object]]:
if type(additional_objs) == dict:
if additional_objs.get('deepcopy_off', False) == True:
last_in_obj = copy(sL[-1])
if len(additional_objs) == 1:
additional_objs = None
# XXX: drop the additional objects if only used for deepcopy
# toggling.
else:
last_in_obj = deepcopy(sL[-1])
else:
last_in_obj = deepcopy(sL[-1])
_input: Dict[str, Any] = self.policy_update_exception(
self.get_policy_input(sweep_dict, sub_step, sH, last_in_obj, policy_funcs, additional_objs)
)
def generate_record(state_funcs):
def state_scope_tuner(f):
lenf = f.__code__.co_argcount
if lenf == 5:
return self.state_update_exception(f(sweep_dict, sub_step, sH, last_in_obj, _input))
elif lenf == 6:
return self.state_update_exception(f(sweep_dict, sub_step, sH, last_in_obj, _input, additional_objs))
for f in state_funcs:
yield state_scope_tuner(f)
def transfer_missing_fields(source, destination):
for k in source:
if k not in destination:
destination[k] = source[k]
del source
return destination
try:
new_state_vars = dict(generate_record(state_funcs))
except (ValueError, TypeError):
raise ValueError("There is a State Update Function which is not returning an proper tuple")
last_in_copy: Dict[str, Any] = transfer_missing_fields(last_in_obj, new_state_vars)
last_in_copy: Dict[str, Any] = self.apply_env_proc(sweep_dict, env_processes, last_in_copy)
last_in_copy['substep'], last_in_copy['timestep'], last_in_copy['run'] = sub_step, time_step, run
sL.append(last_in_copy)
del last_in_copy
return sL
# mech_pipeline - state_update_block
def state_update_pipeline(
self,
sweep_dict: Dict[str, List[object]],
simulation_list,
configs: List[tuple[list[callable], List[callable]]],
env_processes: Dict[str, callable],
time_step: int,
run: int,
additional_objs
) -> List[dict[str, object]]:
sub_step = 0
states_list_copy: List[dict[str, object]] = tuple(simulation_list[-1])
genesis_states: Dict[str, object] = states_list_copy[-1].copy()
# genesis_states: Dict[str, object] = states_list_copy[-1]
if len(states_list_copy) == 1:
genesis_states['substep'] = sub_step
del states_list_copy
states_list: List[dict[str, object]] = [genesis_states]
sub_step += 1
for [s_conf, p_conf] in configs:
states_list: List[dict[str, object]] = self.partial_state_update(
sweep_dict, sub_step, states_list, simulation_list, s_conf, p_conf, env_processes, time_step, run,
additional_objs
)
sub_step += 1
time_step += 1
return states_list
# state_update_pipeline
def run_pipeline(
self,
sweep_dict: Dict[str, List[object]],
states_list: List[dict[str, object]],
configs: List[tuple[list[callable], List[callable]]],
env_processes: Dict[str, callable],
time_seq: range,
run: int,
additional_objs
) -> List[list[dict[str, object]]]:
time_seq: List[int] = [x + 1 for x in time_seq]
simulation_list: List[list[dict[str, object]]] = [states_list]
for time_step in time_seq:
pipe_run: List[dict[str, object]] = self.state_update_pipeline(
sweep_dict, simulation_list, configs, env_processes, time_step, run, additional_objs
)
_, *pipe_run = pipe_run
simulation_list.append(pipe_run)
return simulation_list
def simulation(
self,
sweep_dict: SweepableParameters,
states_list: StateHistory,
configs,
env_processes: EnvProcesses,
time_seq: TimeSeq,
simulation_id: SimulationID,
run: int,
subset_id: SubsetID,
subset_window: SubsetWindow,
configured_N: int,
# remote_ind
additional_objs: Union[None, Dict]=None
):
run += 1
subset_window.appendleft(subset_id)
def execute_run(sweep_dict, states_list, configs, env_processes, time_seq, _run) -> List[dict[str, object]]:
def generate_init_sys_metrics(genesis_states_list, sim_id, _subset_id, _run, _subset_window):
for D in genesis_states_list:
d = D.copy()
d['simulation'], d['subset'], d['run'], d['substep'], d['timestep'] = \
sim_id, _subset_id, _run, 0, 0
yield d
states_list_copy: List[dict[str, object]] = list(
generate_init_sys_metrics(
tuple(states_list), simulation_id, subset_id, run, subset_window)
)
first_timestep_per_run: List[dict[str, object]] = self.run_pipeline(
sweep_dict, states_list_copy, configs, env_processes, time_seq, run, additional_objs
)
del states_list_copy
return first_timestep_per_run
pipe_run = flatten(
[execute_run(sweep_dict, states_list, configs,
env_processes, time_seq, run)]
)
return pipe_run
================================================
FILE: cadCAD/engine/utils.py
================================================
from datetime import datetime
def datetime_range(start, end, delta, dt_format='%Y-%m-%d %H:%M:%S'):
reverse_head = end
[start, end] = [datetime.strptime(x, dt_format) for x in [start, end]]
def _datetime_range(start, end, delta):
current = start
while current < end:
yield current
current += delta
reverse_tail = [dt.strftime(dt_format) for dt in _datetime_range(start, end, delta)]
return reverse_tail + [reverse_head]
def last_index(l):
return len(l)-1
def retrieve_state(l, offset):
return l[last_index(l) + offset + 1]
# @curried
def engine_exception(ErrorType, error_message, exception_function, try_function):
try:
return try_function
except ErrorType:
print(error_message)
return exception_function
================================================
FILE: cadCAD/tools/__init__.py
================================================
from cadCAD.tools.execution import easy_run
from cadCAD.tools.profiling import profile_run
from cadCAD.tools.utils import generic_suf, add_parameter_labels
================================================
FILE: cadCAD/tools/execution/__init__.py
================================================
from cadCAD.tools.execution.easy_run import easy_run
================================================
FILE: cadCAD/tools/execution/easy_run.py
================================================
import inspect
import types
from typing import Dict, Union
import pandas as pd # type: ignore
from cadCAD.configuration import Experiment
from cadCAD.configuration.utils import config_sim
from cadCAD.engine import ExecutionContext, ExecutionMode, Executor
def describe_or_return(v: object) -> object:
"""
Thanks @LinuxIsCool!
"""
if isinstance(v, types.FunctionType):
return f'function: {v.__name__}'
elif isinstance(v, types.LambdaType) and v.__name__ == '<lambda>':
return f'lambda: {inspect.signature(v)}'
elif isinstance(v, list):
return str(v)
else:
return v
def select_M_dict(M_dict: Dict[str, object], keys: set) -> Dict[str, object]:
"""
Thanks @LinuxIsCool!
"""
return {k: describe_or_return(v) for k, v in M_dict.items() if k in keys}
def select_config_M_dict(configs: list, i: int, keys: set) -> Dict[str, object]:
return select_M_dict(configs[i].sim_config['M'], keys)
def easy_run(
state_variables,
params,
psubs,
N_timesteps,
N_samples,
use_label=False,
assign_params: Union[bool, set] = True,
drop_substeps=True,
exec_mode='local',
deepcopy_off=False,
supress_print=False
) -> pd.DataFrame:
"""
Run cadCAD simulations without headaches.
"""
# Set-up sim_config
simulation_parameters = {'N': N_samples,
'T': range(N_timesteps), 'M': params}
sim_config = config_sim(simulation_parameters) # type: ignore
# Create a new experiment
exp = Experiment()
exp.append_configs(
sim_configs=sim_config,
initial_state=state_variables,
partial_state_update_blocks=psubs,
)
configs = exp.configs
# Set-up cadCAD executor
if exec_mode == 'local':
_exec_mode = ExecutionMode().local_mode
elif exec_mode == 'single':
_exec_mode = ExecutionMode().single_mode
exec_context = ExecutionContext(_exec_mode, additional_objs={'deepcopy_off': deepcopy_off})
executor = Executor(exec_context=exec_context, configs=configs, supress_print=supress_print)
# Execute the cadCAD experiment
(records, tensor_field, _) = executor.execute()
# Parse the output as a pandas DataFrame
df = pd.DataFrame(records)
if drop_substeps == True:
# Drop all intermediate substeps
first_ind = (df.substep == 0) & (df.timestep == 0)
last_ind = df.substep == max(df.substep)
inds_to_drop = first_ind | last_ind
df = df.loc[inds_to_drop].drop(columns=['substep'])
else:
pass
if assign_params == False:
pass
else:
M_dict = configs[0].sim_config['M']
params_set = set(M_dict.keys())
if assign_params == True:
pass
else:
params_set &= assign_params # type: ignore
# Logic for getting the assign params criteria
if type(assign_params) is list:
selected_params = set(assign_params) & params_set # type: ignore
elif type(assign_params) is set:
selected_params = assign_params & params_set
else:
selected_params = params_set
# Attribute parameters to each row*
params_dict = select_config_M_dict(configs, 0, selected_params)
# Handles all cases of parameter types including list
for key, value in params_dict.items():
df[key] = df.apply(lambda _: value, axis=1)
for i, (_, n_df) in enumerate(df.groupby(['simulation', 'subset', 'run'])):
params_dict = select_config_M_dict(configs, i, selected_params)
for key, value in params_dict.items():
df.loc[n_df.index, key] = df.loc[n_df.index].apply(
lambda _: value, axis=1)
# Based on Vitor Marthendal (@marthendalnunes) snippet
if use_label == True:
psub_map = {
order + 1: psub.get('label', '') for (order, psub) in enumerate(psubs)
}
psub_map[0] = 'Initial State'
df['substep_label'] = df.substep.map(psub_map)
return df
================================================
FILE: cadCAD/tools/preparation.py
================================================
from typing import Mapping
from cadCAD.tools.execution import easy_run
from pandas import DataFrame # type: ignore
from cadCAD.types import *
from cadCAD.tools.types import *
from itertools import product
from dataclasses import dataclass
def sweep_cartesian_product(sweep_params: SweepableParameters) -> SweepableParameters:
"""
Makes a cartesian product from dictionary values.
This is useful for plugging inside the sys_params dict, like:
```python
sweep_params = {'a': [0.1, 0.2], 'b': [1, 2]}
product_sweep
sys_params = {**cartesian_product_sweep(sweep_params),
'c': [0.1]}
```
Usage:
>>> sweep_params = {'a': [0.1, 0.2], 'b': [1, 2]}
>>> cartesian_product_sweep(sweep_params)
{'a': [0.1, 0.1, 0.2, 0.2], 'b': [1, 2, 1, 2]}
"""
cartesian_product = product(*sweep_params.values())
transpose_cartesian_product = zip(*cartesian_product)
zipped_sweep_params = zip(sweep_params.keys(), transpose_cartesian_product)
sweep_dict = dict(zipped_sweep_params)
sweep_dict = {k: list(v) for k, v in sweep_dict.items()}
return sweep_dict
def prepare_params(params: SystemParameters,
cartesian_sweep: bool = False) -> Mapping[str, List[object]]:
simple_params = {k: [v.value]
for k, v in params.items()
if type(v) is Param}
sweep_params: SweepableParameters = {k: v.value
for k, v in params.items()
if type(v) is ParamSweep}
if cartesian_sweep is True:
sweep_params = sweep_cartesian_product(sweep_params)
else:
pass
cleaned_params = {**simple_params, **sweep_params}
return cleaned_params
def prepare_state(state: InitialState) -> Mapping[str, object]:
cleaned_state = {k: v.value
for k, v in state.items()}
return cleaned_state
@dataclass
class ConfigurationWrapper():
initial_state: InitialState
params: SystemParameters
timestep_block: StateUpdateBlocks
timesteps: int
samples: int
def run(self, *args, **kwargs) -> DataFrame:
output = easy_run(prepare_state(self.initial_state),
prepare_params(self.params),
self.timestep_block,
self.timesteps,
self.samples,
*args,
**kwargs)
return output
================================================
FILE: cadCAD/tools/profiling/__init__.py
================================================
from cadCAD.tools.profiling.profile_run import profile_run
================================================
FILE: cadCAD/tools/profiling/profile_run.py
================================================
from typing import Dict
from cadCAD.tools import easy_run
from cadCAD.types import StateUpdateBlocks, Parameters, State, StateUpdateBlock
from time import time
import pandas as pd # type: ignore
def MEASURE_TIME_SUF(p, s, h, v, p_i): return ('run_time', time())
MEASURING_BLOCK: StateUpdateBlock = {
'label': 'Time Measure',
'policies': {},
'variables': {
'run_time': MEASURE_TIME_SUF
}
} # type: ignore
def profile_psubs(psubs: StateUpdateBlocks, profile_substeps=True) -> StateUpdateBlocks:
"""
Updates a TimestepBlock so that a time measuring function is added.
"""
new_timestep_block: StateUpdateBlocks = []
new_timestep_block.append(MEASURING_BLOCK)
if profile_substeps is True:
for psub in psubs:
new_timestep_block.append(psub)
new_timestep_block.append(MEASURING_BLOCK)
else:
pass
return new_timestep_block
def profile_run(state_variables: State,
params: Parameters,
psubs: StateUpdateBlocks,
*args,
profile_substeps=True,
**kwargs) -> pd.DataFrame:
if profile_substeps is True:
kwargs.update(drop_substeps=False)
new_psubs = profile_psubs(psubs, profile_substeps)
state_variables.update({'run_time': None})
return easy_run(state_variables,
params,
new_psubs,
*args,
**kwargs)
================================================
FILE: cadCAD/tools/profiling/visualizations.py
================================================
from tqdm.auto import tqdm
import pandas as pd
import plotly.express as px
import numpy as np
def visualize_elapsed_time_per_ts(df: pd.DataFrame, relative=False) -> None:
indexes = ['simulation', 'run', 'timestep', 'substep']
z_df = df.set_index(indexes)
first_time = z_df.query(
'timestep == 1 & substep == 1').reset_index([-1, -2]).run_time
s = (z_df.run_time - first_time)
s.name = 'time_since_start'
z_df = z_df.join(s)
s = z_df.groupby(indexes[:-1]).time_since_start.max()
fig_df = s.reset_index()
if relative is True:
s = fig_df.groupby(indexes[:-2]).time_since_start.diff()
s.name = 'psub_duration'
fig_df = fig_df.join(s)
y_col = 'psub_duration'
else:
y_col = 'time_since_start'
fig = px.box(fig_df,
x='timestep',
y=y_col)
return fig
def visualize_substep_impact(df: pd.DataFrame, relative=True, **kwargs) -> None:
indexes = ['simulation', 'run', 'timestep', 'substep']
new_df = df.copy()
new_df = new_df.assign(psub_time=np.nan).set_index(indexes)
# Calculate the run time associated with PSUBs
for ind, gg_df in tqdm(df.query('substep > 0').groupby(indexes[:-1])):
g_df = gg_df.reset_index()
N_rows = len(g_df)
substep_rows = list(range(N_rows))[1:-1:2]
for substep_row in substep_rows:
t1 = g_df.run_time[substep_row - 1]
t2 = g_df.run_time[substep_row + 1]
dt = t2 - t1
g_df.loc[substep_row, 'psub_time'] = dt
g_df = g_df.set_index(indexes)
new_df.loc[g_df.index, 'psub_time'] = g_df.psub_time
fig_df = new_df.reset_index().dropna(subset=['psub_time'])
if 'substep_label' in fig_df.columns:
x_col = 'substep_label'
else:
x_col = 'substep'
fig_df[x_col] = fig_df[x_col] / 2
if relative is True:
fig_df = fig_df.assign(relative_psub_time=fig_df.groupby(indexes[:-1]).psub_time.apply(lambda x: x / x.sum()))
y_col = 'relative_psub_time'
else:
y_col = 'psub_time'
inds = fig_df[y_col] < fig_df[y_col].quantile(0.95)
inds &= fig_df[y_col] > fig_df[y_col].quantile(0.05)
fig = px.box(fig_df[inds],
x=x_col,
y=y_col,
**kwargs)
return fig
================================================
FILE: cadCAD/tools/types.py
================================================
from typing import NamedTuple, Tuple, Dict, Union, List
class InitialValue(NamedTuple):
value: object
type: type
class Param(NamedTuple):
value: object
type: type
class ParamSweep(NamedTuple):
value: List[object]
type: type
InitialState = Dict[str, InitialValue]
SystemParameters = Dict[str, Union[Param, ParamSweep]]
================================================
FILE: cadCAD/tools/utils.py
================================================
from cadCAD.types import *
import pandas as pd
def generic_suf(variable: str, signal: str = "") -> StateUpdateFunction:
"""
Generate a State Update Function that assigns the signal value to the
given variable. By default, the signal has the same identifier as the
variable.
"""
if signal is "":
signal = variable
else:
pass
def suf(_1, _2, _3, _4, signals: PolicyOutput) -> StateUpdateTuple:
return (variable, signals[signal])
return suf
def add_parameter_labels(configs: list, df: pd.DataFrame) -> pd.DataFrame:
"""Utility function to add the parameters to a dataframe after processing
Args:
configs (list): The configurations of the simulations
df (pd.DataFrame): Simulation dataframe
Returns:
pd.DataFrame: Simulation dataframe with labels
"""
# Find the relevant parameters
sim_params = pd.DataFrame([x.sim_config["M"] for x in configs])
sim_params[["subset", "simulation", "run"]] = [
[x.subset_id, x.simulation_id, x.run_id] for x in configs
]
# Fix because run_id is 0 indexed, but cadCAD dataframe is 1 indexed for runs
sim_params["run"] += 1
# Join
sim_params = sim_params.set_index(["subset", "simulation", "run"])
df = df.join(sim_params, on=["subset", "simulation", "run"])
return df
================================================
FILE: cadCAD/types.py
================================================
from typing import TypedDict, Callable, Union, Dict, List, Tuple, Iterable
from collections import deque
State = Dict[str, object]
Parameters = Dict[str, object]
SweepableParameters = Dict[str, List[object]]
Substep = int
StateHistory = List[List[State]]
PolicyOutput = Dict[str, object]
StateVariable = object
PolicyFunction = Callable[[Parameters, Substep, StateHistory, State], PolicyOutput]
StateUpdateFunction = Callable[[Parameters, Substep, StateHistory, State, PolicyOutput], Tuple[str, StateVariable]]
class StateUpdateBlock(TypedDict):
policies: Dict[str, PolicyFunction]
variables: Dict[str, StateUpdateFunction]
StateUpdateBlocks = List[StateUpdateBlock]
class ConfigurationDict(TypedDict):
T: Iterable # Generator for the timestep variable
N: int # Number of MC Runs
M: Union[Parameters, SweepableParameters] # Parameters / List of Parameter to Sweep
TargetValue = object
EnvProcess: Callable[[State, SweepableParameters, TargetValue], TargetValue]
EnvProcesses = Dict[str, Callable]
TimeSeq = Iterable
SimulationID = int
Run = int
SubsetID = int
SubsetWindow = Iterable
N_Runs = int
ExecutorFunction = Callable[[Parameters, StateHistory, StateUpdateBlocks, EnvProcesses, TimeSeq, SimulationID, Run, SubsetID, SubsetWindow, N_Runs], object]
ExecutionParameter = Tuple[ExecutorFunction, Parameters, StateHistory, StateUpdateBlocks, EnvProcesses, TimeSeq, SimulationID, Run, SubsetID, SubsetWindow, N_Runs]
class SessionDict(TypedDict):
user_id: str
experiment_id: int
session_id: str
simulation_id: int
run_id: int
subset_id: int
subset_window: deque
================================================
FILE: cadCAD/utils/__init__.py
================================================
from functools import reduce
from collections import defaultdict
from itertools import product
import warnings
from typing import Union
from cadCAD.types import *
from typing import List, Dict, Union
import functools
import operator
from pandas import DataFrame # type: ignore
class SilentDF(DataFrame):
def __repr__(self):
return str(hex(id(DataFrame)))
def append_dict(dict: dict, new_dict: dict) -> dict:
"""
>>> append_dict({1: 2, 3: 4}, {3: 5})
{1: 2, 3: 5}
"""
dict.update(new_dict)
return dict
def arrange_cols(df: DataFrame, reverse=False) -> DataFrame:
"""
Reorders `df` columns so that the variable order is `session_metrics`
followed by `sys_metrics` and `results_cols`
"""
session_metrics = ['session_id', 'user_id', 'simulation_id', 'run_id']
sys_metrics = ['run', 'timestep', 'substep']
result_cols = list(set(df.columns) - set(session_metrics) - set(sys_metrics))
result_cols.sort(reverse=reverse)
return df[session_metrics + sys_metrics + result_cols]
class IndexCounter:
def __init__(self):
self.i = 0
def __call__(self):
self.i += 1
return self.i
def compose(*functions: Tuple[callable]) -> object:
return reduce(lambda f, g: lambda x: f(g(x)), functions, lambda x: x)
def pipe(x: object) -> object:
return x
def print_pipe(x: object) -> object:
print(x)
return x
def tupalize(k: object, vs: Union[list, dict]):
"""
>>> tupalize(1, 1)
[(1, 1)]
>>> tupalize(1, [2, 3])
[(1, 2), (1, 3)]
"""
l = []
if isinstance(vs, list):
for v in vs:
l.append((k, v))
else:
l.append((k, vs))
return l
def flattenDict(l: dict) -> list:
"""
>>> flattenDict({1: [1, 2, 3], 4: 5})
[{1: 1, 4: 5}, {1: 2, 4: 5}, {1: 3, 4: 5}]
"""
flat_list = [tupalize(k, vs) for k, vs in l.items()]
flat_dict = [dict(items) for items in product(*flat_list)]
return flat_dict
def flatten(l: Union[list, dict]):
if isinstance(l, list):
return functools.reduce(operator.iconcat, l, [])
elif isinstance(l, dict):
return flattenDict(l)
def flatMap(f, collection):
return flatten(list(map(f, collection)))
def dict_filter(dictionary, condition):
return dict([(k, v) for k, v in dictionary.items() if condition(v)])
def get_max_dict_val_len(g: Dict[str, List[int]]) -> int:
return len(max(g.values(), key=len))
def tabulate_dict(d: Dict[str, List[int]]) -> Dict[str, List[int]]:
max_len = get_max_dict_val_len(d)
_d = {}
for k, vl in d.items():
if len(vl) != max_len:
_d[k] = vl + list([vl[-1]] * (max_len-1))
else:
_d[k] = vl
return _d
def flatten_tabulated_dict(d: Dict[str, List[int]]) -> List[dict[str, int]]:
max_len = get_max_dict_val_len(d)
dl: List[dict] = [{} for i in range(max_len)]
for k, vl in d.items():
for v, i in zip(vl, list(range(len(vl)))):
dl[i][k] = v
return dl
def contains_type(_collection, type):
return any(isinstance(x, type) for x in _collection)
def drop_right(l, n):
return l[:len(l) - n]
def key_filter(l, keyname):
if (type(l) == list):
return [v[keyname] for v in l]
# Keeping support to dictionaries for backwards compatibility
# Should be removed in the future
warnings.warn(
"The use of a dictionary to describe Partial State Update Blocks will be deprecated. Use a list instead.",
FutureWarning)
return [v[keyname] for k, v in l.items()]
def groupByKey(l):
d = defaultdict(list)
for key, value in l:
d[key].append(value)
return list(dict(d).items()).pop()
# @curried
def rename(new_name, f):
f.__name__ = new_name
return f
def curry_pot(f, *argv):
sweep_ind = f.__name__[0:5] == 'sweep'
arg_len = len(argv)
if sweep_ind is True and arg_len == 4:
return f(argv[0])(argv[1])(argv[2])(argv[3])
elif sweep_ind is False and arg_len == 4:
return f(argv[0], argv[1], argv[2], argv[3])
elif sweep_ind is True and arg_len == 3:
return f(argv[0])(argv[1])(argv[2])
elif sweep_ind is False and arg_len == 3:
return f(argv[0], argv[1], argv[2])
else:
raise TypeError('curry_pot() needs 3 or 4 positional arguments')
================================================
FILE: cadCAD/utils/execution.py
================================================
from pprint import pprint
from cadCAD import logo, version
from cadCAD.utils import flatten
def print_exec_info(exec_context, configs):
print(logo)
print(f'cadCAD Version: {version}')
print(f'Execution Mode: {exec_context}')
models = len(configs)
sim_strs, run_vals, timestep_vals, params, sub_states = [], [], [], [], set()
for i, config in enumerate(configs):
sim_config = config.sim_config
n_n = sim_config['N']
n_t = len(sim_config['T'])
n_m = len(sim_config['M'])
n_s = len(config.initial_state)
run_vals.append(n_n)
for timestep in [*sim_config['T']]:
timestep_vals.append(timestep)
if type(sim_config['M']) is dict:
params.append(n_m)
else:
n_m = 0
for state_key in list(config.initial_state.keys()):
sub_states.add(state_key)
sim_strs.append(f' Simulation {i}: (Timesteps, Params, Runs, Sub-States) = ({n_t}, {n_m}, {n_n}, {n_s})')
timesteps = len(timestep_vals)
if sum(params) != 0:
param_count = sum(params)
else:
param_count = 0
runs = sum(run_vals)
init_states = len(sub_states)
print("Simulation Dimensions:")
print(
f'Entire Simulation: (Models, Unique Timesteps, Params, Total Runs, Sub-States) = ({models}, {timesteps}, {param_count}, {runs}, {init_states})'
)
for sim_str in sim_strs:
print(sim_str)
================================================
FILE: cadCAD/utils/jupyter.py
================================================
def get_home_dir(user):
return f"s3://jupyterbackups/jupyter/{user}/"
def set_write_path(sc, user, datafolder_path):
return get_home_dir(user) + datafolder_path +f'_{sc.applicationId}'
================================================
FILE: cadCAD/utils/sys_config.py
================================================
from funcy import curry
from cadCAD.configuration.utils import ep_time_step, time_step
def increment(y, incr_by):
return lambda _g, step, sL, s, _input, **kwargs: (y, s[y] + incr_by)
def track(y):
return lambda _g, step, sL, s, _input, **kwargs: (y, s[y].x)
def simple_state_update(y, x):
return lambda _g, step, sH, s, _input, **kwargs: (y, x)
def simple_policy_update(y):
return lambda _g, step, sH, s, **kwargs: y
def update_timestamp(y, timedelta, format):
return lambda _g, step, sL, s, _input, **kwargs: (
y,
ep_time_step(s, dt_str=s[y], fromat_str=format, _timedelta=timedelta)
)
def apply(f, y: str, incr_by: int):
return lambda _g, step, sL, s, _input, **kwargs: (y, curry(f)(s[y])(incr_by))
def add(y: str, incr_by):
return apply(lambda a, b: a + b, y, incr_by)
def increment_state_by_int(y: str, incr_by: int):
return lambda _g, step, sL, s, _input, **kwargs: (y, s[y] + incr_by)
def s(y, x):
return lambda _g, step, sH, s, _input, **kwargs: (y, x)
def time_model(y, substeps, time_delta, ts_format='%Y-%m-%d %H:%M:%S'):
def apply_incriment_condition(s):
if s['substep'] == 0 or s['substep'] == substeps:
return y, time_step(dt_str=s[y], dt_format=ts_format, _timedelta=time_delta)
else:
return y, s[y]
return lambda _g, step, sL, s, _input, **kwargs: apply_incriment_condition(s)
================================================
FILE: documentation/Historically_State_Access.md
================================================
Historical State Access (DEPRECATED)
==
#### Motivation
The current state (values of state variables) is accessed through the `s` list. When the user requires previous state
variable values, they may be accessed through the state history list, `sH`. Accessing the state history should be
implemented without creating unintended feedback loops on the current state.
The 3rd parameter of state and policy update functions (labeled as `sH` of type `list[list[dict]]`) provides access to
past Partial State Update Block (PSUB) given a negative offset number. `access_block` is used to access past PSUBs
(`list[dict]`) from `sH`. For example, an offset of `-2` denotes the second to last PSUB.
#### Exclusion list
Create a list of states to exclude from the reported PSU.
```python
exclusion_list = [
'nonexistent', 'last_x', '2nd_to_last_x', '3rd_to_last_x', '4th_to_last_x'
]
```
##### Example Policy Updates
###### Last partial state update
```python
def last_update(_params, substep, sH, s, **kwargs):
return {"last_x": access_block(
state_history=sH,
target_field="last_x", # Add a field to the exclusion list
psu_block_offset=-1,
exculsion_list=exclusion_list
)
}
```
* Note: Although `target_field` adding a field to the exclusion may seem redundant, it is useful in the case of the
exclusion list being empty while the `target_field` is assigned to a state or a policy key.
##### Define State Updates
###### 2nd to last partial state update
```python
def second2last_update(_params, substep, sH, s, **kwargs):
return {"2nd_to_last_x": access_block(sH, "2nd_to_last_x", -2, exclusion_list)}
```
###### 3rd to last partial state update
```python
def third_to_last_x(_params, substep, sH, s, _input, **kwargs):
return '3rd_to_last_x', access_block(sH, "3rd_to_last_x", -3, exclusion_list)
```
###### 4rd to last partial state update
```python
def fourth_to_last_x(_params, substep, sH, s, _input, **kwargs):
return '4th_to_last_x', access_block(sH, "4th_to_last_x", -4, exclusion_list)
```
###### Non-exsistent partial state update
* `psu_block_offset >= 0` doesn't exist
```python
def nonexistent(_params, substep, sH, s, _input, **kwargs):
return 'nonexistent', access_block(sH, "nonexistent", 0, exclusion_list)
```
#### [Example Simulation:](examples/historical_state_access.py)
#### Example Output:
###### State History
```
+----+-------+-----------+------------+-----+
| | run | substep | timestep | x |
|----+-------+-----------+------------+-----|
| 0 | 1 | 0 | 0 | 0 |
| 1 | 1 | 1 | 1 | 1 |
| 2 | 1 | 2 | 1 | 2 |
| 3 | 1 | 3 | 1 | 3 |
| 4 | 1 | 1 | 2 | 4 |
| 5 | 1 | 2 | 2 | 5 |
| 6 | 1 | 3 | 2 | 6 |
| 7 | 1 | 1 | 3 | 7 |
| 8 | 1 | 2 | 3 | 8 |
| 9 | 1 | 3 | 3 | 9 |
+----+-------+-----------+------------+-----+
```
###### Accessed State History:
Example: `last_x`
```
+----+-----------------------------------------------------------------------------------------------------------------------------------------------------+
| | last_x |
|----+-----------------------------------------------------------------------------------------------------------------------------------------------------|
| 0 | [] |
| 1 | [{'x': 0, 'run': 1, 'substep': 0, 'timestep': 0}] |
| 2 | [{'x': 0, 'run': 1, 'substep': 0, 'timestep': 0}] |
| 3 | [{'x': 0, 'run': 1, 'substep': 0, 'timestep': 0}] |
| 4 | [{'x': 1, 'run': 1, 'substep': 1, 'timestep': 1}, {'x': 2, 'run': 1, 'substep': 2, 'timestep': 1}, {'x': 3, 'run': 1, 'substep': 3, 'timestep': 1}] |
| 5 | [{'x': 1, 'run': 1, 'substep': 1, 'timestep': 1}, {'x': 2, 'run': 1, 'substep': 2, 'timestep': 1}, {'x': 3, 'run': 1, 'substep': 3, 'timestep': 1}] |
| 6 | [{'x': 1, 'run': 1, 'substep': 1, 'timestep': 1}, {'x': 2, 'run': 1, 'substep': 2, 'timestep': 1}, {'x': 3, 'run': 1, 'substep': 3, 'timestep': 1}] |
| 7 | [{'x': 4, 'run': 1, 'substep': 1, 'timestep': 2}, {'x': 5, 'run': 1, 'substep': 2, 'timestep': 2}, {'x': 6, 'run': 1, 'substep': 3, 'timestep': 2}] |
| 8 | [{'x': 4, 'run': 1, 'substep': 1, 'timestep': 2}, {'x': 5, 'run': 1, 'substep': 2, 'timestep': 2}, {'x': 6, 'run': 1, 'substep': 3, 'timestep': 2}] |
| 9 | [{'x': 4, 'run': 1, 'substep': 1, 'timestep': 2}, {'x': 5, 'run': 1, 'substep': 2, 'timestep': 2}, {'x': 6, 'run': 1, 'substep': 3, 'timestep': 2}] |
+----+-----------------------------------------------------------------------------------------------------------------------------------------------------+
```
================================================
FILE: documentation/Policy_Aggregation.md
================================================
Policy Aggregation
==
For each Partial State Update, multiple policy dictionaries are aggregated into a single dictionary to be inputted into
all state functions using an initial reduction function and optional subsequent map functions.
#### Aggregate Function Composition:
```python
# Reduce Function
add = lambda a, b: a + b # Used to add policy values of the same key
# Map Function
mult_by_2 = lambda y: y * 2 # Used to multiply all policy values by 2
policy_ops=[add, mult_by_2]
```
##### Example Policy Updates per Partial State Update (PSU)
```python
def p1_psu1(_params, step, sH, s, **kwargs):
return {'policy1': 1}
def p2_psu1(_params, step, sH, s, **kwargs):
return {'policy2': 2}
```
* `add` not applicable due to lack of redundant policies
* `mult_by_2` applied to all policies
* Result: `{'policy1': 2, 'policy2': 4}`
```python
def p1_psu2(_params, step, sH, s, **kwargs):
return {'policy1': 2, 'policy2': 2}
def p2_psu2(_params, step, sH, s, **kwargs):
return {'policy1': 2, 'policy2': 2}
```
* `add` applicable due to redundant policies
* `mult_by_2` applied to all policies
* Result: `{'policy1': 8, 'policy2': 8}`
```python
def p1_psu3(_params, step, sH, s, **kwargs):
return {'policy1': 1, 'policy2': 2, 'policy3': 3}
def p2_psu3(_params, step, sH, s, **kwargs):
return {'policy1': 1, 'policy2': 2, 'policy3': 3}
```
* `add` applicable due to redundant policies
* `mult_by_2` applied to all policies
* Result: `{'policy1': 4, 'policy2': 8, 'policy3': 12}`
#### Aggregate Policies using functions
```python
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_model(
sim_configs=???,
initial_state=???,
partial_state_update_blocks=???,
policy_ops=[add, mult_by_2] # Default: [lambda a, b: a + b]
)
```
#### Example
##### * [System Model Configuration](examples/policy_aggregation.py)
##### * Simulation Results:
```
+----+---------------------------------------------+-------+------+-----------+------------+
| | policies | run | s1 | substep | timestep |
|----+---------------------------------------------+-------+------+-----------+------------|
| 0 | {} | 1 | 0 | 0 | 0 |
| 1 | {'policy1': 2, 'policy2': 4} | 1 | 1 | 1 | 1 |
| 2 | {'policy1': 8, 'policy2': 8} | 1 | 2 | 2 | 1 |
| 3 | {'policy3': 12, 'policy1': 4, 'policy2': 8} | 1 | 3 | 3 | 1 |
| 4 | {'policy1': 2, 'policy2': 4} | 1 | 4 | 1 | 2 |
| 5 | {'policy1': 8, 'policy2': 8} | 1 | 5 | 2 | 2 |
| 6 | {'policy3': 12, 'policy1': 4, 'policy2': 8} | 1 | 6 | 3 | 2 |
| 7 | {'policy1': 2, 'policy2': 4} | 1 | 7 | 1 | 3 |
| 8 | {'policy1': 8, 'policy2': 8} | 1 | 8 | 2 | 3 |
| 9 | {'policy3': 12, 'policy1': 4, 'policy2': 8} | 1 | 9 | 3 | 3 |
+----+---------------------------------------------+-------+------+-----------+------------+
```
================================================
FILE: documentation/README.md
================================================
Simulation Configuration
==
## Introduction
Given a **Simulation Configuration**, cadCAD produces datasets that represent the evolution of the state of a system
over [discrete time](https://en.wikipedia.org/wiki/Discrete_time_and_continuous_time#Discrete_time). The state of the
system is described by a set of [State Variables](#State-Variables). The dynamic of the system is described by
[Policy Functions](#Policy-Functions) and [State Update Functions](#State-Update-Functions), which are evaluated by
cadCAD according to the definitions set by the user in [Partial State Update Blocks](#Partial-State-Update-Blocks).
A Simulation Configuration is comprised of a [System Model](#System-Model) and a set of [Simulation Properties](#Simulation-Properties).
### Experiments
`cadCAD.configuration.Experiment` is a unique representation of an experiment of one or more configured System Models.
The `append_model` method of `Experiment` appends a System Model configurations, each representing a single `run`.
```python
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_model(
model_id = ..., # OPTIONAL: System Model label
initial_state = ..., # System Model
partial_state_update_blocks = ..., # System Model
policy_ops = ..., # System Model
sim_configs = ..., # Simulation Properties
user_id = ..., # OPTIONAL: Configuration User ID
)
```
Parameters: `append_model`
* **model_id** : str - OPTIONAL: System Model label
* **initial_state** : _dict_ - [State Variables](#State-Variables) and their initial values
* **partial_state_update_blocks** : List[dict[dict]] - list of [Partial State Update Blocks](#Partial-State-Update-Blocks)
* **policy_ops** : List[functions] - See [Policy Aggregation](Policy_Aggregation.md)
* **sim_configs** - See [System Model Parameter Sweep](System_Model_Parameter_Sweep.md)
* **user_id** : str - OPTIONAL: Configuration User ID
## Simulation Properties
Simulation properties are passed to `append_model` in the `sim_configs` parameter. To construct this parameter, we
use the `config_sim` function in `cadCAD.configuration.utils`
```python
from cadCAD.configuration.utils import config_sim
from cadCAD.configuration import Experiment
sim_config_dict = {
"N": ...,
"T": range(...),
"M": ...
}
c = config_sim(sim_config_dict)
exp = Experiment()
exp.append_model(
...
sim_configs = c # Simulation Properties
)
```
### T - Simulation Length
Computer simulations run in discrete time:
>Discrete time views values of variables as occurring at distinct, separate "points in time", or equivalently as being
unchanged throughout each non-zero region of time ("time period")—that is, time is viewed as a discrete variable. (...)
This view of time corresponds to a digital clock that gives a fixed reading of 10:37 for a while, and then jumps to a
new fixed reading of 10:38, etc.
([source: Wikipedia](https://en.wikipedia.org/wiki/Discrete_time_and_continuous_time#Discrete_time))
As is common in many simulation tools, in cadCAD too we refer to each discrete unit of time as a **timestep**. cadCAD
increments a "time counter", and at each step it updates the state variables according to the equations that describe
the system.
The main simulation property that the user must set when creating a Simulation Configuration is the number of timesteps
in the simulation. In other words, for how long do they want to simulate the system that has been modeled.
### N - Number of Runs
cadCAD facilitates running multiple simulations of the same system sequentially, reporting the results of all those
runs in a single dataset. This is especially helpful for running
[Monte Carlo Simulations](https://github.com/cadCAD-org/demos/blob/master/tutorials/robots_and_marbles/robot-marbles-part-4/robot-marbles-part-4.ipynb).
### M - Parameters of the System
Parameters of the system, passed to the state update functions and the policy functions in the `params` parameter are
defined here. See [System Model Parameter Sweep](System_Model_Parameter_Sweep.md) for more information.
## System Model
The System Model describes the system that will be simulated in cadCAD. It is comprised of a set of
[State Variables](##State-Variables) and the [State Update Functions](#State-Update-Functions) that determine the
evolution of the state of the system over time. [Policy Functions](#Policy-Functions) (representations of user policies
or internal system control policies) may also be part of a System Model.
### State Variables
>A state variable is one of the set of variables that are used to describe the mathematical "state" of a dynamical
system. Intuitively, the state of a system describes enough about the system to determine its future behaviour in the
absence of any external forces affecting the system. ([source: Wikipedia](https://en.wikipedia.org/wiki/State_variable))
cadCAD can handle state variables of any Python data type, including custom classes. It is up to the user of cadCAD to
determine the state variables needed to **sufficiently and accurately** describe the system they are interested in.
State Variables are passed to `append_model` along with its initial values, as a Python `dict` where the `dict_keys`
are the names of the variables and the `dict_values` are their initial values.
```python
from cadCAD.configuration import Experiment
genesis_states = {
'state_variable_1': 0,
'state_variable_2': 0,
'state_variable_3': 1.5,
'timestamp': '2019-01-01 00:00:00'
}
exp = Experiment()
exp.append_model(
initial_state = genesis_states,
...
)
```
### State Update Functions
State Update Functions represent equations according to which the state variables change over time. Each state update
function must return a tuple containing a string with the name of the state variable being updated and its new value.
Each state update function can only modify a single state variable. The general structure of a state update function is:
```python
def state_update_function_A(_params, substep, sH, s, _input, **kwargs):
...
return 'state_variable_name', new_value
```
Parameters:
* **_params** : _dict_ - [System parameters](System_Model_Parameter_Sweep.md)
* **substep** : _int_ - Current [substep](#Substep)
* **sH** : _list[list[dict_]] - Historical values of all state variables for the simulation. See
[Historical State Access (DEPRECATED)](Historically_State_Access.md) for details
* **s** : _dict_ - Current state of the system, where the `dict_keys` are the names of the state variables and the
`dict_values` are their current values.
* **_input** : _dict_ - Aggregation of the signals of all policy functions in the current
[Partial State Update Block](#Partial-State-Update-Block)
* **\*\*kwargs** - State Update feature extensions
Return:
* _tuple_ containing a string with the name of the state variable being updated and its new value.
State update functions should not modify any of the parameters passed to it, as those are mutable Python objects that
cadCAD relies on in order to run the simulation according to the specifications.
### Policy Functions
A Policy Function computes one or more signals to be passed to [State Update Functions](#State-Update-Functions)
(via the _\_input_ parameter). Read
[this article](https://github.com/cadCAD-org/demos/blob/master/tutorials/robots_and_marbles/robot-marbles-part-2/robot-marbles-part-2.ipynb)
for details on why and when to use policy functions.
<!-- We would then expand the tutorials with these kind of concepts
#### Policies
Policies consist of the potential action made available through mechanisms. The action taken is expected to be the
result of a conditional determination of the past state.
While executed the same, the modeller can approach policies dependent on the availability of a mechanism to a population.
- ***Control Policy***
When the controlling or deploying entity has the ability to act in order to affect some aspect of the system, this is a
control policy.
- ***User Policy*** model agent behaviors in reaction to state variables and exogenous variables. The resulted user
action will become an input to PSUs. Note that user behaviors should not directly update value of state variables.
The action taken, as well as the potential to act, through a mechanism is a behavior. -->
The general structure of a policy function is:
```python
def policy_function_1(_params, substep, sH, s, **kwargs):
...
return {'signal_1': value_1, ..., 'signal_N': value_N}
```
Parameters:
* **_params** : _dict_ - [System parameters](System_Model_Parameter_Sweep.md)
* **substep** : _int_ - Current [substep](#Substep)
* **sH** : _list[list[dict_]] - Historical values of all state variables for the simulation. See
[Historical State Access (DEPRECATED)](Historically_State_Access.md) for details
* **s** : _dict_ - Current state of the system, where the `dict_keys` are the names of the state variables and the
`dict_values` are their current values.
* **\*\*kwargs** - Policy Update feature extensions
Return:
* _dict_ of signals to be passed to the state update functions in the same
[Partial State Update Block](#Partial-State-Update-Blocks)
Policy functions should not modify any of the parameters passed to it, as those are mutable Python objects that cadCAD
relies on in order to run the simulation according to the specifications.
At each [Partial State Update Block](#Partial-State-Update-Blocks) (PSUB), the `dicts` returned by all policy functions
within that PSUB dictionaries are aggregated into a single `dict` using an initial reduction function
(a key-wise operation, default: `dic1['keyA'] + dic2['keyA']`) and optional subsequent map functions. The resulting
aggregated `dict` is then passed as the `_input` parameter to the state update functions in that PSUB. For more
information on how to modify the aggregation method, see [Policy Aggregation](Policy_Aggregation.md).
### Partial State Update Blocks
A **Partial State Update Block** (PSUB) is a set of State Update Functions and Policy Functions such that State Update
Functions in the set are independent from each other and Policies in the set are independent from each other and from
the State Update Functions in the set. In other words, if a state variable is updated in a PSUB, its new value cannot
impact the State Update Functions and Policy Functions in that PSUB - only those in the next PSUB.
Partial State Update Blocks are passed to `append_model` as a list of Python `dicts` where the `dict_keys` are named
`"policies"` and `"variables"` and the values are also Python `dicts` where the keys are the names of the policy and
state update functions and the values are the functions.
```python
from cadCAD.configuration import Experiment
PSUBs = [
{
"policies": {
"b_1": policy_function_1,
...
"b_J": policy_function_J
},
"variables": {
"s_1": state_update_function_1,
...
"s_K": state_update_function_K
}
}, #PSUB_1,
{...}, #PSUB_2,
...
{...} #PSUB_M
]
exp = Experiment()
exp.append_model(
...
partial_state_update_blocks = PSUBs,
...
)
```
#### Substep
At each timestep, cadCAD iterates over the `partial_state_update_blocks` list. For each Partial State Update Block,
cadCAD returns a record containing the state of the system at the end of that PSUB. We refer to that subdivision of a
timestep as a `substep`.
## Result Dataset
cadCAD returns a dataset containing the evolution of the state variables defined by the user over time, with three `int`
indexes:
* `subset` - identifies the subset per sweepable parameter produced by a parameter sweep (ver. `0.3.1`'s result was
multiple datasets; A single dataset per sweepable parameter).
* `run` - identifies the [run](#N-Number-of-Runs)
* `timestep` - discrete unit of time (the total number of timesteps is defined by the user in the
[T Simulation Parameter](#T-Simulation-Length))
* `substep` - subdivision of timestep (the number of [substeps](#Substeps) is the same as the number of Partial State
Update Blocks)
* `simulation` - **Alpha: Ignore**
Therefore, the total number of records in the resulting dataset is `N` x `T` x `len(partial_state_update_blocks)`
#### [System Simulation Execution](Simulation_Execution.md)
================================================
FILE: documentation/Simulation_Execution.md
================================================
# Simulation Execution
System Simulations are executed with the execution engine executor (`cadCAD.engine.Executor`) given System Model
Configurations. There are multiple simulation Execution Modes and Execution Contexts.
## Steps
### 1. *Choose Execution Mode*
#### Simulation Execution Modes:
`cadCAD` executes a process per System Model Configuration and a thread per System Simulation.
#### Class: `cadCAD.engine.ExecutionMode`
#### Attributes:
* **Local Mode (Default):** Automatically selects Single Threaded or Multi-Process Modes (Example:
`cadCAD.engine.ExecutionMode().local_mode`).
* **Single Threaded Mode:** A single threaded Execution Mode for a single System Model Configuration (Example:
`cadCAD.engine.ExecutionMode().single_mode`).
* **Multi-Process Mode:** Execution Mode for System Model Simulations which executes Multiple processes within
multiple processes per given System Model Configuration (Example: `cadCAD.engine.ExecutionMode().multi_mode`).
### 2. *Create Execution Context using Execution Mode*
```python
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
local_mode_ctx = ExecutionContext(context=exec_mode.local_mode)
```
### 3. *Create Simulation Executor*
```python
from cadCAD.engine import Executor
from ... import exp # import of an instantiated of an `cadCAD.configuration.Experiment` object
simulation = Executor(exec_context=local_mode_ctx, configs=exp.configs)
```
### 4. *Execute Simulation & Produce System Event Dataset*
A Simulation execution produces a System Event Dataset and the Tensor Field applied to initial states used to create it.
```python
import pandas as pd
raw_system_events, tensor_field, sessions = simulation.execute()
# Simulation Result Types:
# raw_system_events: List[dict]
# tensor_field: pd.DataFrame
# Result System Events DataFrame
simulation_result = pd.DataFrame(raw_system_events)
```
#### Example Tensor Field
```
+----+-----+--------------------------------+--------------------------------+
| | m | b1 | s1 |
|----+-----+--------------------------------+--------------------------------|
| 0 | 1 | <function p1m1 at 0x10c458ea0> | <function s1m1 at 0x10c464510> |
| 1 | 2 | <function p1m2 at 0x10c464048> | <function s1m2 at 0x10c464620> |
| 2 | 3 | <function p1m3 at 0x10c464400> | <function s1m3 at 0x10c464730> |
+----+-----+--------------------------------+--------------------------------+
```
#### Example Result: System Events DataFrame
```
+----+------+-----------+------------+--------+-----+---------+----------+
| | s1 | s2 | simulation | subset | run | substep | timestep |
|----+------+-----------|------------+--------+-----+---------+----------|
| 0 | 0 | 0.0 | 0 | 0 | 1 | 0 | 0 |
| 1 | 1 | 4 | 0 | 0 | 1 | 1 | 1 |
| 2 | 2 | 6 | 0 | 0 | 1 | 2 | 1 |
| 3 | 3 | [ 30 300] | 0 | 0 | 1 | 3 | 1 |
| 4 | 0 | 0.0 | 1 | 0 | 1 | 0 | 0 |
| 5 | 1 | 4 | 1 | 0 | 1 | 1 | 1 |
| 6 | 2 | 6 | 1 | 0 | 1 | 2 | 1 |
| 7 | 3 | [ 30 300] | 1 | 0 | 1 | 3 | 1 |
+----+------+-----------+------------+--------+-----+---------+----------+
```
## Execution Examples:
### Single Simulation Execution (Single Threaded Execution)
Example System Model Configurations:
* [System Model A](examples/sys_model_A.py)
* [System Model B](examples/sys_model_B.py)
Example Simulation Executions:
* [System Model A](examples/sys_model_A_exec.py)
* [System Model B](examples/sys_model_B_exec.py)
```python
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
from documentation.examples import sys_model_A
from documentation.examples.sys_model_A import exp
exec_mode = ExecutionMode()
# Single Process Execution using a Single System Model Configuration:
# sys_model_A
local_mode_ctx = ExecutionContext(context=exec_mode.local_mode)
sys_model_A_simulation = Executor(exec_context=local_mode_ctx, configs=exp.configs)
sys_model_A_raw_result, sys_model_A_tensor_field, sessions = sys_model_A_simulation.execute()
sys_model_A_result = pd.DataFrame(sys_model_A_raw_result)
print()
print("Tensor Field: sys_model_A")
print(tabulate(sys_model_A_tensor_field, headers='keys', tablefmt='psql'))
print("Result: System Events DataFrame")
print(tabulate(sys_model_A_result, headers='keys', tablefmt='psql'))
print()
```
#### Multiple Simulation Execution
##### Multi-Process / Threaded Execution
Example System Model Configurations:
* [System Model A](examples/sys_model_A.py)
* [System Model B](examples/sys_model_B.py)
Example Simulation Executions:
* [System Model AB](examples/sys_model_AB_exec.py)
```python
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
from documentation.examples import sys_model_A, sys_model_B, system_model_AB_exp
exec_mode = ExecutionMode()
# # Multiple Processes Execution using Multiple System Model Configurations:
# # sys_model_A & sys_model_B
local_mode_ctx = ExecutionContext(context=exec_mode.local_mode)
sys_model_AB_simulation = Executor(exec_context=local_mode_ctx, configs=system_model_AB_exp.configs)
sys_model_AB_raw_result, sys_model_AB_tensor_field, sessions = sys_model_AB_simulation.execute()
sys_model_AB_result = pd.DataFrame(sys_model_AB_raw_result)
print()
print("Tensor Field:")
print(tabulate(sys_model_AB_tensor_field, headers='keys', tablefmt='psql'))
print("Result: System Events DataFrame:")
print(tabulate(sys_model_AB_result, headers='keys', tablefmt='psql'))
print()
```
##### System Model Parameter Sweep [Info](System_Model_Parameter_Sweep.md)
Example: [Param Sweep](examples/param_sweep.py)
```python
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
from documentation.examples import param_sweep
from documentation.examples.param_sweep import exp
exec_mode = ExecutionMode()
local_mode_ctx = ExecutionContext(context=exec_mode.local_mode)
run = Executor(exec_context=local_mode_ctx, configs=exp.configs)
raw_result, tensor_field, sessions = run.execute()
result = pd.DataFrame(raw_result)
print()
print("Tensor Field:")
print(tabulate(tensor_field, headers='keys', tablefmt='psql'))
print("Output:")
print(tabulate(result, headers='keys', tablefmt='psql'))
print()
```
================================================
FILE: documentation/System_Configuration.md
================================================
# Display System Model Configurations:
## Announcement:
See [CHANGELOG](CHANGELOG.md)
The `cadCAD.configuration.Experiment().configs` (System Model Configurations) `list` has been flattened
to contain single run `Configuration` objects. This functionality will be restored in a
subsequent release by a class that returns the original representation in ver. `0.3.1`.
* The conversion utilities have been provided to restore its original representation of configurations with
runs >= 1
* System Configuration Conversions:
* Configuration as list of Configuration Objects (as in ver. `0.3.1`)
* New: System Configuration as a Pandas DataFrame
* New: System Configuration as list of Dictionaries
## Conversions
##### Note: The following applies as a result of simulation execution
#### Imports:
```python
from cadCAD.configuration.utils import configs_as_objs, configs_as_dataframe, configs_as_dicts
```
#### System Configurations as list of Configuration Objects
Example:
* `configs` is temporarily returned in a flattened format and reformatted into its intended format.
* `Configuration` objects at `0x10790e470` and `0x1143dd630` are reconstituted into objects at `0x10790e7b8`
and `0x116268908` respectively.
```python
from ... import exp # import of an instantiated `cadCAD.configuration.Experiment` object
flattened_configs = exp.configs
print('Flattened Format: Temporary')
pprint(flattened_configs)
print()
print('Intended Format:')
intended_configs = configs_as_objs(flattened_configs)
pprint(intended_configs)
print()
print("Object: cadCAD.configuration.Configuration(...).sim_config")
pprint(intended_configs[0].sim_config)
print()
```
Return:
```
Flattened Format: Temporary
[<cadCAD.configuration.Configuration object at 0x10790e470>,
<cadCAD.configuration.Configuration object at 0x10790e7b8>,
<cadCAD.configuration.Configuration object at 0x1143dd630>,
<cadCAD.configuration.Configuration object at 0x116268908>]
Intended Format:
[<cadCAD.configuration.Configuration object at 0x10790e7b8>,
<cadCAD.configuration.Configuration object at 0x116268908>]
Object: cadCAD.configuration.Configuration(...).sim_config
{'M': [{}],
'N': 2,
'T': range(0, 1),
'run_id': 1,
'simulation_id': 0,
'subset_id': 0,
'subset_window': deque([0, None], maxlen=2)}
```
#### System Configurations as a Pandas DataFrame
```python
flattened_configs = configs
configs_df = configs_as_dataframe(configs)
configs_df
```
#### System Configurations as list of Dictionaries
```python
configs_dicts: list = configs_as_dicts(configs)
pprint(configs_dicts[0]['sim_config'])
```
Return:
```
{'env_processes': {'s3': [<function <lambda> at 0x7f8f9c99bd90>,
<function <lambda> at 0x7f8f9c9a11e0>],
's4': <function env_trigger.<locals>.trigger.<locals>.env_update at 0x7f8f9c9a12f0>},
'exogenous_states': {},
'initial_state': {'s1': 0.0,
's2': 0.0,
's3': 1.0,
's4': 1.0,
'timestamp': '2018-10-01 15:16:24'},
'kwargs': {},
'partial_state_updates': [{'policies': {'p1': <function p1m1 at 0x7f8f9c985ea0>,
'p2': <function p2m1 at 0x7f8f9c985f28>},
'variables': {'s1': <function s1m1 at 0x7f8f9c99b268>,
's2': <function s2m1 at 0x7f8f9c99b2f0>,
's3': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99bae8>,
's4': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99bea0>,
'timestamp': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99b730>}},
{'policies': {'p1': <function p1m2 at 0x7f8f9c99b048>,
'p2': <function p2m2 at 0x7f8f9c99b0d0>},
'variables': {'s1': <function s1m2 at 0x7f8f9c99b378>,
's2': <function s2m2 at 0x7f8f9c99b400>,
's3': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99bbf8>,
's4': <function var_trigger.<locals>.<lambda> at 0x7f8f9c9a1048>,
'timestamp': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99b840>}},
{'policies': {'p1': <function p1m3 at 0x7f8f9c99b158>,
'p2': <function p2m3 at 0x7f8f9c99b1e0>},
'variables': {'s1': <function s1m3 at 0x7f8f9c99b488>,
's2': <function s2m3 at 0x7f8f9c99b510>,
's3': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99bd08>,
's4': <function var_trigger.<locals>.<lambda> at 0x7f8f9c9a1158>,
'timestamp': <function var_trigger.<locals>.<lambda> at 0x7f8f9c99b950>}}],
'policy_ops': [<function <lambda> at 0x7f8f9cb39158>],
'run_id': 249,
'seeds': {'a': <mtrand.RandomState object at 0x7f8f9c9a21f8>,
'b': <mtrand.RandomState object at 0x7f8f9c9a2240>,
'c': <mtrand.RandomState object at 0x7f8f9c9a2288>,
'z': <mtrand.RandomState object at 0x7f8f9ca04948>},
'session_id': 'cadCAD_user=0_249',
'sim_config': {'M': {'alpha': 1, 'beta': 2, 'gamma': 3, 'omega': 7},
'N': 250,
'T': range(0, 5000),
'run_id': 249,
'simulation_id': 0},
'simulation_id': 0,
'user_id': 'cadCAD_user'}
```
================================================
FILE: documentation/System_Model_Parameter_Sweep.md
================================================
System Model Parameter Sweep
==
Parametrization of a System Model configuration that produces multiple configurations.
##### Set Parameters
*Note:* `params` values require up to a maximum of 2 distinct lengths
```python
params = {
'alpha': [1],
'beta': [2, 5],
'gamma': [3, 4],
'omega': [7]
}
```
The parameters above produce 2 Runs per Simulation.
* Run 1:
* `alpha = 1`
* `beta = 2`
* `gamma = 3`
* `omega = 7`
* Run 2:
* `alpha = 1`
* `beta = 5`
* `gamma = 4`
* `omega = 7`
All parameters can also be set to include a single parameter each, which will result in a single simulation.
##### Example State Updates
Previous State:
`y = 0`
```python
def state_update(_params, step, sH, s, _input, **kwargs):
y = 'state'
x = s['state'] + _params['alpha'] + _params['gamma']
return y, x
```
* Updated State:
* Simulation 1: `y = 4 = 0 + 1 + 3`
* Simulation 2: `y = 5 = 0 + 1 + 4`
##### Example Policy Updates
```python
# Internal States per Mechanism
def policies(_params, step, sH, s, **kwargs):
return {'beta': _params['beta'], 'gamma': _params['gamma']}
```
* Simulation 1: `{'beta': 2, 'gamma': 3]}`
* Simulation 2: `{'beta': 5, 'gamma': 4}`
##### Configure Simulation
```python
from cadCAD.configuration.utils import config_sim
g = {
'alpha': [1],
'beta': [2, 5],
'gamma': [3, 4],
'omega': [7]
}
sim_config = config_sim(
{
"N": 2,
"T": range(5),
"M": g,
}
)
```
#### Example
##### * [System Model Configuration](examples/param_sweep.py)
================================================
FILE: documentation/__init__.py
================================================
================================================
FILE: documentation/cadCAD-v0.4.23-Model-Upgrade-Guide.md
================================================
<table>
<tr>
<th>
Feature
</th>
<th>
ver. 0.4.23
</th>
<th>
ver. 0.3.1 (Deprecated)
</th>
</tr>
<tr>
<td>
<h5>
Experiments
</h5>
</td>
<td>
<pre lang="python">
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_configs(...)
</pre>
</td>
<td>
<pre lang="python">
from cadCAD.configuration import append_configs
append_configs(…)
</pre>
</td>
</tr>
<tr>
<td>
<h5>
Local Execution Mode
</h5>
</td>
<td>
<pre lang="python">
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
</pre>
</td>
<td>
<p>Multi-Threaded</p>
<pre lang="python">
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
single_ctx = ExecutionContext(context=exec_mode.multi_proc)
</pre>
<p>Single-Threaded</p>
<pre lang="python">
from cadCAD.engine import ExecutionMode, ExecutionContext
exec_mode = ExecutionMode()
multi_ctx = ExecutionContext(context=exec_mode.single_proc)
</pre>
</td>
</tr>
<tr>
<td>
<h5>
cadCAD Post-Processing Enhancements / Modifications
</h5>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
simulation = Executor(exec_context=local_ctx, configs=configs)
raw_result, sys_model, _ = simulation.execute()
result = pd.DataFrame(raw_result)
print(tabulate(result, headers='keys', tablefmt='psql'))
</pre>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
multi_ctx = ExecutionContext(context=exec_mode.multi_proc)
simulation = Executor(exec_context=multi_ctx, configs=configs)
i = 0
config_names = ['sys_model_A', 'sys_model_B']
for raw_result, _ in simulation.execute():
result = pd.DataFrame(raw_result)
print()
print(f"{config_names[i]} Result: System Events DataFrame:")
print(tabulate(result, headers='keys', tablefmt='psql'))
print()
i += 1
</pre>
</td>
</tr>
<tr>
<td>
<h5>
Expandable state and policy update parameter
</h5>
</td>
<td>
<pre lang="python">
def state_update(_params, substep, sH, s, _input, **kwargs):
…
return 'state_variable_name', new_value
def policy(_params, substep, sH, s, **kwargs):
…
return {'signal_1': value_1, …, 'signal_N': value_N}
</pre>
</td>
<td>
<pre lang="python">
def state_update(_params, substep, sH, s, _input):
…
return 'state_variable_name', new_value
def policy(_params, substep, sH, s):
…
return {'signal_1': value_1, …, 'signal_N': value_N}
</pre>
</td>
</tr>
</table>
================================================
FILE: documentation/cadCAD-v0.4.27-Model-Upgrade-Guide.md
================================================
<table>
<tr>
<th>
Feature
</th>
<th>
ver. 0.4.27
</th>
<th>
ver. 0.4.23
</th>
</tr>
<tr>
<td>
<h5>
Experiments & System Model Configurations
</h5>
</td>
<td>
<pre lang="python">
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_model(
model_id = 'sys_model_1', # System Model
initial_state = ..., # System Model
partial_state_update_blocks = ..., # System Model
policy_ops = ..., # System Model
sim_configs = ..., # Simulation Properties
)
exp.append_model(
model_id = 'sys_model_2', # System Model
...
)
configs = exp.configs
</pre>
</td>
<td>
<pre lang="python">
from cadCAD import configs
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_configs(...)
</pre>
</td>
</tr>
<tr>
<td>
<h5>
cadCAD Post-Processing Modifications
</h5>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
from simulations.regression_tests.experiments import multi_exp
from simulations.regression_tests.models import config_multi_1, config_multi_2
exec_mode = ExecutionMode()
local_proc_ctx = ExecutionContext(context=exec_mode.local_mode)
run = Executor(exec_context=local_proc_ctx, configs=multi_exp.configs)
raw_result, tensor_fields, _ = run.execute()
result = pd.DataFrame(raw_result)
print(tabulate(tensor_fields[0], headers='keys', tablefmt='psql'))
print(tabulate(result, headers='keys', tablefmt='psql'))
</pre>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
simulation = Executor(exec_context=local_ctx, configs=configs)
raw_result, sys_model, _ = simulation.execute()
result = pd.DataFrame(raw_result)
print(tabulate(result, headers='keys', tablefmt='psql'))
</pre>
</td>
</tr>
</table>
================================================
FILE: documentation/cadCAD-v0.4.28-Model-Upgrade-Guide.md
================================================
<table>
<tr>
<th>
Feature
</th>
<th>
ver. 0.4.28
</th>
<th>
ver. 0.4.23
</th>
</tr>
<tr>
<td>
<h5>
Experiments & System Model Configurations
</h5>
</td>
<td>
<pre lang="python">
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_model(
model_id = 'sys_model_1', # System Model - OPTIONAL
initial_state = ..., # System Model
partial_state_update_blocks = ..., # System Model
policy_ops = ..., # System Model
sim_configs = ..., # Simulation Properties
)
exp.append_model(...)
configs = exp.configs
</pre>
</td>
<td>
<pre lang="python">
from cadCAD import configs
from cadCAD.configuration import Experiment
exp = Experiment()
exp.append_configs(...)
</pre>
</td>
</tr>
<tr>
<td>
<h5>
cadCAD Post-Processing Modifications
</h5>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
from simulations.regression_tests.experiments import multi_exp
from simulations.regression_tests.models import config_multi_1, config_multi_2
exec_mode = ExecutionMode()
local_proc_ctx = ExecutionContext(context=exec_mode.local_mode)
run = Executor(exec_context=local_proc_ctx, configs=multi_exp.configs)
raw_result, tensor_fields, _ = run.execute()
result = pd.DataFrame(raw_result)
print(tabulate(tensor_fields[0], headers='keys', tablefmt='psql'))
print(tabulate(result, headers='keys', tablefmt='psql'))
</pre>
</td>
<td>
<pre lang="python">
import pandas as pd
from tabulate import tabulate
from cadCAD.engine import ExecutionMode, ExecutionContext, Executor
import system_model_A, system_model_B
from cadCAD import configs
exec_mode = ExecutionMode()
local_ctx = ExecutionContext(context=exec_mode.local_mode)
simulation = Executor(exec_context=local_ctx, configs=configs)
raw_result, sys_model, _ = simulation.execute()
result = pd.DataFrame(raw_result)
print(tabulate(result, headers='keys', tablefmt='psql'))
</pre>
</td>
</tr>
</table>
================================================
FILE: documentation/examples/__init__.py
================================================
from cadCAD.configuration import Experiment
system_model_AB_exp = Experiment()
================================================
FILE: documentation/examples/cadCAD_diagram.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def policy_1(p, s, h, v):\n",
" return {\"pi_1\": v[\"var_1\"]}\n",
"\n",
"\n",
"def policy_2(p, s, h, v):\n",
" return {\"pi_1\": v[\"var_1\"], \"pi_2\": v[\"var_1\"] * v[\"var_2\"]}\n",
"\n",
"\n",
"def suf_1(p, s, h, v, pi):\n",
" return (\"var_1\", pi[\"pi_1\"])\n",
"\n",
"\n",
"def suf_2(p, s, h, v, pi):\n",
" return (\"var_2\", pi[\"pi_2\"])\n",
"\n",
"\n",
"psubs = [\n",
" {\n",
" \"label\": \"Test\",\n",
" \"policies\": {\"policy_1\": policy_1, \"policy_2\": policy_2},\n",
" \"variables\": {\"var_1\": suf_1, \"var_2\": suf_2},\n",
" }\n",
"]\n",
"\n",
"initial_state = {\"var_1\": 0, \"var_2\": 1}\n",
"\n",
"params = {\"param_1\": 0, \"param_2\": 1}"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.append('..')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Generated by graphviz version 2.43.0 (0)\n",
" -->\n",
"<!-- Title: cluster_timestep Pages: 1 -->\n",
"<svg width=\"481pt\" height=\"183pt\"\n",
" viewBox=\"0.00 0.00 693.14 264.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
"<g id=\"graph0\" class=\"graph\" transform=\"scale(1.44 1.44) rotate(0) translate(4 260)\">\n",
"<title>cluster_timestep</title>\n",
"<polygon fill=\"pink\" stroke=\"transparent\" points=\"-4,4 -4,-260 689.14,-260 689.14,4 -4,4\"/>\n",
"<g id=\"clust1\" class=\"cluster\">\n",
"<title>cluster_psub_0</title>\n",
"<polygon fill=\"thistle\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"8,-8 8,-248 677.14,-248 677.14,-8 8,-8\"/>\n",
"<text text-anchor=\"middle\" x=\"342.57\" y=\"-232.8\" font-family=\"Times,serif\" font-size=\"14.00\">Partial State Update Block</text>\n",
"</g>\n",
"<g id=\"clust3\" class=\"cluster\">\n",
"<title>cluster_variables_0</title>\n",
"<polygon fill=\"skyblue\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"16,-88 16,-217 106,-217 106,-88 16,-88\"/>\n",
"<text text-anchor=\"middle\" x=\"61\" y=\"-201.8\" font-family=\"Times,serif\" font-size=\"14.00\">State</text>\n",
"</g>\n",
"<g id=\"clust4\" class=\"cluster\">\n",
"<title>cluster_policy_0</title>\n",
"<polygon fill=\"thistle\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"126,-16 126,-217 390.64,-217 390.64,-16 126,-16\"/>\n",
"<text text-anchor=\"middle\" x=\"258.32\" y=\"-201.8\" font-family=\"Times,serif\" font-size=\"14.00\">Policies</text>\n",
"</g>\n",
"<g id=\"clust5\" class=\"cluster\">\n",
"<title>cluster_suf_0</title>\n",
"<polygon fill=\"thistle\" stroke=\"black\" stroke-dasharray=\"5,2\" points=\"418.64,-16 418.64,-217 561.64,-217 561.64,-16 418.64,-16\"/>\n",
"<text text-anchor=\"middle\" x=\"490.14\" y=\"-201.8\" font-family=\"Times,serif\" font-size=\"14.00\">State Update Functions</text>\n",
"</g>\n",
"<!-- state_0 -->\n",
"<g id=\"node1\" class=\"node\">\n",
"<title>state_0</title>\n",
"<path fill=\"honeydew\" stroke=\"black\" d=\"M88,-74.73C88,-76.53 75.9,-78 61,-78 46.1,-78 34,-76.53 34,-74.73 34,-74.73 34,-45.27 34,-45.27 34,-43.47 46.1,-42 61,-42 75.9,-42 88,-43.47 88,-45.27 88,-45.27 88,-74.73 88,-74.73\"/>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M88,-74.73C88,-72.92 75.9,-71.45 61,-71.45 46.1,-71.45 34,-72.92 34,-74.73\"/>\n",
"<text text-anchor=\"middle\" x=\"61\" y=\"-56.3\" font-family=\"Times,serif\" font-size=\"14.00\">State 1</text>\n",
"</g>\n",
"<!-- policy_policy_1_0 -->\n",
"<g id=\"node5\" class=\"node\">\n",
"<title>policy_policy_1_0</title>\n",
"<polygon fill=\"palegreen\" stroke=\"black\" stroke-width=\"2\" points=\"246,-90 134,-90 134,-30 246,-30 258,-60 246,-90\"/>\n",
"<text text-anchor=\"middle\" x=\"196\" y=\"-56.3\" font-family=\"Times,serif\" font-size=\"14.00\">policy_1 (policy_1)</text>\n",
"</g>\n",
"<!-- state_0->policy_policy_1_0 -->\n",
"<g id=\"edge6\" class=\"edge\">\n",
"<title>state_0->policy_policy_1_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M88.28,-60C98.61,-60 111.02,-60 123.67,-60\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"123.82,-63.5 133.82,-60 123.82,-56.5 123.82,-63.5\"/>\n",
"</g>\n",
"<!-- policy_policy_2_0 -->\n",
"<g id=\"node6\" class=\"node\">\n",
"<title>policy_policy_2_0</title>\n",
"<polygon fill=\"palegreen\" stroke=\"black\" stroke-width=\"2\" points=\"246,-180 134,-180 134,-120 246,-120 258,-150 246,-180\"/>\n",
"<text text-anchor=\"middle\" x=\"196\" y=\"-146.3\" font-family=\"Times,serif\" font-size=\"14.00\">policy_2 (policy_2)</text>\n",
"</g>\n",
"<!-- state_0->policy_policy_2_0 -->\n",
"<g id=\"edge7\" class=\"edge\">\n",
"<title>state_0->policy_policy_2_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M88.43,-72.68C94.43,-76.03 100.62,-79.87 106,-84 116.22,-91.85 116.09,-96.76 126,-105 127.13,-105.94 128.29,-106.88 129.46,-107.82\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"127.34,-110.61 137.4,-113.92 131.61,-105.05 127.34,-110.61\"/>\n",
"</g>\n",
"<!-- variable_var_2_0 -->\n",
"<g id=\"node2\" class=\"node\">\n",
"<title>variable_var_2_0</title>\n",
"<path fill=\"honeydew\" stroke=\"black\" d=\"M98,-182.73C98,-184.53 81.42,-186 61,-186 40.58,-186 24,-184.53 24,-182.73 24,-182.73 24,-153.27 24,-153.27 24,-151.47 40.58,-150 61,-150 81.42,-150 98,-151.47 98,-153.27 98,-153.27 98,-182.73 98,-182.73\"/>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M98,-182.73C98,-180.92 81.42,-179.45 61,-179.45 40.58,-179.45 24,-180.92 24,-182.73\"/>\n",
"<text text-anchor=\"middle\" x=\"61\" y=\"-164.3\" font-family=\"Times,serif\" font-size=\"14.00\">var_2 (int)</text>\n",
"</g>\n",
"<!-- variable_var_2_0->policy_policy_2_0 -->\n",
"<g id=\"edge4\" class=\"edge\">\n",
"<title>variable_var_2_0->policy_policy_2_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M98.34,-163.08C106.33,-162 115.06,-160.82 123.9,-159.62\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"124.45,-163.08 133.89,-158.27 123.51,-156.14 124.45,-163.08\"/>\n",
"</g>\n",
"<!-- variable_var_1_0 -->\n",
"<g id=\"node3\" class=\"node\">\n",
"<title>variable_var_1_0</title>\n",
"<path fill=\"honeydew\" stroke=\"black\" d=\"M98,-128.73C98,-130.53 81.42,-132 61,-132 40.58,-132 24,-130.53 24,-128.73 24,-128.73 24,-99.27 24,-99.27 24,-97.47 40.58,-96 61,-96 81.42,-96 98,-97.47 98,-99.27 98,-99.27 98,-128.73 98,-128.73\"/>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M98,-128.73C98,-126.92 81.42,-125.45 61,-125.45 40.58,-125.45 24,-126.92 24,-128.73\"/>\n",
"<text text-anchor=\"middle\" x=\"61\" y=\"-110.3\" font-family=\"Times,serif\" font-size=\"14.00\">var_1 (int)</text>\n",
"</g>\n",
"<!-- variable_var_1_0->policy_policy_1_0 -->\n",
"<g id=\"edge3\" class=\"edge\">\n",
"<title>variable_var_1_0->policy_policy_1_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M98.34,-99.24C106.51,-95.93 115.44,-92.3 124.47,-88.64\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"125.94,-91.82 133.89,-84.81 123.31,-85.33 125.94,-91.82\"/>\n",
"</g>\n",
"<!-- variable_var_1_0->policy_policy_2_0 -->\n",
"<g id=\"edge5\" class=\"edge\">\n",
"<title>variable_var_1_0->policy_policy_2_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M98.34,-123.84C106.42,-126.02 115.25,-128.41 124.18,-130.83\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"123.32,-134.22 133.89,-133.46 125.15,-127.47 123.32,-134.22\"/>\n",
"</g>\n",
"<!-- agg_0 -->\n",
"<g id=\"node4\" class=\"node\">\n",
"<title>agg_0</title>\n",
"<ellipse fill=\"greenyellow\" stroke=\"black\" stroke-width=\"2\" cx=\"338.32\" cy=\"-105\" rx=\"44.15\" ry=\"44.15\"/>\n",
"<text text-anchor=\"middle\" x=\"338.32\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">Aggregation</text>\n",
"</g>\n",
"<!-- suf_var_1_0 -->\n",
"<g id=\"node7\" class=\"node\">\n",
"<title>suf_var_1_0</title>\n",
"<polygon fill=\"red\" stroke=\"black\" stroke-width=\"2\" points=\"522.64,-180 444.64,-180 444.64,-120 522.64,-120 534.64,-150 522.64,-180\"/>\n",
"<text text-anchor=\"middle\" x=\"489.64\" y=\"-146.3\" font-family=\"Times,serif\" font-size=\"14.00\">var_1 (suf_1)</text>\n",
"</g>\n",
"<!-- agg_0->suf_var_1_0 -->\n",
"<g id=\"edge8\" class=\"edge\">\n",
"<title>agg_0->suf_var_1_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M380.92,-117.54C397.59,-122.56 417.02,-128.42 434.75,-133.76\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"433.82,-137.13 444.4,-136.67 435.84,-130.43 433.82,-137.13\"/>\n",
"</g>\n",
"<!-- suf_var_2_0 -->\n",
"<g id=\"node8\" class=\"node\">\n",
"<title>suf_var_2_0</title>\n",
"<polygon fill=\"red\" stroke=\"black\" stroke-width=\"2\" points=\"522.64,-90 444.64,-90 444.64,-30 522.64,-30 534.64,-60 522.64,-90\"/>\n",
"<text text-anchor=\"middle\" x=\"489.64\" y=\"-56.3\" font-family=\"Times,serif\" font-size=\"14.00\">var_2 (suf_2)</text>\n",
"</g>\n",
"<!-- agg_0->suf_var_2_0 -->\n",
"<g id=\"edge9\" class=\"edge\">\n",
"<title>agg_0->suf_var_2_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M380.92,-92.46C397.59,-87.44 417.02,-81.58 434.75,-76.24\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"435.84,-79.57 444.4,-73.33 433.82,-72.87 435.84,-79.57\"/>\n",
"</g>\n",
"<!-- policy_policy_1_0->agg_0 -->\n",
"<g id=\"edge1\" class=\"edge\">\n",
"<title>policy_policy_1_0->agg_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M258.14,-79.61C267.47,-82.6 277.01,-85.66 286.1,-88.57\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"285.16,-91.95 295.75,-91.67 287.3,-85.28 285.16,-91.95\"/>\n",
"</g>\n",
"<!-- policy_policy_2_0->agg_0 -->\n",
"<g id=\"edge2\" class=\"edge\">\n",
"<title>policy_policy_2_0->agg_0</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M258.14,-130.39C267.47,-127.4 277.01,-124.34 286.1,-121.43\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"287.3,-124.72 295.75,-118.33 285.16,-118.05 287.3,-124.72\"/>\n",
"</g>\n",
"<!-- state_1 -->\n",
"<g id=\"node9\" class=\"node\">\n",
"<title>state_1</title>\n",
"<ellipse fill=\"none\" stroke=\"black\" cx=\"633.39\" cy=\"-105\" rx=\"36\" ry=\"18\"/>\n",
"<text text-anchor=\"middle\" x=\"633.39\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">state_1</text>\n",
"</g>\n",
"<!-- suf_var_1_0->state_1 -->\n",
"<g id=\"edge10\" class=\"edge\">\n",
"<title>suf_var_1_0->state_1</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M534.72,-136.01C553.37,-130.09 574.92,-123.24 593.03,-117.5\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"594.23,-120.79 602.7,-114.43 592.11,-114.12 594.23,-120.79\"/>\n",
"</g>\n",
"<!-- suf_var_2_0->state_1 -->\n",
"<g id=\"edge11\" class=\"edge\">\n",
"<title>suf_var_2_0->state_1</title>\n",
"<path fill=\"none\" stroke=\"black\" d=\"M534.72,-73.99C553.37,-79.91 574.92,-86.76 593.03,-92.5\"/>\n",
"<polygon fill=\"black\" stroke=\"black\" points=\"592.11,-95.88 602.7,-95.57 594.23,-89.21 592.11,-95.88\"/>\n",
"</g>\n",
"</g>\n",
"</svg>\n"
],
"text/plain": [
"<graphviz.dot.Digraph at 0x7f28a138d590>"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from cadCAD_diagram.config_diagram import diagram\n",
"\n",
"diagram(initial_state, params, psubs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: documentation/examples/cadCAD_tools_example.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
" # cadCAD Easy Run for the Minimal Prey & Predator model"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/opt/homebrew/Caskroom/miniconda/base/lib/python3.11/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"import plotly.express as px\n",
"import numpy as np\n",
"import sys\n",
"import seaborn as sns\n",
"from tqdm.auto import tqdm\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" ## Minimal Prey and Predator Model"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"TIMESTEPS = 100\n",
"SAMPLES = 10\n",
"\n",
"initial_conditions = {\n",
" 'prey_population': 100,\n",
" 'predator_population': 15\n",
"}\n",
"\n",
"params = {\n",
" \"prey_birth_rate\": [1.0],\n",
" \"predator_birth_rate\": [0.01],\n",
" \"predator_death_const\": [1.0],\n",
" \"prey_death_const\": [0.03],\n",
" # Precision of the simulation. Lower is more accurate / slower\n",
" \"dt\": [0.01, 0.1, 0.05]\n",
"}\n",
"\n",
"\n",
"def p_predator_births(params, step, sL, s):\n",
" dt = params['dt']\n",
" predator_population = s['predator_population']\n",
" prey_population = s['prey_population']\n",
" birth_fraction = params['predator_birth_rate'] + \\\n",
" np.random.random() * 0.0002\n",
" births = birth_fraction * prey_population * predator_population * dt\n",
" return {'add_to_predator_population': births}\n",
"\n",
"\n",
"def p_prey_births(params, step, sL, s):\n",
" dt = params['dt']\n",
" population = s['prey_population']\n",
" birth_fraction = params['prey_birth_rate'] + np.random.random() * 0.1\n",
" births = birth_fraction * population * dt\n",
" return {'add_to_prey_population': births}\n",
"\n",
"\n",
"def p_predator_deaths(params, step, sL, s):\n",
" dt = params['dt']\n",
" population = s['predator_population']\n",
" death_rate = params['predator_death_const'] + np.random.random() * 0.005\n",
" deaths = death_rate * population * dt\n",
" return {'add_to_predator_population': -1.0 * deaths}\n",
"\n",
"\n",
"def p_prey_deaths(params, step, sL, s):\n",
" dt = params['dt']\n",
" death_rate = params['prey_death_const'] + np.random.random() * 0.1\n",
" prey_population = s['prey_population']\n",
" predator_population = s['predator_population']\n",
" deaths = death_rate * prey_population * predator_population * dt\n",
" return {'add_to_prey_population': -1.0 * deaths}\n",
"\n",
"\n",
"def s_prey_population(params, step, sL, s, _input):\n",
" y = 'prey_population'\n",
" x = s['prey_population'] + _input['add_to_prey_population']\n",
" return (y, x)\n",
"\n",
"\n",
"def s_predator_population(params, step, sL, s, _input):\n",
" y = 'predator_population'\n",
" x = s['predator_population'] + _input['add_to_predator_population']\n",
" return (y, x)\n",
"\n",
"\n",
"partial_state_update_blocks = [\n",
" {\n",
" 'label': 'Lotka-Volterra Equations',\n",
" 'policies': {\n",
" 'predator_births': p_predator_births,\n",
" 'prey_births': p_prey_births,\n",
" 'predator_deaths': p_predator_deaths,\n",
" 'prey_deaths': p_prey_deaths,\n",
" },\n",
" 'variables': {\n",
" 'prey_population': s_prey_population,\n",
" 'predator_population': s_predator_population\n",
" }\n",
" },\n",
" {\n",
" 'label': 'Do Nothing',\n",
" 'policies': {\n",
" \n",
" },\n",
" 'variables': {\n",
" \n",
" }\n",
" }\n",
"]\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.append(\"../..\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
" ___________ ____\n",
" ________ __ ___/ / ____/ | / __ \\\n",
" / ___/ __` / __ / / / /| | / / / /\n",
"/ /__/ /_/ / /_/ / /___/ ___ |/ /_/ /\n",
"\\___/\\__,_/\\__,_/\\____/_/ |_/_____/\n",
"by cadCAD\n",
"\n",
"cadCAD Version: 0.4.28\n",
"Execution Mode: local_proc\n",
"Simulation Dimensions:\n",
"Entire Simulation: (Models, Unique Timesteps, Params, Total Runs, Sub-States) = (1, 100, 5, 30, 2)\n",
" Simulation 0: (Timesteps, Params, Runs, Sub-States) = (100, 5, 30, 2)\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Initializing configurations: 100%|██████████| 30/30 [00:00<00:00, 669.03it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Execution Method: parallelize_simulations\n",
"Execution Mode: parallelized\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"Flattening results: 100%|██████████| 30/30 [00:00<00:00, 324.56it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Total execution time: 0.86s\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
}
],
"source": [
"from cadCAD.tools import easy_run\n",
"\n",
"df = easy_run(initial_conditions,\n",
" params,\n",
" partial_state_update_blocks,\n",
" TIMESTEPS,\n",
" SAMPLES,\n",
" assign_params=True,\n",
" drop_substeps=False)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"plotlyServerURL": "https://plot.ly"
},
"data": [
{
"hovertemplate": "color=1<br>x=%{x}<br>y=%{y}<extra></extra>",
"legendgroup": "1",
"line": {
"color": "#636efa",
"dash": "solid"
},
"marker": {
"symbol": "circle"
},
"mode": "lines",
"name": "1",
"showlegend": true,
"type": "scattergl",
"x": [
100,
99.29993160631898,
99.29993160631898,
99.38148169571427,
99.38148169571427,
99.33344287701127,
99.33344287701127,
99.78052429956195,
99.78052429956195,
99.54958970085868,
99.54958970085868,
99.5415288602601,
99.5415288602601,
99.20043010937233,
99.20043010937233,
98.38175972088112,
98.38175972088112,
98.08171337358606,
98.08171337358606,
98.3146135545321,
98.3146135545321,
98.15163483592876,
98.15163483592876,
98.2617316271704,
98.2617316271704,
98.4307104729456,
98.4307104729456,
98.78953357215462,
98.78953357215462,
98.79985727776051,
98.79985727776051,
99.23692266333096,
99.23692266333096,
98.39279376646044,
98.39279376646044,
98.38874784974972,
98.38874784974972,
98.47174801810372,
98.47174801810372,
98.2916973612909,
98.2916973612909,
98.77808827688989,
98.77808827688989,
98.74409721517681,
98.74409721517681,
99.24190042277999,
99.24190042277999,
98.84599456847697,
98.84599456847697,
99.09135177709153,
99.09135177709153,
99.10434894166539,
99.10434894166539,
99.57328874254877,
99.57328874254877,
99.21501237298247,
99.21501237298247,
99.25020700392601,
99.25020700392601,
99.63288351336783,
99.63288351336783,
99.40473856709445,
99.40473856709445,
98.7710291940686,
98.7710291940686,
98.5407493936721,
98.5407493936721,
97.7508566239193,
97.7508566239193,
97.71072647387824,
97.71072647387824,
97.06612114764668,
97.06612114764668,
96.60023270185529,
96.60023270185529,
96.97950519720334,
96.97950519720334,
97.13875284807816,
97.13875284807816,
97.28197178315625,
97.28197178315625,
96.58033571299049,
96.58033571299049,
96.25643172904921,
96.25643172904921,
96.6261400000404,
96.6261400000404,
96.52079469759674,
96.52079469759674,
96.14265950886788,
96.14265950886788,
96.29506225210642,
96.29506225210642,
95.82910831575578,
95.82910831575578,
96.35112441973347,
96.35112441973347,
96.16329269178928,
96.16329269178928,
95.87027246972195,
95.87027246972195,
95.7307727305111,
95.7307727305111,
95.52782525481386,
95.52782525481386,
95.3266966060663,
95.3266966060663,
94.73244989096955,
94.73244989096955,
94.85863986535756,
94.85863986535756,
94.5348526851931,
94.5348526851931,
94.27697635190073,
94.27697635190073,
93.47026915554588,
93.47026915554588,
93.23682795968988,
93.23682795968988,
93.37231139151324,
93.37231139151324,
93.15534439539415,
93.15534439539415,
93.24217287585641,
93.24217287585641,
93.78860723896459,
93.78860723896459,
93.77560714675685,
93.77560714675685,
93.0999726791823,
93.0999726791823,
92.90606801326578,
92.90606801326578,
93.2831221906634,
93.2831221906634,
93.01583352672039,
93.01583352672039,
92.70359835023112,
92.70359835023112,
92.00530036542673,
92.00530036542673,
92.42100834060409,
92.42100834060409,
92.7895835919253,
92.7895835919253,
92.73717490018763,
92.73717490018763,
92.76883196952322,
92.76883196952322,
93.08317487684629,
93.08317487684629,
92.33451766496589,
92.33451766496589,
92.13498897315472,
92.13498897315472,
92.09928886790658,
92.09928886790658,
92.22680948420437,
92.22680948420437,
91.62546599412939,
91.62546599412939,
91.12033628158707,
91.12033628158707,
90.91703470428618,
90.91703470428618,
90.51203613658468,
90.51203613658468,
90.76461607129751,
90.76461607129751,
91.12216656725413,
91.12216656725413,
91.25249379875453,
91.25249379875453,
91.58012004487541,
91.58012004487541,
91.06486472813755,
91.06486472813755,
91.18797649222013,
91.18797649222013,
90.69671638734081,
90.69671638734081,
91.26790712942591,
91.26790712942591,
90.79702047570592,
90.79702047570592,
90.23186056687439,
90.23186056687439,
89.67634735880623,
89.67634735880623,
90.06987166421722,
90.06987166421722,
89.65940405481305,
89.65940405481305,
89.14786912937605,
89.14786912937605,
88.68086196244481,
88.68086196244481,
88.08424380115036,
88.08424380115036,
87.92914636627808,
87.92914636627808,
100,
92.99931606318984,
92.99931606318984,
93.74968330870446,
93.74968330870446,
93.35200242595599,
93.35200242595599,
97.61217815984494,
97.61217815984494,
95.54997817570786,
95.54997817570786,
95.635092884268,
95.635092884268,
92.61880754039443,
92.61880754039443,
85.39231796273906,
85.39231796273906,
83.12912862094319,
83.12912862094319,
85.40561642328275,
85.40561642328275,
84.62039860344953,
84.62039860344953,
86.16474388636449,
86.16474388636449,
88.32785188924211,
88.32785188924211,
92.10765900299248,
92.10765900299248,
93.26077770768009,
93.26077770768009,
97.9845294627281,
97.9845294627281,
91.85128228934461,
91.85128228934461,
92.94825282196534,
92.94825282196534,
94.908060967487,
94.908060967487,
94.71326538271349,
94.71326538271349,
100.13862169564187,
100.13862169564187,
101.27435286478284,
101.27435286478284,
107.18723419784558,
107.18723419784558,
105.11150551338793,
105.11150551338793,
108.89332418604562,
108.89332418604562,
110.45800926706632,
110.45800926706632,
116.48523146511381,
116.48523146511381,
114.1292438507272,
114.1292438507272,
115.6710564645791,
115.6710564645791,
120.72623435030194,
120.72623435030194,
118.93046533906094,
118.93046533906094,
112.25835075871757,
112.25835075871757,
110.00527663324144,
110.00527663324144,
101.43663074912372,
101.43663074912372,
101.02867217786005,
101.02867217786005,
94.33053060847377,
94.33053060847377,
89.75476827975842,
89.75476827975842,
93.29245542951959,
93.29245542951959,
94.9186805787271,
94.9186805787271,
96.46551527277536,
96.46551527277536,
89.87389035969865,
89.87389035969865,
87.16376777592785,
87.16376777592785,
90.71447055509934,
90.71447055509934,
90.19536558085994,
90.19536558085994,
87.37997504032674,
87.37997504032674,
89.23463194487961,
89.23463194487961,
85.88495474723042,
85.88495474723042,
90.91229934650016,
90.91229934650016,
90.15196996137051,
90.15196996137051,
88.67602042222708,
88.67602042222708,
88.50743002514825,
88.50743002514825,
87.96325417167918,
87.96325417167918,
87.47813607760847,
87.47813607760847,
84.01053269460503,
84.01053269460503,
86.21927941269837,
86.21927941269837,
85.20026287932083,
85.20026287932083,
84.74514940369397,
84.74514940369397,
80.44680552355874,
80.44680552355874,
80.41520185078912,
80.41520185078912,
83.13776682112793,
83.13776682112793,
83.62311825579216,
83.62311825579216,
86.27091768763398,
86.27091768763398,
92.29068344923925,
92.29068344923925,
94.59113655600862,
94.59113655600862,
92.23253576603186,
92.23253576603186,
93.36203549432568,
93.36203549432568,
98.80311561440442,
98.80311561440442,
99.52383859763862,
99.52383859763862,
100.02610961464065,
100.02610961464065,
97.48148429129367,
97.48148429129367,
103.34545533154333,
103.34545533154333,
109.43020057074936,
109.43020057074936,
112.12304973808557,
112.12304973808557,
115.52003033279902,
115.52003033279902,
121.55801868602819,
121.55801868602819,
117.08958909341867,
117.08958909341867,
117.75922193520242,
117.75922193520242,
119.85458925018938,
119.85458925018938,
123.66274001251523,
123.66274001251523,
119.15604184133105,
119.15604184133105,
115.53291901293304,
115.53291901293304,
114.89363074478746,
114.89363074478746,
111.82198288405428,
111.82198288405428,
115.87910014654939,
115.87910014654939,
121.11836567425608,
121.11836567425608,
123.6936575690804,
123.6936575690804,
128.58085298226052,
128.58085298226052,
121.95792293344105,
121.95792293344105,
123.60374048079915,
123.60374048079915,
116.48935752258258,
116.48935752258258,
123.55808091733965,
123.55808091733965,
115.82597098962636,
115.82597098962636,
106.77349871746914,
106.77349871746914,
98.16867422379417,
98.16867422379417,
101.74594768140646,
101.74594768140646,
95.1243047124417,
95.1243047124417,
87.65512294066052,
87.65512294066052,
81.34224880318148,
81.34224880318148,
74.25512295493493,
74.25512295493493,
72.08158085400078,
72.08158085400078,
100,
96.49965803159492,
96.49965803159492,
96.89282234085528,
96.89282234085528,
96.66395700156504,
96.66395700156504,
98.84511851049147,
98.84511851049147,
97.72022468264952,
97.72022468264952,
97.69539195252099,
97.69539195252099,
96.04474200505408,
96.04474200505408,
92.12122707795169,
92.12122707795169,
90.7524578823741,
90.7524578823741,
91.86620599866295,
91.86620599866295,
91.18321550679761,
91.18321550679761,
91.77076300242805,
91.77076300242805,
92.64886983215455,
92.64886983215455,
94.4115531951917,
94.4115531951917,
94.60105803410664,
94.60105803410664,
96.77426923210655,
96.77426923210655,
92.95584004241334,
92.95584004241334,
93.0988602155345,
93.0988602155345,
93.66372732853505,
93.66372732853505,
93.03719912084956,
93.03719912084956,
95.452716939988,
95.452716939988,
95.51738150606599,
95.51738150606599,
98.05425788440175,
98.05425788440175,
96.45618107548292,
96.45618107548292,
97.85694974565604,
97.85694974565604,
98.17927424291095,
98.17927424291095,
100.65800191623217,
100.65800191623217,
99.23013457226213,
99.23013457226213,
99.68075990493776,
99.68075990493776,
101.76892968159176,
101.76892968159176,
100.93152794995164,
100.93152794995164,
98.15507445203599,
98.15507445203599,
97.32448920917646,
97.32448920917646,
93.8712393709979,
93.8712393709979,
93.94055110646606,
93.94055110646606,
91.25561322951836,
91.25561322951836,
89.45152914637421,
89.45152914637421,
91.38648456820863,
91.38648456820863,
92.40702828733023,
92.40702828733023,
93.37466906480901,
93.37466906480901,
90.60421743023397,
90.60421743023397,
89.54727537765943,
89.54727537765943,
91.50562818437629,
91.50562818437629,
91.43629555560625,
91.43629555560625,
90.22387819072098,
90.22387819072098,
91.286734506583,
91.286734506583,
89.70792037849891,
89.70792037849891,
92.36523442648821,
92.36523442648821,
92.01299752297405,
92.01299752297405,
91.26989588133198,
91.26989588133198,
91.16338302394053,
91.16338302394053,
90.8266416539283,
90.8266416539283,
90.48973346931841,
90.48973346931841,
88.53135087902336,
88.53135087902336,
89.58435442398849,
89.58435442398849,
88.8253866031861,
88.8253866031861,
88.3389425664917,
88.3389425664917,
85.6690510129246,
85.6690510129246,
85.33734615018504,
85.33734615018504,
86.51603920143914,
86.51603920143914,
86.33828493348129,
86.33828493348129,
87.36304955582791,
87.36304955582791,
90.23284652388249,
90.23284652388249,
90.91718361444943,
90.91718361444943,
88.99242906220593,
88.99242906220593,
88.99938883525597,
88.99938883525597,
91.30743180141197,
91.30743180141197,
91.05295854841216,
91.05295854841216,
90.67174729743687,
90.67174729743687,
88.7604154487307,
88.7604154487307,
91.21667934978741,
91.21667934978741,
93.6338256538187,
93.6338256538187,
94.37558676757705,
94.37558676757705,
95.47321941160612,
95.47321941160612,
97.77075917390104,
97.77075917390104,
95.59282238220855,
95.59282238220855,
95.72059903293992,
95.72059903293992,
96.52854695498216,
96.52854695498216,
98.09563460829936,
98.09563460829936,
96.51532722863074,
96.51532722863074,
95.4085247939158,
95.4085247939158,
95.52465821995634,
95.52465821995634,
94.8087017899729,
94.8087017899729,
96.88999312951138,
96.88999312951138,
99.41181607140739,
99.41181607140739,
101.0349689701463,
101.0349689701463,
103.52291722549926,
103.52291722549926,
102.29402709966891,
102.29402709966891,
103.89599855840412,
103.89599855840412,
102.66305598951656,
102.66305598951656,
106.35128198458621,
106.35128198458621,
105.2153347445648,
105.2153347445648,
103.54761969318548,
103.54761969318548,
101.9104419064554,
101.9104419064554,
104.68762728033715,
104.68762728033715,
103.74561762296544,
103.74561762296544,
102.29177188240789,
102.29177188240789,
100.99065500957371,
100.99065500957371,
99.08829773238939,
99.08829773238939,
99.25050915772879,
99.25050915772879
],
"xaxis": "x",
"y": [
15,
15.0025881631884,
15.0025881631884,
15.001496702185737,
15.001496702185737,
15.002361804039465,
15.002361804039465,
15.003883708196188,
15.003883708196188,
15.00568393679195,
15.00568393679195,
15.007415489103332,
15.007415489103332,
15.007679903819312,
15.007679903819312,
15.00785846437848,
15.00785846437848,
15.005149988731267,
15.005149988731267,
15.003693218370183,
15.003693218370183,
15.003201208353792,
15.003201208353792,
15.001441720795848,
gitextract_5t5ec52q/
├── .github/
│ ├── FUNDING.yml
│ └── workflows/
│ ├── cadcad-ci.yml
│ └── cadcad-publish.yml
├── .gitignore
├── AUTHORS.txt
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE.txt
├── README.md
├── ascii_art.py
├── cadCAD/
│ ├── __init__.py
│ ├── configuration/
│ │ ├── __init__.py
│ │ └── utils/
│ │ ├── __init__.py
│ │ ├── depreciationHandler.py
│ │ ├── policyAggregation.py
│ │ └── userDefinedObject.py
│ ├── diagram/
│ │ ├── __init__.py
│ │ └── config_diagram.py
│ ├── engine/
│ │ ├── __init__.py
│ │ ├── execution.py
│ │ ├── simulation.py
│ │ └── utils.py
│ ├── tools/
│ │ ├── __init__.py
│ │ ├── execution/
│ │ │ ├── __init__.py
│ │ │ └── easy_run.py
│ │ ├── preparation.py
│ │ ├── profiling/
│ │ │ ├── __init__.py
│ │ │ ├── profile_run.py
│ │ │ └── visualizations.py
│ │ ├── types.py
│ │ └── utils.py
│ ├── types.py
│ └── utils/
│ ├── __init__.py
│ ├── execution.py
│ ├── jupyter.py
│ └── sys_config.py
├── documentation/
│ ├── Historically_State_Access.md
│ ├── Policy_Aggregation.md
│ ├── README.md
│ ├── Simulation_Execution.md
│ ├── System_Configuration.md
│ ├── System_Model_Parameter_Sweep.md
│ ├── __init__.py
│ ├── cadCAD-v0.4.23-Model-Upgrade-Guide.md
│ ├── cadCAD-v0.4.27-Model-Upgrade-Guide.md
│ ├── cadCAD-v0.4.28-Model-Upgrade-Guide.md
│ └── examples/
│ ├── __init__.py
│ ├── cadCAD_diagram.ipynb
│ ├── cadCAD_tools_example.ipynb
│ ├── example_1.py
│ ├── historical_state_access.py
│ ├── param_sweep.py
│ ├── policy_aggregation.py
│ ├── sys_model_A.py
│ ├── sys_model_AB_exec.py
│ ├── sys_model_A_exec.py
│ ├── sys_model_B.py
│ └── sys_model_B_exec.py
├── expected_results/
│ ├── param_sweep_4.pkl
│ ├── param_sweep_5.pkl
│ ├── param_sweep_psub0_4.pkl
│ ├── param_sweep_timestep1_4.pkl
│ ├── policy_agg_4.pkl
│ └── policy_agg_5.pkl
├── requirements.txt
├── setup.py
└── testing/
├── __init__.py
├── experiments/
│ └── __init__.py
├── generic_test.py
├── models/
│ ├── __init__.py
│ ├── param_sweep.py
│ └── policy_aggregation.py
├── results_comparison.py
├── test_additional_objs.py
├── test_arg_count.py
├── test_param_count.py
├── test_print.py
├── test_results_signature.py
├── test_row_count.py
├── test_runs.py
├── tests/
│ ├── __init__.py
│ ├── a_b_tests/
│ │ ├── 0_4_23_record_count.json
│ │ └── multi_model_row_count_0_4_23.py
│ ├── append_mod_test.py
│ ├── expected_results/
│ │ ├── param_sweep_4.pkl
│ │ ├── param_sweep_5.pkl
│ │ ├── param_sweep_psub0_4.pkl
│ │ ├── param_sweep_timestep1_4.pkl
│ │ ├── policy_agg_4.pkl
│ │ └── policy_agg_5.pkl
│ ├── import_cadCAD.ipynb
│ ├── test_cadCAD_exp.py
│ ├── test_import_cadCAD_test.py
│ ├── test_multi_model_row_count.py
│ ├── test_param_sweep.py
│ ├── test_policy_aggregation.py
│ ├── test_run1psub0.py
│ ├── test_runs_not_zero.py
│ └── test_timestep1psub0.py
├── tools/
│ └── test_tools.py
└── utils.py
SYMBOL INDEX (305 symbols across 48 files)
FILE: cadCAD/configuration/__init__.py
class Configuration (line 13) | class Configuration():
method __init__ (line 14) | def __init__(self, user_id, model_id, subset_id, subset_window, sim_co...
class Experiment (line 46) | class Experiment(object):
method __init__ (line 47) | def __init__(self):
method append_model (line 62) | def append_model(
class Identity (line 181) | class Identity:
method __init__ (line 182) | def __init__(self, policy_id: Dict[str, int] = {'identity': 0}) -> None:
method p_identity (line 185) | def p_identity(self, var_dict, sub_step, sL, s, **kwargs):
method policy_identity (line 188) | def policy_identity(self, k: str) -> callable:
method no_state_identity (line 191) | def no_state_identity(self, var_dict, sub_step, sL, s, _input, **kwargs):
method state_identity (line 194) | def state_identity(self, k: str) -> callable:
method apply_identity_funcs (line 199) | def apply_identity_funcs(self,
class Processor (line 212) | class Processor:
method __init__ (line 213) | def __init__(self, id: Identity = Identity()) -> None:
method create_matrix_field (line 221) | def create_matrix_field(self, partial_state_updates, key: str) -> Data...
method generate_config (line 234) | def generate_config(self, initial_state, partial_state_updates, exo_proc
FILE: cadCAD/configuration/utils/__init__.py
class TensorFieldReport (line 13) | class TensorFieldReport:
method __init__ (line 14) | def __init__(self, config_proc):
method create_tensor_field (line 17) | def create_tensor_field(self, partial_state_updates, exo_proc, keys=['...
function configs_as_spec (line 29) | def configs_as_spec(configs):
function configs_as_objs (line 44) | def configs_as_objs(configs):
function configs_as_dicts (line 51) | def configs_as_dicts(configs):
function configs_as_dataframe (line 58) | def configs_as_dataframe(configs):
function state_update (line 70) | def state_update(y, x):
function policy (line 73) | def policy(y, x):
function bound_norm_random (line 76) | def bound_norm_random(rng, low, high):
function time_step (line 85) | def time_step(dt_str, dt_format='%Y-%m-%d %H:%M:%S', _timedelta = tstep_...
function ep_time_step (line 92) | def ep_time_step(s_condition, dt_str, fromat_str='%Y-%m-%d %H:%M:%S', _t...
function exo_update_per_ts (line 99) | def exo_update_per_ts(ep):
function trigger_condition (line 110) | def trigger_condition(s, pre_conditions, cond_opp):
function apply_state_condition (line 115) | def apply_state_condition(pre_conditions, cond_opp, y, f, _g, step, sL, ...
function var_trigger (line 129) | def var_trigger(y, f, pre_conditions, cond_op):
function var_substep_trigger (line 134) | def var_substep_trigger(substeps):
function env_trigger (line 143) | def env_trigger(end_substep):
function config_sim (line 164) | def config_sim(config_dict: ConfigurationDict):
function psub_list (line 201) | def psub_list(psu_block, psu_steps):
function psub (line 205) | def psub(policies, state_updates):
function genereate_psubs (line 212) | def genereate_psubs(policy_grid, states_grid, policies, state_updates):
function access_block (line 222) | def access_block(state_history, target_field, psu_block_offset, exculsio...
function partial_state_sweep_filter (line 241) | def partial_state_sweep_filter(state_field, partial_state_updates):
function state_sweep_filter (line 249) | def state_sweep_filter(raw_exogenous_states):
function sweep_partial_states (line 254) | def sweep_partial_states(_type, in_config):
function sweep_states (line 272) | def sweep_states(state_type, states, in_config):
FILE: cadCAD/configuration/utils/depreciationHandler.py
function sanitize_config (line 4) | def sanitize_config(config):
function sanitize_partial_state_updates (line 17) | def sanitize_partial_state_updates(partial_state_updates):
FILE: cadCAD/configuration/utils/policyAggregation.py
function get_base_value (line 1) | def get_base_value(x):
function policy_to_dict (line 12) | def policy_to_dict(v):
FILE: cadCAD/configuration/utils/userDefinedObject.py
function val_switch (line 8) | def val_switch(v):
class udcView (line 15) | class udcView(object):
method __init__ (line 16) | def __init__(self, d, masked_members):
method __repr__ (line 20) | def __repr__(self):
class udcBroker (line 32) | class udcBroker(object):
method __init__ (line 33) | def __init__(self, obj, function_filter=['__init__']):
method get_members (line 43) | def get_members(self):
method get_view (line 46) | def get_view(self, masked_members):
method get_namedtuple (line 49) | def get_namedtuple(self):
function UDO (line 53) | def UDO(udo, masked_members=['obj']):
function udoPipe (line 57) | def udoPipe(obj_view):
FILE: cadCAD/diagram/config_diagram.py
function extract_var_key (line 9) | def extract_var_key(raw_line: str, var_id: str) -> str:
function extract_vars_from_source (line 29) | def extract_vars_from_source(source: str, var_id: str) -> set:
function extract_keys (line 48) | def extract_keys(f: callable) -> dict:
function relate_psub (line 63) | def relate_psub(psub: dict) -> dict:
function generate_relations (line 93) | def generate_relations(psubs) -> list:
function generate_time_graph (line 105) | def generate_time_graph() -> Digraph:
function generate_variables_cluster (line 111) | def generate_variables_cluster(variables: dict, i: int, suffix="") -> Di...
function generate_params_cluster (line 127) | def generate_params_cluster(params: dict, i: int) -> Digraph:
function generate_psub_graph (line 143) | def generate_psub_graph(i: int):
function relate_params (line 154) | def relate_params(graph: Digraph, params, i, origin=-1) -> Digraph:
function generate_policies_cluster (line 162) | def generate_policies_cluster(policies: dict, i: int, psub_graph) -> Dig...
function relate (line 189) | def relate(
function generate_sufs_cluster (line 203) | def generate_sufs_cluster(sufs: dict, i: int, psub_graph, agg=False) -> ...
function relate_params_to_sufs (line 223) | def relate_params_to_sufs(graph, sufs, i) -> Digraph:
function diagram (line 232) | def diagram(initial_state, params, psubs):
function diagram_from_config (line 292) | def diagram_from_config(config):
FILE: cadCAD/engine/__init__.py
class ExecutionMode (line 19) | class ExecutionMode:
function auto_mode_switcher (line 29) | def auto_mode_switcher(config_amt: int):
class ExecutionContext (line 40) | class ExecutionContext:
method __init__ (line 41) | def __init__(self, context=ExecutionMode.local_mode, method=None, addi...
class Executor (line 71) | class Executor:
method __init__ (line 72) | def __init__(self,
method execute (line 84) | def execute(self) -> Tuple[object, object, Dict[str, object]]:
FILE: cadCAD/engine/execution.py
function single_proc_exec (line 13) | def single_proc_exec(
function parallelize_simulations (line 46) | def parallelize_simulations(
function local_simulations (line 108) | def local_simulations(
FILE: cadCAD/engine/simulation.py
function policy_scope_tuner (line 15) | def policy_scope_tuner(args: tuple,
function compose (line 28) | def compose(init_reduction_funct: Aggregator,
class Executor (line 45) | class Executor:
method __init__ (line 46) | def __init__(
method get_policy_input (line 57) | def get_policy_input(
method apply_env_proc (line 113) | def apply_env_proc(
method partial_state_update (line 147) | def partial_state_update(
method state_update_pipeline (line 210) | def state_update_pipeline(
method run_pipeline (line 245) | def run_pipeline(
method simulation (line 267) | def simulation(
FILE: cadCAD/engine/utils.py
function datetime_range (line 4) | def datetime_range(start, end, delta, dt_format='%Y-%m-%d %H:%M:%S'):
function last_index (line 18) | def last_index(l):
function retrieve_state (line 22) | def retrieve_state(l, offset):
function engine_exception (line 27) | def engine_exception(ErrorType, error_message, exception_function, try_f...
FILE: cadCAD/tools/execution/easy_run.py
function describe_or_return (line 11) | def describe_or_return(v: object) -> object:
function select_M_dict (line 25) | def select_M_dict(M_dict: Dict[str, object], keys: set) -> Dict[str, obj...
function select_config_M_dict (line 32) | def select_config_M_dict(configs: list, i: int, keys: set) -> Dict[str, ...
function easy_run (line 36) | def easy_run(
FILE: cadCAD/tools/preparation.py
function sweep_cartesian_product (line 10) | def sweep_cartesian_product(sweep_params: SweepableParameters) -> Sweepa...
function prepare_params (line 33) | def prepare_params(params: SystemParameters,
function prepare_state (line 51) | def prepare_state(state: InitialState) -> Mapping[str, object]:
class ConfigurationWrapper (line 58) | class ConfigurationWrapper():
method run (line 65) | def run(self, *args, **kwargs) -> DataFrame:
FILE: cadCAD/tools/profiling/profile_run.py
function MEASURE_TIME_SUF (line 8) | def MEASURE_TIME_SUF(p, s, h, v, p_i): return ('run_time', time())
function profile_psubs (line 20) | def profile_psubs(psubs: StateUpdateBlocks, profile_substeps=True) -> St...
function profile_run (line 35) | def profile_run(state_variables: State,
FILE: cadCAD/tools/profiling/visualizations.py
function visualize_elapsed_time_per_ts (line 7) | def visualize_elapsed_time_per_ts(df: pd.DataFrame, relative=False) -> N...
function visualize_substep_impact (line 36) | def visualize_substep_impact(df: pd.DataFrame, relative=True, **kwargs) ...
FILE: cadCAD/tools/types.py
class InitialValue (line 3) | class InitialValue(NamedTuple):
class Param (line 8) | class Param(NamedTuple):
class ParamSweep (line 13) | class ParamSweep(NamedTuple):
FILE: cadCAD/tools/utils.py
function generic_suf (line 5) | def generic_suf(variable: str, signal: str = "") -> StateUpdateFunction:
function add_parameter_labels (line 22) | def add_parameter_labels(configs: list, df: pd.DataFrame) -> pd.DataFrame:
FILE: cadCAD/types.py
class StateUpdateBlock (line 15) | class StateUpdateBlock(TypedDict):
class ConfigurationDict (line 22) | class ConfigurationDict(TypedDict):
class SessionDict (line 41) | class SessionDict(TypedDict):
FILE: cadCAD/utils/__init__.py
class SilentDF (line 15) | class SilentDF(DataFrame):
method __repr__ (line 16) | def __repr__(self):
function append_dict (line 20) | def append_dict(dict: dict, new_dict: dict) -> dict:
function arrange_cols (line 29) | def arrange_cols(df: DataFrame, reverse=False) -> DataFrame:
class IndexCounter (line 41) | class IndexCounter:
method __init__ (line 42) | def __init__(self):
method __call__ (line 45) | def __call__(self):
function compose (line 50) | def compose(*functions: Tuple[callable]) -> object:
function pipe (line 54) | def pipe(x: object) -> object:
function print_pipe (line 58) | def print_pipe(x: object) -> object:
function tupalize (line 63) | def tupalize(k: object, vs: Union[list, dict]):
function flattenDict (line 78) | def flattenDict(l: dict) -> list:
function flatten (line 88) | def flatten(l: Union[list, dict]):
function flatMap (line 95) | def flatMap(f, collection):
function dict_filter (line 99) | def dict_filter(dictionary, condition):
function get_max_dict_val_len (line 103) | def get_max_dict_val_len(g: Dict[str, List[int]]) -> int:
function tabulate_dict (line 107) | def tabulate_dict(d: Dict[str, List[int]]) -> Dict[str, List[int]]:
function flatten_tabulated_dict (line 119) | def flatten_tabulated_dict(d: Dict[str, List[int]]) -> List[dict[str, in...
function contains_type (line 130) | def contains_type(_collection, type):
function drop_right (line 134) | def drop_right(l, n):
function key_filter (line 138) | def key_filter(l, keyname):
function groupByKey (line 149) | def groupByKey(l):
function rename (line 157) | def rename(new_name, f):
function curry_pot (line 162) | def curry_pot(f, *argv):
FILE: cadCAD/utils/execution.py
function print_exec_info (line 7) | def print_exec_info(exec_context, configs):
FILE: cadCAD/utils/jupyter.py
function get_home_dir (line 1) | def get_home_dir(user):
function set_write_path (line 4) | def set_write_path(sc, user, datafolder_path):
FILE: cadCAD/utils/sys_config.py
function increment (line 5) | def increment(y, incr_by):
function track (line 9) | def track(y):
function simple_state_update (line 13) | def simple_state_update(y, x):
function simple_policy_update (line 17) | def simple_policy_update(y):
function update_timestamp (line 21) | def update_timestamp(y, timedelta, format):
function apply (line 28) | def apply(f, y: str, incr_by: int):
function add (line 32) | def add(y: str, incr_by):
function increment_state_by_int (line 36) | def increment_state_by_int(y: str, incr_by: int):
function s (line 40) | def s(y, x):
function time_model (line 44) | def time_model(y, substeps, time_delta, ts_format='%Y-%m-%d %H:%M:%S'):
FILE: documentation/examples/historical_state_access.py
function last_update (line 14) | def last_update(_g, substep, sH, s):
function second2last_update (line 24) | def second2last_update(_g, substep, sH, s):
function add (line 32) | def add(y, x):
function nonexsistant (line 37) | def nonexsistant(_g, substep, sH, s, _input):
function last_x (line 42) | def last_x(_g, substep, sH, s, _input):
function second_to_last_x (line 47) | def second_to_last_x(_g, substep, sH, s, _input):
function third_to_last_x (line 52) | def third_to_last_x(_g, substep, sH, s, _input):
function fourth_to_last_x (line 57) | def fourth_to_last_x(_g, substep, sH, s, _input):
FILE: documentation/examples/param_sweep.py
function some_function (line 13) | def some_function(x):
function gamma (line 32) | def gamma(_params, step, sH, s):
function omega (line 36) | def omega(_params, step, sH, s):
function alpha (line 41) | def alpha(_params, step, sH, s, _input):
function alpha_plus_gamma (line 44) | def alpha_plus_gamma(_params, step, sH, s, _input):
function beta (line 48) | def beta(_params, step, sH, s, _input):
function policies (line 52) | def policies(_params, step, sH, s, _input):
function sweeped (line 56) | def sweeped(_params, step, sH, s, _input):
FILE: documentation/examples/policy_aggregation.py
function p1m1 (line 9) | def p1m1(_g, step, sH, s):
function p2m1 (line 11) | def p2m1(_g, step, sH, s):
function p1m2 (line 14) | def p1m2(_g, step, sH, s):
function p2m2 (line 16) | def p2m2(_g, step, sH, s):
function p1m3 (line 19) | def p1m3(_g, step, sH, s):
function p2m3 (line 21) | def p2m3(_g, step, sH, s):
function add (line 26) | def add(y, x):
function policies (line 29) | def policies(_g, step, sH, s, _input):
FILE: documentation/examples/sys_model_A.py
function p1m1 (line 17) | def p1m1(_g, step, sH, s):
function p2m1 (line 19) | def p2m1(_g, step, sH, s):
function p1m2 (line 22) | def p1m2(_g, step, sH, s):
function p2m2 (line 24) | def p2m2(_g, step, sH, s):
function p1m3 (line 27) | def p1m3(_g, step, sH, s):
function p2m3 (line 29) | def p2m3(_g, step, sH, s):
function s1m1 (line 34) | def s1m1(_g, step, sH, s, _input):
function s2m1 (line 38) | def s2m1(_g, step, sH, s, _input):
function s1m2 (line 43) | def s1m2(_g, step, sH, s, _input):
function s2m2 (line 47) | def s2m2(_g, step, sH, s, _input):
function s1m3 (line 52) | def s1m3(_g, step, sH, s, _input):
function s2m3 (line 56) | def s2m3(_g, step, sH, s, _input):
function policies (line 61) | def policies(_g, step, sH, s, _input):
function es3 (line 71) | def es3(_g, step, sH, s, _input):
function es4 (line 76) | def es4(_g, step, sH, s, _input):
function update_timestamp (line 81) | def update_timestamp(_g, step, sH, s, _input):
FILE: documentation/examples/sys_model_B.py
function p1m1 (line 17) | def p1m1(_g, step, sH, s):
function p2m1 (line 19) | def p2m1(_g, step, sH, s):
function p1m2 (line 22) | def p1m2(_g, step, sH, s):
function p2m2 (line 24) | def p2m2(_g, step, sH, s):
function p1m3 (line 27) | def p1m3(_g, step, sH, s):
function p2m3 (line 29) | def p2m3(_g, step, sH, s):
function s1m1 (line 34) | def s1m1(_g, step, sH, s, _input):
function s2m1 (line 38) | def s2m1(_g, step, sH, s, _input):
function s1m2 (line 43) | def s1m2(_g, step, sH, s, _input):
function s2m2 (line 47) | def s2m2(_g, step, sH, s, _input):
function s1m3 (line 52) | def s1m3(_g, step, sH, s, _input):
function s2m3 (line 56) | def s2m3(_g, step, sH, s, _input):
function es3 (line 66) | def es3(_g, step, sH, s, _input):
function es4 (line 71) | def es4(_g, step, sH, s, _input):
function update_timestamp (line 76) | def update_timestamp(_g, step, sH, s, _input):
FILE: testing/generic_test.py
function generate_assertions_df (line 5) | def generate_assertions_df(df, expected_results, target_cols, evaluations):
function make_generic_test (line 27) | def make_generic_test(params):
FILE: testing/models/param_sweep.py
function some_function (line 10) | def some_function(x):
function gamma (line 33) | def gamma(_g, step, sL, s, **kwargs):
function omega (line 37) | def omega(_g, step, sL, s, **kwargs):
function alpha (line 42) | def alpha(_g, step, sL, s, _input, **kwargs):
function beta (line 46) | def beta(_g, step, sL, s, _input, **kwargs):
function policies (line 50) | def policies(_g, step, sL, s, _input, **kwargs):
function sweeped (line 54) | def sweeped(_g, step, sL, s, _input, **kwargs):
FILE: testing/models/policy_aggregation.py
function p1m1 (line 5) | def p1m1(_g, step, sL, s, **kwargs):
function p2m1 (line 7) | def p2m1(_g, step, sL, s, **kwargs):
function p1m2 (line 10) | def p1m2(_g, step, sL, s, **kwargs):
function p2m2 (line 12) | def p2m2(_g, step, sL, s, **kwargs):
function p1m3 (line 15) | def p1m3(_g, step, sL, s, **kwargs):
function p2m3 (line 17) | def p2m3(_g, step, sL, s, **kwargs):
function add (line 22) | def add(y, x):
function policies (line 25) | def policies(_g, step, sH, s, _input, **kwargs):
FILE: testing/results_comparison.py
function compare_results_pytest (line 7) | def compare_results_pytest(result_diff):
function compare_results (line 22) | def compare_results(result_diff):
function dataframe_difference (line 48) | def dataframe_difference(df1, df2):
FILE: testing/test_additional_objs.py
function describe_or_return (line 11) | def describe_or_return(v: object) -> object:
function select_M_dict (line 23) | def select_M_dict(M_dict: Dict[str, object], keys: set) -> Dict[str, obj...
function select_config_M_dict (line 30) | def select_config_M_dict(configs: list, i: int, keys: set) -> Dict[str, ...
function drop_substeps (line 34) | def drop_substeps(_df):
function assign_params (line 41) | def assign_params(_df: pd.DataFrame, configs) -> pd.DataFrame:
function create_experiment (line 80) | def create_experiment(N_RUNS=2, N_TIMESTEPS=3, params: dict=SWEEP_PARAMS):
function test_deepcopy_off (line 166) | def test_deepcopy_off():
FILE: testing/test_arg_count.py
function test_sufs (line 11) | def test_sufs():
FILE: testing/test_param_count.py
function run_experiment (line 23) | def run_experiment(exp: Experiment, mode: str):
function param_count_test_suf_generator (line 30) | def param_count_test_suf_generator(provided_params):
function param_count_test_policy_generator (line 37) | def param_count_test_policy_generator(provided_params):
function create_experiments (line 44) | def create_experiments(N_simulations=3, N_sweeps=3, N_runs=3, N_timestep...
function expected_rows (line 68) | def expected_rows(N_simulations, N_sweeps, N_runs, N_timesteps, N_subste...
function test_row_count_single (line 74) | def test_row_count_single(N_sim, N_sw, N_r, N_t, N_s, P):
function test_row_count_local (line 81) | def test_row_count_local(N_sim, N_sw, N_r, N_t, N_s, P):
FILE: testing/test_print.py
function run_experiment (line 22) | def run_experiment(exp: Experiment, mode: str, supress_print=False):
function param_count_test_suf_generator (line 29) | def param_count_test_suf_generator(provided_params):
function param_count_test_policy_generator (line 36) | def param_count_test_policy_generator(provided_params):
function create_experiments (line 43) | def create_experiments(N_simulations=3, N_sweeps=3, N_runs=3, N_timestep...
function test_print (line 68) | def test_print(capfd):
FILE: testing/test_results_signature.py
function run_experiment (line 17) | def run_experiment(exp: Experiment, mode: str) -> List[Dict]:
function create_experiments (line 24) | def create_experiments(N_simulations=3, N_sweeps=3, N_runs=3, N_timestep...
function expected_rows (line 49) | def expected_rows(N_simulations, N_sweeps, N_runs, N_timesteps, N_subste...
function test_identifiers_value_counts_single (line 54) | def test_identifiers_value_counts_single(N_sim, N_sw, N_r, N_t, N_s):
function test_identifiers_value_counts_multi (line 64) | def test_identifiers_value_counts_multi(N_sim, N_sw, N_r, N_t, N_s):
FILE: testing/test_row_count.py
function run_experiment (line 12) | def run_experiment(exp: Experiment, mode: str) -> List[Dict]:
function create_experiments (line 19) | def create_experiments(N_simulations=3, N_sweeps=3, N_runs=3, N_timestep...
function expected_rows (line 44) | def expected_rows(N_simulations, N_sweeps, N_runs, N_timesteps, N_subste...
function test_row_count_single (line 49) | def test_row_count_single(N_sim, N_sw, N_r, N_t, N_s):
function test_row_count_multi (line 56) | def test_row_count_multi(N_sim, N_sw, N_r, N_t, N_s):
function test_row_count_local (line 66) | def test_row_count_local(N_sim, N_sw, N_r, N_t, N_s):
FILE: testing/test_runs.py
function describe_or_return (line 13) | def describe_or_return(v: object) -> object:
function select_M_dict (line 25) | def select_M_dict(M_dict: Dict[str, object], keys: set) -> Dict[str, obj...
function select_config_M_dict (line 32) | def select_config_M_dict(configs: list, i: int, keys: set) -> Dict[str, ...
function drop_substeps (line 36) | def drop_substeps(_df):
function assign_params (line 43) | def assign_params(_df: pd.DataFrame, configs) -> pd.DataFrame:
function create_experiment (line 80) | def create_experiment(N_RUNS=2, N_TIMESTEPS=3, params: dict = SWEEP_PARA...
function test_mc_sweep_experiment (line 163) | def test_mc_sweep_experiment(mode):
function test_unique_sweep_experiment (line 168) | def test_unique_sweep_experiment(mode):
function test_mc_single_experiment (line 173) | def test_mc_single_experiment(mode):
function test_unique_single_experiment (line 178) | def test_unique_single_experiment(mode):
function experiment_assertions (line 188) | def experiment_assertions(exp: Experiment, mode: Optional[str]=None) -> ...
FILE: testing/tests/append_mod_test.py
function append_model_id (line 10) | def append_model_id(model_ids, sim_config, genesis_states, env_process, ...
class AppendModelTest (line 24) | class AppendModelTest(unittest.TestCase):
method test_index_model_ids (line 25) | def test_index_model_ids(self):
method test_same_model_ids (line 36) | def test_same_model_ids(self):
method test_different_model_ids (line 50) | def test_different_model_ids(self):
method test_mix_model_ids (line 64) | def test_mix_model_ids(self):
FILE: testing/tests/test_cadCAD_exp.py
function empty_experiment (line 10) | def empty_experiment():
function test_experiment (line 21) | def test_experiment(empty_experiment):
FILE: testing/tests/test_import_cadCAD_test.py
function test_jupyter_nbconvert_row_count (line 4) | def test_jupyter_nbconvert_row_count():
FILE: testing/tests/test_multi_model_row_count.py
class MultiModelRowCountResults (line 12) | class MultiModelRowCountResults():
function multi_model_row_count (line 26) | def multi_model_row_count():
function test_row_count (line 108) | def test_row_count(multi_model_row_count: MultiModelRowCountResults):
function test_row_count_from_api (line 113) | def test_row_count_from_api(multi_model_row_count: MultiModelRowCountRes...
function test_row_count_from_results (line 117) | def test_row_count_from_results(multi_model_row_count: MultiModelRowCoun...
function test_row_count_from_sys_model_A (line 121) | def test_row_count_from_sys_model_A(multi_model_row_count: MultiModelRow...
function test_row_count_from_sys_model_B (line 125) | def test_row_count_from_sys_model_B(multi_model_row_count: MultiModelRow...
function test_row_count_from_sys_model_C (line 129) | def test_row_count_from_sys_model_C(multi_model_row_count: MultiModelRow...
function test_a_b_row_count (line 133) | def test_a_b_row_count(multi_model_row_count: MultiModelRowCountResults):
FILE: testing/tests/test_param_sweep.py
function ParamSweep (line 9) | def ParamSweep():
function test_pytest_compare_results (line 20) | def test_pytest_compare_results(ParamSweep):
FILE: testing/tests/test_policy_aggregation.py
function PolicyAggregation (line 9) | def PolicyAggregation():
function test_pytest_compare_results (line 20) | def test_pytest_compare_results(PolicyAggregation):
FILE: testing/tests/test_run1psub0.py
function Run1Psub0 (line 7) | def Run1Psub0():
function test_pytest_compare_results (line 16) | def test_pytest_compare_results(Run1Psub0):
FILE: testing/tests/test_runs_not_zero.py
function test_runs_not_zero (line 4) | def test_runs_not_zero():
FILE: testing/tests/test_timestep1psub0.py
function Timestep1Psub0 (line 7) | def Timestep1Psub0():
function test_timestep1psub0 (line 18) | def test_timestep1psub0(Timestep1Psub0):
FILE: testing/tools/test_tools.py
function test_easy_run (line 32) | def test_easy_run():
FILE: testing/utils.py
function gen_metric_row (line 5) | def gen_metric_row(row, cols):
function gen_metric_dict (line 20) | def gen_metric_dict(df, cols):
function assertEqual (line 25) | def assertEqual(_1, _2, _3=None):
Condensed preview — 101 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (911K chars).
[
{
"path": ".github/FUNDING.yml",
"chars": 762,
"preview": "# These are supported funding model platforms\n\ngithub: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [u"
},
{
"path": ".github/workflows/cadcad-ci.yml",
"chars": 2050,
"preview": "# This workflow will install Python dependencies and run tests with multiple versions of Python\n\nname: cadCAD CI\n\non:\n "
},
{
"path": ".github/workflows/cadcad-publish.yml",
"chars": 851,
"preview": "# This workflow will publish the cadCAD library to PyPI when a release is created\n\nname: cadCAD Publish\n\non:\n release:\n"
},
{
"path": ".gitignore",
"chars": 602,
"preview": "*.idea\n*.vscode\n*.ipynb_checkpoints\n*.DS_Store\n*.eggs\n*.pytest_cache\n*.mypy_cache\n*.csv\n*.egg-info\n*.sqlite3\n*.pyc\n*venv"
},
{
"path": "AUTHORS.txt",
"chars": 1050,
"preview": "Authors\n=======\n\ncadCAD was originally implemented by Joshua E. Jodesty and designed by Michael Zargham, Markus B. Koch,"
},
{
"path": "CHANGELOG.md",
"chars": 18288,
"preview": "# Changelog:\n\n## 0.5.3 - April 19, 2024\n\n### Bugs fixed\n\n- Solve issue on which nested initial states are being copied f"
},
{
"path": "CONTRIBUTING.md",
"chars": 1403,
"preview": "# Contributing to cadCAD\n\n:+1::tada: First off, thanks for taking the time to contribute! :tada::+1:\n\nThe following is a"
},
{
"path": "LICENSE.txt",
"chars": 1073,
"preview": "MIT License\n\nCopyright (c) 2018-2020 BlockScience\n\nPermission is hereby granted, free of charge, to any person obtaining"
},
{
"path": "README.md",
"chars": 2244,
"preview": "```\n ___________ ____\n ________ __ ___/ / ____/ | / __ \\\n / ___/ __` / __ / / / /| | / / / /\n"
},
{
"path": "ascii_art.py",
"chars": 833,
"preview": "production = r'''\n __________ ____\n ________ __ _____/ ____/ | / __ \\\n / ___/ __` / __ / / /"
},
{
"path": "cadCAD/__init__.py",
"chars": 433,
"preview": "import os\n\nimport dill\n\nfrom cadCAD.configuration import Experiment\n\nname = \"cadCAD\"\nversion = \"0.5.3\"\nexperiment = Expe"
},
{
"path": "cadCAD/configuration/__init__.py",
"chars": 10555,
"preview": "from typing import Dict, Callable, List, Tuple\nfrom pandas.core.frame import DataFrame # type: ignore\nfrom datetime impo"
},
{
"path": "cadCAD/configuration/utils/__init__.py",
"chars": 10306,
"preview": "import pandas as pd # type: ignore\nfrom datetime import datetime, timedelta\nfrom collections import Counter\nfrom functoo"
},
{
"path": "cadCAD/configuration/utils/depreciationHandler.py",
"chars": 1065,
"preview": "from copy import deepcopy\n\n\ndef sanitize_config(config):\n for key, value in config.kwargs.items():\n if key == "
},
{
"path": "cadCAD/configuration/utils/policyAggregation.py",
"chars": 916,
"preview": "def get_base_value(x):\n if isinstance(x, str):\n return ''\n elif isinstance(x, int):\n return 0\n el"
},
{
"path": "cadCAD/configuration/utils/userDefinedObject.py",
"chars": 1671,
"preview": "from collections import namedtuple\nfrom inspect import getmembers, ismethod\nfrom pandas.core.frame import DataFrame\n\nfro"
},
{
"path": "cadCAD/diagram/__init__.py",
"chars": 70,
"preview": "from cadCAD.diagram.config_diagram import diagram, diagram_from_config"
},
{
"path": "cadCAD/diagram/config_diagram.py",
"chars": 9699,
"preview": "from graphviz import Digraph\nimport inspect\nimport re\n\n\n### Inspect functions\n\n\ndef extract_var_key(raw_line: str, var_i"
},
{
"path": "cadCAD/engine/__init__.py",
"chars": 10058,
"preview": "from time import time\nfrom typing import Callable, Dict, List, Any, Tuple, Union, Sequence, Mapping\nfrom tqdm.auto impor"
},
{
"path": "cadCAD/engine/execution.py",
"chars": 4765,
"preview": "from typing import Callable, Dict, List, Any, Tuple, Sequence\nfrom pathos.multiprocessing import ProcessPool # type: ign"
},
{
"path": "cadCAD/engine/simulation.py",
"chars": 10628,
"preview": "from typing import Any, Callable, Dict, List, Tuple\nfrom copy import deepcopy, copy\nfrom functools import reduce\nfrom fu"
},
{
"path": "cadCAD/engine/utils.py",
"chars": 816,
"preview": "from datetime import datetime\n\n\ndef datetime_range(start, end, delta, dt_format='%Y-%m-%d %H:%M:%S'):\n reverse_head ="
},
{
"path": "cadCAD/tools/__init__.py",
"chars": 156,
"preview": "from cadCAD.tools.execution import easy_run\nfrom cadCAD.tools.profiling import profile_run\nfrom cadCAD.tools.utils impor"
},
{
"path": "cadCAD/tools/execution/__init__.py",
"chars": 52,
"preview": "from cadCAD.tools.execution.easy_run import easy_run"
},
{
"path": "cadCAD/tools/execution/easy_run.py",
"chars": 4087,
"preview": "import inspect\nimport types\nfrom typing import Dict, Union\n\nimport pandas as pd # type: ignore\nfrom cadCAD.configuratio"
},
{
"path": "cadCAD/tools/preparation.py",
"chars": 2475,
"preview": "from typing import Mapping\nfrom cadCAD.tools.execution import easy_run\nfrom pandas import DataFrame # type: ignore\nfrom "
},
{
"path": "cadCAD/tools/profiling/__init__.py",
"chars": 58,
"preview": "from cadCAD.tools.profiling.profile_run import profile_run"
},
{
"path": "cadCAD/tools/profiling/profile_run.py",
"chars": 1476,
"preview": "from typing import Dict\nfrom cadCAD.tools import easy_run\nfrom cadCAD.types import StateUpdateBlocks, Parameters, State,"
},
{
"path": "cadCAD/tools/profiling/visualizations.py",
"chars": 2350,
"preview": "from tqdm.auto import tqdm\nimport pandas as pd\nimport plotly.express as px\nimport numpy as np\n\n\ndef visualize_elapsed_ti"
},
{
"path": "cadCAD/tools/types.py",
"chars": 347,
"preview": "from typing import NamedTuple, Tuple, Dict, Union, List\n\nclass InitialValue(NamedTuple):\n value: object\n type: typ"
},
{
"path": "cadCAD/tools/utils.py",
"chars": 1354,
"preview": "from cadCAD.types import *\nimport pandas as pd\n\n\ndef generic_suf(variable: str, signal: str = \"\") -> StateUpdateFunction"
},
{
"path": "cadCAD/types.py",
"chars": 1621,
"preview": "from typing import TypedDict, Callable, Union, Dict, List, Tuple, Iterable\nfrom collections import deque\n\nState = Dict[s"
},
{
"path": "cadCAD/utils/__init__.py",
"chars": 4360,
"preview": "from functools import reduce\nfrom collections import defaultdict\nfrom itertools import product\nimport warnings\nfrom typi"
},
{
"path": "cadCAD/utils/execution.py",
"chars": 1455,
"preview": "from pprint import pprint\n\nfrom cadCAD import logo, version\nfrom cadCAD.utils import flatten\n\n\ndef print_exec_info(exec_"
},
{
"path": "cadCAD/utils/jupyter.py",
"chars": 194,
"preview": "def get_home_dir(user):\n return f\"s3://jupyterbackups/jupyter/{user}/\"\n\ndef set_write_path(sc, user, datafolder_path)"
},
{
"path": "cadCAD/utils/sys_config.py",
"chars": 1416,
"preview": "from funcy import curry\nfrom cadCAD.configuration.utils import ep_time_step, time_step\n\n\ndef increment(y, incr_by):\n "
},
{
"path": "documentation/Historically_State_Access.md",
"chars": 5328,
"preview": "Historical State Access (DEPRECATED)\n==\n#### Motivation\nThe current state (values of state variables) is accessed throug"
},
{
"path": "documentation/Policy_Aggregation.md",
"chars": 3211,
"preview": "Policy Aggregation\n==\n\nFor each Partial State Update, multiple policy dictionaries are aggregated into a single dictiona"
},
{
"path": "documentation/README.md",
"chars": 12454,
"preview": "Simulation Configuration\n==\n\n## Introduction\n\nGiven a **Simulation Configuration**, cadCAD produces datasets that repres"
},
{
"path": "documentation/Simulation_Execution.md",
"chars": 6637,
"preview": "# Simulation Execution\nSystem Simulations are executed with the execution engine executor (`cadCAD.engine.Executor`) giv"
},
{
"path": "documentation/System_Configuration.md",
"chars": 5696,
"preview": "# Display System Model Configurations:\n\n## Announcement:\n\nSee [CHANGELOG](CHANGELOG.md)\n\nThe `cadCAD.configuration.Exper"
},
{
"path": "documentation/System_Model_Parameter_Sweep.md",
"chars": 1569,
"preview": "System Model Parameter Sweep\n==\nParametrization of a System Model configuration that produces multiple configurations.\n\n"
},
{
"path": "documentation/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "documentation/cadCAD-v0.4.23-Model-Upgrade-Guide.md",
"chars": 3389,
"preview": "<table>\n <tr>\n <th>\n Feature\n </th>\n <th>\n ver. 0.4.23\n </th>\n "
},
{
"path": "documentation/cadCAD-v0.4.27-Model-Upgrade-Guide.md",
"chars": 2325,
"preview": "<table>\n <tr>\n <th>\n Feature\n </th>\n <th>\n ver. 0.4.27\n </th>\n "
},
{
"path": "documentation/cadCAD-v0.4.28-Model-Upgrade-Guide.md",
"chars": 2285,
"preview": "<table>\n <tr>\n <th>\n Feature\n </th>\n <th>\n ver. 0.4.28\n </th>\n "
},
{
"path": "documentation/examples/__init__.py",
"chars": 80,
"preview": "from cadCAD.configuration import Experiment\n\nsystem_model_AB_exp = Experiment()\n"
},
{
"path": "documentation/examples/cadCAD_diagram.ipynb",
"chars": 13472,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n "
},
{
"path": "documentation/examples/cadCAD_tools_example.ipynb",
"chars": 613800,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \" # cadCAD Easy Run for the Minimal "
},
{
"path": "documentation/examples/example_1.py",
"chars": 1596,
"preview": "from pprint import pprint\n\nimport pandas as pd\nfrom tabulate import tabulate\n\nfrom cadCAD.engine import ExecutionMode, E"
},
{
"path": "documentation/examples/historical_state_access.py",
"chars": 3126,
"preview": "import pandas as pd\nfrom tabulate import tabulate\n\nfrom cadCAD.configuration.utils import config_sim, access_block\nfrom "
},
{
"path": "documentation/examples/param_sweep.py",
"chars": 2842,
"preview": "import pprint\n\nimport pandas as pd\nfrom tabulate import tabulate\n\nfrom cadCAD.configuration.utils import env_trigger, va"
},
{
"path": "documentation/examples/policy_aggregation.py",
"chars": 2133,
"preview": "import pandas as pd\nfrom tabulate import tabulate\n\nfrom cadCAD.configuration.utils import config_sim\nfrom cadCAD.engine "
},
{
"path": "documentation/examples/sys_model_A.py",
"chars": 3760,
"preview": "import numpy as np\nfrom datetime import timedelta\n\nfrom cadCAD.configuration.utils import bound_norm_random, config_sim,"
},
{
"path": "documentation/examples/sys_model_AB_exec.py",
"chars": 907,
"preview": "import pandas as pd\nfrom tabulate import tabulate\nfrom cadCAD.engine import ExecutionMode, ExecutionContext, Executor\nfr"
},
{
"path": "documentation/examples/sys_model_A_exec.py",
"chars": 853,
"preview": "import pandas as pd\nfrom tabulate import tabulate\nfrom cadCAD.engine import ExecutionMode, ExecutionContext, Executor\nfr"
},
{
"path": "documentation/examples/sys_model_B.py",
"chars": 3510,
"preview": "import numpy as np\nfrom datetime import timedelta\n\nfrom cadCAD.configuration.utils import bound_norm_random, config_sim,"
},
{
"path": "documentation/examples/sys_model_B_exec.py",
"chars": 763,
"preview": "import pandas as pd\nfrom tabulate import tabulate\n# The following imports NEED to be in the exact order\nfrom cadCAD.engi"
},
{
"path": "requirements.txt",
"chars": 234,
"preview": "-i https://pypi.org/simple\n\nparameterized>=0.7.4\npytest>=6.0.2\ntabulate>=0.8.7\nwheel>=0.38.1\npandas>=1.1.5\nfuncy>=1.16\nd"
},
{
"path": "setup.py",
"chars": 2098,
"preview": "from setuptools import find_packages, setup\n\nshort_description = (\n \"cadCAD: a differential games based simulation so"
},
{
"path": "testing/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "testing/experiments/__init__.py",
"chars": 105,
"preview": "from cadCAD.configuration import Experiment\n\nexp_param_sweep = Experiment()\nexp_policy_agg = Experiment()"
},
{
"path": "testing/generic_test.py",
"chars": 2130,
"preview": "import unittest\nfrom functools import reduce\nfrom tabulate import tabulate\n\ndef generate_assertions_df(df, expected_resu"
},
{
"path": "testing/models/__init__.py",
"chars": 63,
"preview": "from cadCAD.configuration import Experiment\n\nexp = Experiment()"
},
{
"path": "testing/models/param_sweep.py",
"chars": 2511,
"preview": "import pprint\n\nfrom cadCAD import experiment\nfrom cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils"
},
{
"path": "testing/models/policy_aggregation.py",
"chars": 1688,
"preview": "from cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils import config_sim\n\n# Policies per Mechanism\n"
},
{
"path": "testing/results_comparison.py",
"chars": 4308,
"preview": "import unittest\nimport pandas as pd # type: ignore\nfrom tabulate import tabulate # type: ignore\nfrom pandas._testing imp"
},
{
"path": "testing/test_additional_objs.py",
"chars": 6517,
"preview": "from typing import Dict, List\nfrom cadCAD.engine import Executor, ExecutionContext, ExecutionMode\nfrom cadCAD.configurat"
},
{
"path": "testing/test_arg_count.py",
"chars": 1242,
"preview": "from typing import Dict, List\nfrom cadCAD.engine import Executor, ExecutionContext, ExecutionMode\nfrom cadCAD.configurat"
},
{
"path": "testing/test_param_count.py",
"chars": 3019,
"preview": "from cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils import config_sim\nfrom cadCAD.engine import "
},
{
"path": "testing/test_print.py",
"chars": 3087,
"preview": "from cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils import config_sim\nfrom cadCAD.engine import "
},
{
"path": "testing/test_results_signature.py",
"chars": 2538,
"preview": "from cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils import config_sim\nfrom cadCAD.engine import "
},
{
"path": "testing/test_row_count.py",
"chars": 2515,
"preview": "from cadCAD.configuration import Experiment\nfrom cadCAD.configuration.utils import config_sim\nfrom cadCAD.engine import "
},
{
"path": "testing/test_runs.py",
"chars": 8038,
"preview": "from typing import Dict, List, Optional\nfrom cadCAD.engine import Executor, ExecutionContext, ExecutionMode\nfrom cadCAD."
},
{
"path": "testing/tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "testing/tests/a_b_tests/0_4_23_record_count.json",
"chars": 20,
"preview": "{\"record_count\": 74}"
},
{
"path": "testing/tests/a_b_tests/multi_model_row_count_0_4_23.py",
"chars": 1603,
"preview": "import json, pathlib, pandas as pd\n\nfrom cadCAD import configs\nfrom cadCAD.configuration import Experiment\nfrom cadCAD.e"
},
{
"path": "testing/tests/append_mod_test.py",
"chars": 3029,
"preview": "import unittest\nfrom copy import deepcopy\n\nfrom testing.models import exp\nfrom testing.models.param_sweep import sim_con"
},
{
"path": "testing/tests/import_cadCAD.ipynb",
"chars": 1109,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"id\": \"15b9b09a\",\n \"metadata\": {},\n \"outputs\":"
},
{
"path": "testing/tests/test_cadCAD_exp.py",
"chars": 730,
"preview": "import pandas as pd\nfrom tabulate import tabulate\nfrom testing.results_comparison import dataframe_difference, compare_r"
},
{
"path": "testing/tests/test_import_cadCAD_test.py",
"chars": 574,
"preview": "import os, subprocess, json\nfrom testing.utils import assertEqual\n\ndef test_jupyter_nbconvert_row_count():\n command ="
},
{
"path": "testing/tests/test_multi_model_row_count.py",
"chars": 6420,
"preview": "import json\nimport os\nimport pandas as pd # type: ignore\nfrom cadCAD.configuration import Experiment\nfrom cadCAD.engine "
},
{
"path": "testing/tests/test_param_sweep.py",
"chars": 766,
"preview": "from testing.models import param_sweep\nfrom cadCAD.engine import ExecutionMode, ExecutionContext, Executor\nimport pandas"
},
{
"path": "testing/tests/test_policy_aggregation.py",
"chars": 866,
"preview": "import pandas as pd # type: ignore\nfrom testing.models import policy_aggregation as policy_agg\nfrom testing.results_comp"
},
{
"path": "testing/tests/test_run1psub0.py",
"chars": 626,
"preview": "import pandas as pd # type: ignore\nfrom testing.results_comparison import dataframe_difference, compare_results_pytest\ni"
},
{
"path": "testing/tests/test_runs_not_zero.py",
"chars": 421,
"preview": "from cadCAD.configuration.utils import config_sim\nfrom testing.utils import assertEqual\n\ndef test_runs_not_zero():\n v"
},
{
"path": "testing/tests/test_timestep1psub0.py",
"chars": 650,
"preview": "import pandas as pd\nfrom testing.results_comparison import dataframe_difference, compare_results_pytest\nimport pytest\n\n\n"
},
{
"path": "testing/tools/test_tools.py",
"chars": 1159,
"preview": "from cadCAD.tools.execution import easy_run\n\n# def test_empty_easy_run():\n# state_variables = {\n# }\n#\n# para"
},
{
"path": "testing/utils.py",
"chars": 790,
"preview": "#\n# def record_generator(row, cols):\n# return {col: row[col] for col in cols}\n\ndef gen_metric_row(row, cols):\n re"
}
]
// ... and 12 more files (download for full content)
About this extraction
This page contains the full source code of the BlockScience/cadCAD GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 101 files (840.0 KB), approximately 270.5k tokens, and a symbol index with 305 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.