[
  {
    "path": ".coveragerc",
    "content": "[run]\nomit =\n    .tox/*\n    setup.py\n"
  },
  {
    "path": ".dockerignore",
    "content": "**/*.pyc\n**/*.pyo\n**/*.pyd\n.git\n.tox\n**/__pycache__\n**/.DS_Store\n**/*.egg-info\n"
  },
  {
    "path": ".gitattributes",
    "content": "scripts/* linguist-documentation\n"
  },
  {
    "path": ".gitignore",
    "content": "*.py[cod]\n__pycache__/\n*.egg-info\n\n_build\nbuild/\ndist/\nprof/\nvenv/\nuv.lock\n\n.ipynb_checkpoints/\n\n.*\n!.github/\n!.travis.yml\n!.coveragerc\n!.gitignore\n!.gitattributes\n!.dockerignore\n!.pyup.yml\n"
  },
  {
    "path": "CHANGELOG.md",
    "content": "Changelog\n=========\n\nDevelopment\n-----------\n\n\nopendsm-1.2.7\n-----------\n\n* Switch build to use pyproject.toml and uv \n* Change pyproject.toml build system to hatchling \n* Update to example data to include GHI\n* Update `load_test_data` function to always pull from GitHub.\n* Add comparison groups. This feature is still in development. Final API is unfinished.\n* Consolidate clustering for hourly model and CG clustering\n* Add cluster voting\n* Include new indices and update `ClusteringMetrics` class\n* Implemented Numba and revised functions in adaptive_loss and stats.basic\n* Update dependencies\n\nopendsm-1.2.6\n-----------\n\n* Fixed bug in `from_series` instantiation of daily data class\n\nopendsm-1.2.5\n-----------\n\n* Expose SpectralClustering's assign_labels options. `discretize` and `cluster_qr` are not always deterministic with seed\n* Added more metrics to BaselineMetrics\n\nopendsm-1.2.4\n-----\n\n* Bug fix of metrics that squeaked through\n\nopendsm-1.2.3\n-----\n\n* Spectral clustering is now seeded appropriately\n* Fixed bug making seed in main hourly settings not be passed to clustering settings\n* Including new metrics in baseline_metrics\n\nopendsm-1.2.2\n-----\n\n* Change daily model to use CVRMSE_adj and PNRMSE_adj as intended\n* Autocorrelation function is now consistent\n\nopendsm-1.2.1\n-----\n\n* Revert how autocorr is calculated prior to 1.2.0\n\nopendsm-1.2.0\n-----\n\n* Add hourly model uncertainty\n* Daily model uses BaselineMetrics natively now, accessible from model.baseline_metrics\n* Data classes now accept dictionaries to modify DQ criteria. This is an R&D feature\n* Migrate to modern Python logger interface to solve deprecation warnings\n\nopendsm-1.1.0\n-----\n\n* Updated the Hourly model\n* Performed new optimization for Hourly model configuration\n* Developed adaptive robust weighting per hour-of-day for the hourly model\n* Updated adaptive loss function. Previously it assumed too large of a range of outliers and made choosing alphas < 0 unlikely\n* Altered clustering methodology, it now uses spectral clustering\n* Changed temperature binning to be fixed bins\n* Made temporal bins/temperature bins act together on temperature\n* Disallow negative CVRMSE in Hourly model\n* Added daily CVRMSE >= 0 and PNRMSE sufficiency requirements\n* Partially updated Daily model to use baseline_metrics\n* Changed extreme values warning flag to check using IQR rule instead of median +- IQR which is incorrect\n* Fix warning data on `high_frequency_temperature_data` warning.\n* Squash numpy divide-by-zero warnings in caltrack Hourly metrics.\n\nopendsm-1.0.0\n-----\n\n* Initial OpenDSM release\n\neemeter-4.1.1\n-----\n\n* Add GHI sufficiency check requiring 90% coverage for each month\n* Add weights propogation from data class to daily model via \"weights\" column\n* Converted daily model settings from attrs to pydantic\n* Refactored daily model initial guess optimization to use consolidated optimize function\n* Add experimental daily weighting for hourly model fitting (if one day is crazy, it will be down weighted in the fit)\n\neemeter-4.1.0\n-----\n\n* Add new hourly model to support solar meters and improve nonsolar results\n\neemeter-4.0.8\n-----\n\n* Add github action to publish to pypi\n* Bump to latest packages and remove all deprecation/future warnings as of 2024-12-20.\n* Allow identical observations to not raise exception for daily model in `linear_fit`.\n* Handle ambiguous and nonexistent local times when creating billing dataclass \n* Fix serialization and deserialization of hourly CalTRACK metrics.\n* Rename HourlyBaselineData.sufficiency_warnings -> HourlyBaselineData.warnings\n* Add disqualification field to HourlyBaselineData and HourlyReportingData\n* Fix bug where HourlyBaselineData and HourlyReportingData wasn't actually NaNning zero rows when `is_electricity=True`.\n* Constrain eemeter daily model balance points to T_min_seg and T_max_seg rather than T_min and T_max.\n* Fix bug in `linear_fit` due to SciPy's `theilslopes(y, x)` not following the same order as `linregress(x, y)`\n\neemeter-4.0.7\n-----\n\n* Handle ambiguous and nonexistent local times when creating daily dataclass\n\neemeter-4.0.6\n-----\n\n* Update docs.\n* Update typehints on core daily and utility functions.\n* Minor change to loading test data to ensure the reporting period is a year ahead of the baseline period.\n\neemeter-4.0.5\n-----\n\n* Flip slope when deserializing legacy hdd_only models\n\neemeter-4.0.4\n-----\n\n* Add support for deserializing legacy hourly models\n* Fix legacy daily model deserialization\n\neemeter-4.0.3\n-----\n\n* Move masking behavior for rows with missing temperature from reporting dataclass to prediction output\n* Add disqualification check to billing model predict()\n\neemeter-4.0.2\n-----\n\n* Force index to use nanosecond precision\n* Compute coverage using same offset as initial reads to fix issues when downsampling hourly data\n* Update test data location\n* Fix bug in daily plotting to remove NaN values if input\n* Refactor sufficiency criteria to be more explicit and easier to manage\n\neemeter-4.0.1\n-----\n\n* Correct dataframe input behavior and final row temperature aggregation\n* Remove unnecessary datetime normalization in order to respect hour of day\n* Convert timestamps in certain warnings to strings to allow serialization\n* Allow configuration of segment_type in HourlyModel wrapper\n\n\neemeter-4.0.0\n-----\n\n* Update daily model methods, API, and serialization\n* Provide new API for hourly model to match daily syntax and prepare for future additions\n* Add baseline and reporting dataclasses to support compliant initialization of meter and temperature data\n\neemeter-3.2.0\n-----\n\n* Addition of modules and amendments in support of international facility for EEMeter, including principally:\n* Addition of quickstart.py; updating setup.py and __init__/py accordingly.\n* Inclusion of temperature conversion amendments to design_matrices; features; and derivatives.\n* Addition of new tests and samples.\n* Amendments to tutorial.ipynb.\n* Addition of eemeter international.ipynb.\n* Change .iteritems() to .items() in accordance with pandas>=2.0.0\n* .get_loc(x, method=...) to .get_indexer([x],method=...)[0] in accordance with pandas>=2.0.0\n* Updated mean() to mean(numeric_only=True) in accordance to pandas>=2.0.0\n* Updated tests to work with pandas>=2.0.0\n* Update python version in Dockerfile.\n* Update other dependencies (including adding rust) in Dockerfile.\n* Remove pinned dependencies in Pipfile.\n* Relock Pipfile (and do so inside of the docker image).\n* Update pytests to account for changes in newer pandas where categorical variables are no longer included in `df.sum().sum()`.\n* Clarify the functioning of start, end and max_days parameters to `get_reporting_data()` and `get_baseline_data()`.\n\neemeter-3.1.1\n-----\n\n* Update observed_mean calculation to account for solar (negative usage) to provide\nsensible cvrmse calculations.\n\neemeter-3.1.0\n-----\n\n* Remove missing hour_of_week categories in the CalTrack hourly methods so they predict null for those hours. \n\neemeter-3.0.0\n-----\n\n* Remove python27 support.\n* Update Pipfile lock.\n* Update `fit_temperature_bins` to potentially take an `occupancy_lookup` in order to\n  fit different temperature bins for occupied/unoccupied modes. *This changes the args passed to eemeter.create_caltrack_hourly_segmented_design_matrices, where it now requires a set of bins for occupied and unoccupied temperatures separately.*\n* Update CalTRACK hourly model formula to use different bins for occupied and\n  unoccupied mode.\n\neemeter-2.10.11\n-------\n\n* Fix tests and make changes to ensure tests pass on pandas version 1.2.1.\n* Fix bug in segmentation.py causing a section of tutorial to fail.\n\neemeter-2.10.0\n------\n\n* Add additional terms into ModelMetrics() class which can be used in fractional savings uncertainy computations.\n\neemeter-2.9.2\n-----\n\n* Remove fixing of versions of libraries in setup.py to avoid unforeseen issues with library updates.\n\neemeter-2.9.1\n-----\n\n* Fix versions of libraries in setup.py to avoid unforeseen issues with library updates.\n\neemeter-2.9.0\n-----\n\n* Clarify blackout period.\n\neemeter-2.8.6\n-----\n\n* Fix issue with `get_reporting_data` and `get_baseline_data` when passing data with non-UTC timezones.\n\neemeter-2.8.5\n-----\n\n* Add functions to clean billing/daily data according to caltrack rules.\n\neemeter-2.8.4\n-----\n\n* Further limit segments used in hourly `totals_metrics` to only calculate when weight=1.\n\neemeter-2.8.3\n-----\n\n* Update hourly `totals_metrics` calculation to properly use only the segment of the model.\n\neemeter-2.8.2\n-----\n\n* Add `totals_metrics` to hourly models.\n\neemeter-2.8.1\n-----\n\n* Fix bug with `get_baseline_data` in regards to recent addition of `n_days_billing_period_overshoot` kwarg.\n\neemeter-2.8.0\n-----\n\n* Update `get_baseline_data` to allow for limit to billing overshoot using `n_days_billing_period_overshoot` kwarg.\n\neemeter-2.7.7\n-----\n\n* Add function to clean billing data to fit caltrack specifications (`clean_caltrack_billing_data`).\n\neemeter-2.7.6\n-----\n\n* Update io functions to support latest pandas (>=0.24.x).\n* Update documentation for CalTRACK Hourly methods.\n* Add tutorial.\n\neemeter-2.7.5\n-----\n\n* Fix completeness check for `get_terms` for last term.\n\neemeter-2.7.4\n-----\n\n* Make more usable outputs for the `get_terms` function (list of eemeter.Term objects).\n\neemeter-2.7.3\n-----\n\n* Update `as_freq` so it has an optional `include_coverage` parameter where it returns a dataframe with one column including the percent coverage of data used to create each sample.\n\neemeter-2.7.2\n-----\n\n* Fixes the columns that are given in an empty prediction result called with the\n  ` with_design_matrix=True` flag set for caltrack usage per day methods.\n* Update bug report github issue template.\n* Add test for `as_freq`.\n\neemeter-2.7.1\n-----\n\n* Change `as_freq` to handle all Null series.\n\neemeter-2.7.0\n-----\n\n* Add `get_terms` method to allow splitting reporting data into any number\n  of terms specified by day length.\n\neemeter-2.6.0\n-----\n\n* Change `fit_caltrack_hourly_model` so it returns a `CalTRACKHourlyModelResults` object rather than a `CalTRACKHourlyModel`, in order to bring it in line with the `caltrack_usage_per_day` model outputs.\n\neemeter-2.5.4-post1\n-----------\n\n* Update MANIFEST.in to fix release and update `./bump_version.sh` script\n  to remove build directories.\n\neemeter-2.5.4\n-----\n\n* Add data fields to the `DataSufficiency` even if there are no warnings when calculating sufficiency.\n\neemeter-2.5.3-post2\n-----------\n\n* Attempt 2 to fix release .whl file by removing local build and dist\n  directories before running `python setup.py upload`.\n\neemeter-2.5.3-post1\n-----------\n\n* Fix release .whl file which had some extra directories.\n* Add draft MAINTAINERS.md.\n\neemeter-2.5.3\n-----\n\n* Fix `metered_savings` behavior so that it does not fail to compute error bands when there is 0 variance in the baseline.\n\neemeter-2.5.2\n-----\n\n* Fix `as_freq` behavior to preserve sum and add a null last index at the target\n  frequency if necessary.\n\neemeter-2.5.1\n-----\n\n* Capture an additional exception type (`KeyError`) in recently adjusted\n  `get_baseline_data` and `get_reporting_data` methods.\n\neemeter-2.5.0\n-----\n\n* Add parameters to `get_baseline_data` and `get_reporting_data` to help make\n  these methods a bit more correct for billing data.\n* Preserve nulls properly in `as_freq`.\n* Update jupyter version to be compatible with latest tornado version.\n\neemeter-2.4.0\n-----\n\n* Fix for bug that occasionally leads to `LinAlgError: SVD did not converge` error when fitting caltrack hourly models by addressing multi-collinearity when only a single occupancy mode is detected\n\neemeter-2.3.1\n-----\n\n* Hot fix for bug that occasionally leads to `LinAlgError: SVD did not converge` error when fitting caltrack hourly models by converting the weights from `np.float64` ton `np.float32`.\n\neemeter-2.3.0\n-----\n\n* Fix bug where the model prediction includes features in the last row that should be null.\n* Fix in `transform.get_baseline_data` and `transform.get_reporting_data` to enable pulling a full year of data even with irregular billing periods\n\neemeter-2.2.10\n------\n\n* Added option in `transform.as_freq` to handle instantaneous data such as temperature and other weather variables.\n\neemeter-2.2.9\n-----\n\n* Predict with empty formula now returns NaNs.\n\neemeter-2.2.8\n-----\n\n* Update `compute_occupancy_feature` so it can handle instances where there are less than 168 values in the data.\n\neemeter-2.2.7\n-----\n\n* SegmentModel becomes CalTRACKSegmentModel, which includes a hard-coded check that the same hours of week are in the model fit parameters and the prediction design matrix.\n\neemeter-2.2.6\n-----\n\n* Reverts small data bug fix.\n\neemeter-2.2.5\n-----\n\n* Fix bug with small data (1<week) for hourly occupancy feature calculation.\n* Bump dev eeweather version.\n* Add `bump_version` script.\n* Filter two specific warnings when running tests:\n  statsmodels pandas .ix warning, and eemeter model fitting warning.\n\neemeter-2.2.4\n-----\n\n* Add `json()` serialization for `SegmentModel` and `SegmentedModel`.\n\neemeter-2.2.3\n-----\n\n* Change `max_value` to float so that it can be json serialized even if the input is int64s.\n\neemeter-2.2.2\n-----\n\n* Add warning to `caltrack_sufficiency_criteria` regarding extreme values.\n\neemeter-2.2.1\n-----\n\n* Fix bug in fractional savings uncertainty calculations using billing data.\n\neemeter-2.2.0\n-----\n\n* Add fractional savings uncertainty to modeled savings derivatives.\n\neemeter-2.1.8\n-----\n\n* Update so that models built with empty temperature data won't result in error.\n\neemeter-2.1.7\n-----\n\n* Update so that models built from a single record won't result in error.\n\neemeter-2.1.6\n-----\n\n* Update multiple places where `df.empty` is used and replaced with `df.dropna().empty`.\n* Update documentation for running CalTRACK hourly methods.\n\neemeter-2.1.5\n-----\n\n* Fix zero division error in metrics calculation for several metrics that\n  would otherwise cause division by zero errors in fsu_error_band calculation.\n\neemeter-2.1.4\n-----\n\n* Fix zero division error in metrics calculation for series of length 1.\n\neemeter-2.1.3\n-----\n\n* Fix bug related to caltrack billing design matrix creation during empty temperature traces.\n\neemeter-2.1.2\n-----\n\n* Add automatic t-stat computation for metered savings error bands, the\n  implementation of which requires expicitly adding scipy to setup.py\n  requirements.\n* Don't compute error bands if reporting period data is empty for metered\n  savings.\n\neemeter-2.1.1\n-----\n\n* Fix degree day ranges (30-90) for prefab caltrack design matrix creation\n  methods.\n* Fix the warning for total degree days to use total degree days instead of\n  average degree days.\n\neemeter-2.1.0\n-----\n\n* Update the `use_billing_presets` option in `fit_caltrack_usage_per_day_model`\n  to use a minimum data sufficiency requirement for qualifying CandidateModels\n  (similar to daily methods).\n* Add an error when attempting to use billing presets without passing a weights\n  column to facilitate weighted least squares.\n\neemeter-2.0.5\n-----\n\n* Give better error for duplicated meter index in compute temperature features.\n\neemeter-2.0.4\n-----\n\n* Change metrics input length error to warning.\n\neemeter-2.0.3\n-----\n\n* Apply black code style for easy opinionated PEP 008 formatting\n* Apply JSON-safe float conversion to all metrics.\n\neemeter-2.0.2\n-----\n\n* Cont. fixing JSON representation of NaN values\n\neemeter-2.0.1\n-----\n\n* Fixed JSON representation of model classes\n\neemeter-2.0.0\n-----\n\n* Initial release of 2.x.x series\n"
  },
  {
    "path": "CHARTER.md",
    "content": "# Technical Charter (the \"Charter\") for OpenDSM, a Series of LF Projects, LLC\n\nAdopted August 28, 2024\n\n\nThis charter (the “Charter”) sets forth the responsibilities and procedures for \ntechnical contribution to, and oversight of, OpenDSM, which has been \nestablished as OpenDSM a Series of LF Projects, LLC (the “Project”).  \nLF Projects, LLC (“LF Projects”) is a Delaware series limited liability company. \nAll Contributors to the Project must comply with the terms of this Charter. \n\n## 1) Mission and Scope of the Project\n-----------------------------------\n\na) The mission of the Project is to develop an open source project with the\nfollowing goals:\n\n- i) Build an open source library for predicting energy consumption of utility meters \n  based on their historical usage and additional covariate data;\n- ii) Maintain a production-ready library to act as a measurement system for \n  distributed energy resources that can be replicated by all parties in markets and \n  expand capabilities over time;\n- iii) Ensure transparency and trust by providing open source code that enables \n  reliable validation and mutual accountability in energy markets;\n- iv) Provide reusable components with documented APIs and consistent security\n  practices; and\n- v) Develop an ecosystem of developers, suppliers, OEMs, systems integrators,\n  and customers all using this common platform as the basis to define common\n  transactional units for distributed energy resources in the market.\n\nb) The scope of the Project includes methods and software development under OSI-approved\nopen source licenses supporting the mission, documentation, testing, integration, and\nthe creation of other artifacts that aid the development, deployment, operation, or\nadoption of the open source software project.\n\n## 2) Technical Steering Committee\n-------------------------------\n\na) The Technical Steering Committee (the \"TSC\") will be responsible for all\ntechnical oversight of the open source Project.\n\nb) The TSC voting members are initially the Project’s Committers. At the inception of \nthe project, the Committers of the Project will be as set forth within the \n[MAINTAINERS](MAINTAINERS.md) file within the Project’s code repository. The TSC may \nchoose an alternative approach for determining the voting members of the TSC, and any \nsuch alternative approach will be documented in the [MAINTAINERS](MAINTAINERS.md) file. \nAny meetings of the Technical Steering Committee are intended to be open to the public, \nand can be conducted electronically, via teleconference, or in person.\n\nc) TSC projects generally will involve Contributors and Committers. The TSC may adopt or \nmodify roles so long as the roles are documented in the CONTRIBUTING file. Unless \notherwise documented:\n\n- i) Contributors include anyone in the technical community that contributes code, \n  documentation, or other technical artifacts to the Project;\n- ii) \"Committers are Contributors to whom the Project has granted the privilege of \n  modifying (“committing”) source code, documentation or other technical artifacts \n  directly in a Project repository; and\n- iii) A Contributor may become a Committer by a three-quarter majority of the existing \n  Committers. A Committer may be removed by a majority approval of the other existing \n  Committers.\n\nd) Participation in the Project through becoming a Contributor and Committer is open to \nanyone so long as they abide by the terms of this Charter.\n\ne) The TSC may (1) establish work flow procedures for the prioritization, submission, \napproval, and closure/archiving of projects, (2) set requirements for the promotion of \nContributors to Committer status, as applicable, and (3) amend, adjust, refine and/or \neliminate the roles of Contributors, and Committers, and create new roles, and publicly \ndocument any TSC roles, as it sees fit.\n\nf) The TSC may elect a TSC Chair, who will preside over meetings of the TSC and will \nserve until their resignation or replacement by the TSC.  The TSC Chair, or any other \nTSC member so designated by the TSC, will serve as the primary communication contact \nbetween the Project and LF Energy Foundation of The Linux Foundation.\n\ng) Responsibilities: The TSC will be responsible for all aspects of oversight relating to\nthe Project, which may include:\n\n- i) Coordinating the technical direction of the Project;\n- ii) Approving project or system proposals (including, but not limited to, incubation, \n  deprecation, and changes to a sub-project’s scope);\n- iii) organizing sub-projects and removing sub-projects;\n- iv) creating sub-committees or working groups to focus on cross-project technical \n  issues and requirements;\n- v) appointing representatives to work with other open source or open standards \n  communities;\n- vi) coordinate with other LF Energy technical projects, including selecting a \n  representative to participate in the LF Energy Technical Advisory Council (TAC);\n- vii) establishing community norms, workflows, issuing releases, and security issue \n  reporting policies;\n- viii) approving and implementing policies and processes for contributing (to be \n  published in the project repository) and coordinating with the Series Manager to \n  resolve matters or concerns that may arise as set forth in Section 7 of this Charter;\n- ix) discussions, seeking consensus, and where necessary, voting on technical matters \n  relating to the code base that affect multiple projects; and\n- x) coordinating any marketing, events, or communications regarding the Project with the\n  LF Projects Manager or their designee.\n\n## 3) TSC Voting\n-------------\n\na) While the Project aims to operate as a consensus-based community, if any TSC\ndecision requires a vote to move the Project forward, the voting members of\nthe TSC will vote using the Approval Voting method.\n\nb) Quorum for TSC meetings requires at least fifty percent of all voting members\nof the TSC to be present. The TSC may continue to meet if quorum is not met,\nbut will be prevented from making any decisions at the meeting.\n\nc) Except as provided in Section 7.c. and 8.a, decisions by vote at a meeting require a \nmajority vote of those in attendance, provided quorum is met. Decisions made by \nelectronic vote without a meeting require a majority vote of all voting members of the \nTSC.\n\nd) In the event a vote cannot be resolved by the TSC, any voting member of the\nTSC may refer the matter to the Series Manager for assistance in reaching a\nresolution.\n\n## 4) Compliance with Policies\n---------------------------\n\na) This Charter is subject to the Series Agreement for the Project and the\nOperating Agreement of LF Projects. Contributors will comply with the policies\nof LF Projects as may be adopted and amended by LF Projects, including, without\nlimitation the policies listed at\n[https://lfprojects.org/policies/](https://lfprojects.org/policies/).\n\nb) The TSC may adopt a code of conduct (\"CoC\") for the Project, which is\nsubject to approval by the Series Manager.  Contributors to the Project will\ncomply with the CoC or, in the event that a Project-specific CoC has not been\napproved, the LF Projects [Code of Conduct](CODE_OF_CONDUCT.md) listed at\n[https://lfprojects.org/policies/](https://lfprojects.org/policies/).\n\nc) When amending or adopting any policy applicable to the Project, LF Projects\nwill publish such policy, as to be amended or adopted, on its web site at least\n30 days prior to such policy taking effect; provided, however, that in the case\nof any amendment of the Trademark Policy or Terms of Use of LF Projects, any\nsuch amendment is effective upon publication on LF Project’s web site.\n\nd) All participants must allow open participation from any individual or\norganization meeting the requirements for contributing under this Charter and\nany policies adopted for all participants by the TSC, regardless of competitive\ninterests. Put another way, the Project community must not seek to exclude any\nparticipant based on any criteria, requirement, or reason other than those that\nare reasonable and applied on a non-discriminatory basis to all participants in\nthe Project community.\n\ne) The Project will operate in a transparent, open, collaborative, and ethical\nmanner at all times.  All official Project discussions, proposals, timelines,\ndecisions, and status  should be made open and easily accessible to all; while\nprivate discussions may happen, the basis for decisions should be publicly\ndocumented and accessible. Any violations or potential violations of these \nrequirements should be reported immediately to the LF Projects Manager.\n\n## 5) Community Assets\n-------------------\n\na) LF Projects will hold title to all trade or service marks used by the\nProject (\"Project Trademarks\"), whether based on common law or registered\nrights.  Project Trademarks will be transferred and assigned to LF Projects to\nhold on behalf of the Project. Any use of any Project Trademarks by\nparticipants in the Project will be in accordance with the license from LF\nProjects and inure to the benefit of LF Projects.\n\nb) The Project will, as permitted and in accordance with such license from LF\nProjects, develop and own all Project-related online service accounts and\ndomain name registrations created by the Project community.\n\nc) Under no circumstances will LF Projects be expected or required to undertake\nany action on behalf of the Project that is inconsistent with the tax-exempt\nstatus or purpose, as applicable, of LFP, Inc. or LF Projects, LLC.\n\n## 6) General Rules and Operations\n-------------------------------\n\na) The Project will:\n\n- i) engage in the work of the project in a professional manner consistent with\n  maintaining a cohesive community, while also maintaining the goodwill and\n  esteem of LF Projects, LFP, Inc. and other partner organizations in the open\n  source software community; and\n- ii) respect the rights of all trademark owners, including any branding and\n  trademark usage guidelines.\n\n## 7) Intellectual Property Policy\n-------------------------------\n\na) Participants acknowledge that the copyright in all new contributions will be\nretained by the copyright holder as independent works of authorship and that no\ncontributor or copyright holder will be required to assign copyrights to the\nProject.\n\nb) Except as described in Section 7.c., all contributions to the Project are\nsubject to the following:\n\n- i) All new inbound code contributions to the Project must be made using an\n  OSI-approved open source license specified for the Project within the\n  [\"LICENSE\"](LICENSE) file within the Project’s code repository (the \"Project\n  License\").\n- ii) All new inbound code contributions must also be accompanied by a Developer\n  Certificate of Origin ([http://developercertificate.org](http://developercertificate.org))\n  sign-off in the source code system that is submitted through a TSC-approved\n  contribution process which will bind the authorized contributor, and, if not\n  self-employed, their employer to the applicable license(s);\n- iii) All outbound code will be made available under the Project License.\n- iv) Documentation will be received and made available by the Project under\n  the Creative Commons Attribution 4.0 International License (available at\n  [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)).\n- v) To the extent a contribution includes or consists of data sets, any rights in such \n  data will be made available under the CDLA-Permissive 1.0 License.\n- vi) The Project may seek to integrate and contribute back to other open\n  source projects (\"Upstream Projects\"). In such cases, the Project will\n  conform to all license requirements of the Upstream Projects, including\n  dependencies, leveraged by the Project.  Upstream Project code contributions\n  will comply with the contribution process and license terms for the\n  applicable Upstream Project.\n\nc) The TSC may approve the use of an alternative license or licenses for inbound or \noutbound contributions on an exception basis. To request an exception, please describe \nthe contribution, the alternative open source license(s), and the justification for using \nan alternative open source license in the Project. License exceptions must be approved by \na two-thirds vote of the entire TSC.\n\nd) Contributed files should contain license information, such as SPDX short\nform identifiers, indicating the open source license or licenses pertaining\nto the file.\n\n## 8) Amendments\n-------------\n\nThis charter may be amended by a two-thirds vote of the entire TSC and is\nsubject to approval by LF Projects.\n"
  },
  {
    "path": "CODE_OF_CONDUCT.md",
    "content": "# Contributor Covenant Code of Conduct\n\n## Our Pledge\n\nIn the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.\n\n## Our Standards\n\nExamples of behavior that contributes to creating a positive environment include:\n\n* Using welcoming and inclusive language\n* Being respectful of differing viewpoints and experiences\n* Gracefully accepting constructive criticism\n* Focusing on what is best for the community\n* Showing empathy towards other community members\n\nExamples of unacceptable behavior by participants include:\n\n* The use of sexualized language or imagery and unwelcome sexual attention or advances\n* Trolling, insulting/derogatory comments, and personal or political attacks\n* Public or private harassment\n* Publishing others' private information, such as a physical or electronic address, without explicit permission\n* Other conduct which could reasonably be considered inappropriate in a professional setting\n\n## Our Responsibilities\n\nProject maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.\n\nProject maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.\n\n## Scope\n\nThis Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.\n\n## Enforcement\n\nInstances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at admin@openee.io. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.\n\nProject maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.\n\n## Attribution\n\nThis Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]\n\n[homepage]: http://contributor-covenant.org\n[version]: http://contributor-covenant.org/version/1/4/\n"
  },
  {
    "path": "CONTRIBUTING.md",
    "content": "Contributing\n============\n\nGuidelines\n----------\n\n* Make sure you follow PEP 008 style guide conventions. You can check PEP 008\n  compliance with the included script: `docker-compose run --rm blacken`\n* Commit messages should start with a capital letter (\"Updated models\", not \"updated models\").\n* Write new tests and run old tests! Make sure that % test coverage does not decrease.\n* Contributions are reviewed by a maintainer before acceptance. To facilitate\n  review, please make a [pull request](https://github.com/opendsm/opendsm/pulls/new)\n  and provide a description and follow the checklist in the pull request template.\n  Tests will be automatically run using GitHub Actions after a pull request is created.\n* Prefix new feature branches with `feature/` and bug fix branches with `fix/`\n  and make pull requests directly against `master`.\n* Contributions that add new required dependencies to the library will be\n  given a more thorough review to ensure that those dependency additions\n  1) do not pose a security risk and 2) are absolutely necessary.\n* Contributions that allow for data exfiltration by making external HTTP or TCP\n  requests will not be accepted.\n\nContributor maintenance responsibility\n--------------------------------------\n\nContributions of all kinds are encouraged, and we do not require contributors\nto be responsible for ongoing support of patches they make. However, because\nwe accept \"toss over the wall\" contributions, contributions deemed by the\nmaintainers to be too difficult to maintain will not be accepted.\n\nTesting\n-------\n\nPlease write unit tests for all new features. At time of writing, this\npackage has 100% test coverage. We would like to maintain that coverage level,\nbecause 100% coverage is easier than 99% coverage. This does not necessarily\nmean that all line are tested. For lines that are sufficiently inconvenient to\ntest, we maintain 100% test coverage by adding `# pragma: no cover` comments\nafter the difficult to test lines or blocks.\n\nThis helps us stay on top of un-covered sections without sacrificing the\nconvenience of 100% coverage and without being too overbearing about tests.\n\nTests are run using the following commands (flags are passed to the py.test\nexecutable):\n\n```\ndocker-compose run --rm test                         # run all tests\n\n# cheat sheet of variations\ndocker-compose run --rm test --no-cov                # no coverage\ndocker-compose run --rm test tests/test_features.py  # run a specific suite\ndocker-compose run --rm test -k compute              # filter for tests with `compute` in the name\n```\n\nTest configuration can be found in `tox.ini`, `pytest.ini`, and `tests/conftest.py`.\n\nWhen writing tests using py.test fixtures, place fixtures as close as possible\nto the test functions or classes that call them.\n\nGeneral Discussion\n------------------\n\nDiscussions for this project take place on the\n[opendsm@lists.lfenergy.org](https://lists.lfenergy.org/g/opendsm/)\nmailing list.\n\nLicense\n-------\n\nThis project is licensed under [Apache 2.0](LICENSE).\n\nAll source files must apply the following SPDX header:\n\n``` python\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n```\n\nDeveloper Certificate of Origin\n-------------------------------\n\nThis project uses the\n[Developer Certificate of Origin](https://developercertificate.org/),\nthe text of which is copied below:\n\n```\nDeveloper Certificate of Origin\nVersion 1.1\n\nCopyright (C) 2004, 2006 The Linux Foundation and its contributors.\n1 Letterman Drive\nSuite D4700\nSan Francisco, CA, 94129\n\nEveryone is permitted to copy and distribute verbatim copies of this\nlicense document, but changing it is not allowed.\n\n\nDeveloper's Certificate of Origin 1.1\n\nBy making a contribution to this project, I certify that:\n\n(a) The contribution was created in whole or in part by me and I\n    have the right to submit it under the open source license\n    indicated in the file; or\n\n(b) The contribution is based upon previous work that, to the best\n    of my knowledge, is covered under an appropriate open source\n    license and I have the right under that license to submit that\n    work with modifications, whether created in whole or in part\n    by me, under the same open source license (unless I am\n    permitted to submit under a different license), as indicated\n    in the file; or\n\n(c) The contribution was provided directly to me by some other\n    person who certified (a), (b) or (c) and I have not modified\n    it.\n\n(d) I understand and agree that this project and the contribution\n    are public and that a record of the contribution (including all\n    personal information I submit with it, including my sign-off) is\n    maintained indefinitely and may be redistributed consistent with\n    this project or the open source license(s) involved.\n```\n\nCharter\n-------\n\nThe charter for the open source project can be find in [CHARTER](CHARTER) and\ncontains the following sections:\n\n1. Mission and Scope of the Project\n2. Techincal Steering Committee\n3. TSC Voting\n4. Compliance with Policies\n5. Community Assets\n6. General Rules and Operations\n7. Intellectual Property Policy\n8. Amendments\n\nRelease process\n---------------\n\nPre-release\n\n1. create branch off of master named `feature/<examplefeature>` or `fix/<>` and make desired changes.\n2. edit CHANGELOG.md with changes under a new section called `Development`\n3. create, review, and merge PR for feature/examplefeature\n4. repeat steps 1-3 if desired for other features as convenient, though preference is for frequent version bumps\n\nReleasing\n\n5. bump version - edit `__version__.py` with the new version\n6. Rename `Development` section with the new version in CHANGELOG.md\n7. commit changes\n8. push branch and create PR\n9. merge release branch to master after tests pass and an approved review\n10. create GitHub release describing changes, creating tag `vX.Y.Z` to match new `__version__.py`\n11. approve the resulting pypi-publish action\n\nYou can use `bump_version.sh` to print out commands that rotate the changelog and bump the package version:\n\n```\n./bump_version.sh X.X.X Y.Y.Y\n```\n\nOther resources\n---------------\n\n- [README](README.rst): basic project information written in RST for PyPI preview.\n  Copied and lightly modified for formatting from [docs/index.rst](docs/index.rst)\n- [MAINTAINERS](MAINTAINERS.md): an ordered list of project maintainers.\n- [LICENSE](LICENSE): Apache 2.0.\n- [CHARTER](CHARTER): open source project charter.\n- [CODE_OF_CONDUCT](CODE_OF_CONDUCT.md): code of conduct for contributors.\n"
  },
  {
    "path": "Dockerfile",
    "content": "# syntax=docker/dockerfile:1.7\nFROM python:3.10-slim AS app\n\n# System deps (you had libenchant-2-dev)\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n    build-essential ca-certificates \\\n  && rm -rf /var/lib/apt/lists/*\n\n# Add uv (fast installer)\nCOPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/\n\n# Helpful uv settings: compile bytecode & avoid hardlinks\nENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy \\\n    PYTHONUNBUFFERED=1 PIP_DISABLE_PIP_VERSION_CHECK=1\n\nWORKDIR /app\n\n# ---- deps layer (cacheable) ----\n# Copy only metadata first to maximize Docker layer caching\nCOPY pyproject.toml README.md /app/\n# If you keep a lockfile, copy it too for reproducible installs\n# (safe if missing)\nCOPY uv.lock /app/uv.lock\n\n# Resolve & install *only dependencies* into the system Python\n# Using uv pip compile -> requirements.txt for a stable, cacheable layer\nRUN --mount=type=cache,target=/root/.cache/uv \\\n    uv pip compile pyproject.toml -o /tmp/requirements.txt && \\\n    uv pip install --system -r /tmp/requirements.txt\n\n# ---- project install ----\n# Now add your source and install the project itself\nCOPY opendsm/ /app/opendsm/\n\nRUN --mount=type=cache,target=/root/.cache/uv \\\n    uv pip install --system -e .[dev]\n\nENV PYTHONPATH=/usr/local/bin:/app\nWORKDIR /app\n"
  },
  {
    "path": "LICENSE",
    "content": "                                 Apache License\n                           Version 2.0, January 2004\n                        http://www.apache.org/licenses/\n\n   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n   1. Definitions.\n\n      \"License\" shall mean the terms and conditions for use, reproduction,\n      and distribution as defined by Sections 1 through 9 of this document.\n\n      \"Licensor\" shall mean the copyright owner or entity authorized by\n      the copyright owner that is granting the License.\n\n      \"Legal Entity\" shall mean the union of the acting entity and all\n      other entities that control, are controlled by, or are under common\n      control with that entity. For the purposes of this definition,\n      \"control\" means (i) the power, direct or indirect, to cause the\n      direction or management of such entity, whether by contract or\n      otherwise, or (ii) ownership of fifty percent (50%) or more of the\n      outstanding shares, or (iii) beneficial ownership of such entity.\n\n      \"You\" (or \"Your\") shall mean an individual or Legal Entity\n      exercising permissions granted by this License.\n\n      \"Source\" form shall mean the preferred form for making modifications,\n      including but not limited to software source code, documentation\n      source, and configuration files.\n\n      \"Object\" form shall mean any form resulting from mechanical\n      transformation or translation of a Source form, including but\n      not limited to compiled object code, generated documentation,\n      and conversions to other media types.\n\n      \"Work\" shall mean the work of authorship, whether in Source or\n      Object form, made available under the License, as indicated by a\n      copyright notice that is included in or attached to the work\n      (an example is provided in the Appendix below).\n\n      \"Derivative Works\" shall mean any work, whether in Source or Object\n      form, that is based on (or derived from) the Work and for which the\n      editorial revisions, annotations, elaborations, or other modifications\n      represent, as a whole, an original work of authorship. For the purposes\n      of this License, Derivative Works shall not include works that remain\n      separable from, or merely link (or bind by name) to the interfaces of,\n      the Work and Derivative Works thereof.\n\n      \"Contribution\" shall mean any work of authorship, including\n      the original version of the Work and any modifications or additions\n      to that Work or Derivative Works thereof, that is intentionally\n      submitted to Licensor for inclusion in the Work by the copyright owner\n      or by an individual or Legal Entity authorized to submit on behalf of\n      the copyright owner. For the purposes of this definition, \"submitted\"\n      means any form of electronic, verbal, or written communication sent\n      to the Licensor or its representatives, including but not limited to\n      communication on electronic mailing lists, source code control systems,\n      and issue tracking systems that are managed by, or on behalf of, the\n      Licensor for the purpose of discussing and improving the Work, but\n      excluding communication that is conspicuously marked or otherwise\n      designated in writing by the copyright owner as \"Not a Contribution.\"\n\n      \"Contributor\" shall mean Licensor and any individual or Legal Entity\n      on behalf of whom a Contribution has been received by Licensor and\n      subsequently incorporated within the Work.\n\n   2. Grant of Copyright License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      copyright license to reproduce, prepare Derivative Works of,\n      publicly display, publicly perform, sublicense, and distribute the\n      Work and such Derivative Works in Source or Object form.\n\n   3. Grant of Patent License. Subject to the terms and conditions of\n      this License, each Contributor hereby grants to You a perpetual,\n      worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n      (except as stated in this section) patent license to make, have made,\n      use, offer to sell, sell, import, and otherwise transfer the Work,\n      where such license applies only to those patent claims licensable\n      by such Contributor that are necessarily infringed by their\n      Contribution(s) alone or by combination of their Contribution(s)\n      with the Work to which such Contribution(s) was submitted. If You\n      institute patent litigation against any entity (including a\n      cross-claim or counterclaim in a lawsuit) alleging that the Work\n      or a Contribution incorporated within the Work constitutes direct\n      or contributory patent infringement, then any patent licenses\n      granted to You under this License for that Work shall terminate\n      as of the date such litigation is filed.\n\n   4. Redistribution. You may reproduce and distribute copies of the\n      Work or Derivative Works thereof in any medium, with or without\n      modifications, and in Source or Object form, provided that You\n      meet the following conditions:\n\n      (a) You must give any other recipients of the Work or\n          Derivative Works a copy of this License; and\n\n      (b) You must cause any modified files to carry prominent notices\n          stating that You changed the files; and\n\n      (c) You must retain, in the Source form of any Derivative Works\n          that You distribute, all copyright, patent, trademark, and\n          attribution notices from the Source form of the Work,\n          excluding those notices that do not pertain to any part of\n          the Derivative Works; and\n\n      (d) If the Work includes a \"NOTICE\" text file as part of its\n          distribution, then any Derivative Works that You distribute must\n          include a readable copy of the attribution notices contained\n          within such NOTICE file, excluding those notices that do not\n          pertain to any part of the Derivative Works, in at least one\n          of the following places: within a NOTICE text file distributed\n          as part of the Derivative Works; within the Source form or\n          documentation, if provided along with the Derivative Works; or,\n          within a display generated by the Derivative Works, if and\n          wherever such third-party notices normally appear. The contents\n          of the NOTICE file are for informational purposes only and\n          do not modify the License. You may add Your own attribution\n          notices within Derivative Works that You distribute, alongside\n          or as an addendum to the NOTICE text from the Work, provided\n          that such additional attribution notices cannot be construed\n          as modifying the License.\n\n      You may add Your own copyright statement to Your modifications and\n      may provide additional or different license terms and conditions\n      for use, reproduction, or distribution of Your modifications, or\n      for any such Derivative Works as a whole, provided Your use,\n      reproduction, and distribution of the Work otherwise complies with\n      the conditions stated in this License.\n\n   5. Submission of Contributions. Unless You explicitly state otherwise,\n      any Contribution intentionally submitted for inclusion in the Work\n      by You to the Licensor shall be under the terms and conditions of\n      this License, without any additional terms or conditions.\n      Notwithstanding the above, nothing herein shall supersede or modify\n      the terms of any separate license agreement you may have executed\n      with Licensor regarding such Contributions.\n\n   6. Trademarks. This License does not grant permission to use the trade\n      names, trademarks, service marks, or product names of the Licensor,\n      except as required for reasonable and customary use in describing the\n      origin of the Work and reproducing the content of the NOTICE file.\n\n   7. Disclaimer of Warranty. Unless required by applicable law or\n      agreed to in writing, Licensor provides the Work (and each\n      Contributor provides its Contributions) on an \"AS IS\" BASIS,\n      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n      implied, including, without limitation, any warranties or conditions\n      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n      PARTICULAR PURPOSE. You are solely responsible for determining the\n      appropriateness of using or redistributing the Work and assume any\n      risks associated with Your exercise of permissions under this License.\n\n   8. Limitation of Liability. In no event and under no legal theory,\n      whether in tort (including negligence), contract, or otherwise,\n      unless required by applicable law (such as deliberate and grossly\n      negligent acts) or agreed to in writing, shall any Contributor be\n      liable to You for damages, including any direct, indirect, special,\n      incidental, or consequential damages of any character arising as a\n      result of this License or out of the use or inability to use the\n      Work (including but not limited to damages for loss of goodwill,\n      work stoppage, computer failure or malfunction, or any and all\n      other commercial damages or losses), even if such Contributor\n      has been advised of the possibility of such damages.\n\n   9. Accepting Warranty or Additional Liability. While redistributing\n      the Work or Derivative Works thereof, You may choose to offer,\n      and charge a fee for, acceptance of support, warranty, indemnity,\n      or other liability obligations and/or rights consistent with this\n      License. However, in accepting such obligations, You may act only\n      on Your own behalf and on Your sole responsibility, not on behalf\n      of any other Contributor, and only if You agree to indemnify,\n      defend, and hold each Contributor harmless for any liability\n      incurred by, or claims asserted against, such Contributor by reason\n      of your accepting any such warranty or additional liability.\n\n   END OF TERMS AND CONDITIONS\n\n   APPENDIX: How to apply the Apache License to your work.\n\n      To apply the Apache License to your work, attach the following\n      boilerplate notice, with the fields enclosed by brackets \"[]\"\n      replaced with your own identifying information. (Don't include\n      the brackets!)  The text should be enclosed in the appropriate\n      comment syntax for the file format. We also recommend that a\n      file or class name and description of purpose be included on the\n      same \"printed page\" as the copyright notice for easier\n      identification within third-party archives.\n\n   Copyright 2014-2025 OpenDSM contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n"
  },
  {
    "path": "MAINTAINERS.md",
    "content": "# Maintainers\n\nThe OpenEEmeter was originally created in late 2014 by Phil Ngo and later\ndeveloped and incubated at Recurve Analytics, Inc (formerly Open Energy Efficiency, Inc) and The Impact Lab.\nDevelopment was funded partially by grants from the California Energy\nCommission.\n\n## Committers - Technical Steering Committee (TSC).\n\n- Travis Sikes (TSC chair)\n- Brian Gerke\n- Adam Scheer\n- Steve Suffian\n- McGee Young\n\n## Contributors (alphabetical)\n\nWe express great thanks to all contributors to OpenDSM. This is an\nincomplete list of those who have contributed code, documentation, or technical\nartifacts to the project.\n\n- Alyssia Byers\n- Armin Aligholian\n- Arpan Kotecha\n- Brandon Willard\n- Caleb Canchola\n- Carmen Best\n- Cathy Deng\n- Dave Yeager\n- Eric Dill\n- Hassan Shaban\n- Jason Chulock\n- Joe Glass\n- Joydeep Nag\n- Juan-Pablo Velez\n- kfogel\n- Marc Pare\n- Matt Golden\n- mdrpheus\n- opentaps\n- Peter Olson\n- Reetu Mutti\n- Tom Plagge\n- tsennott\n"
  },
  {
    "path": "MANIFEST.in",
    "content": "include README.md LICENSE pytest.ini\ninclude opendsm/eemeter/samples/*.json opendsm/eemeter/samples/*.csv.gz\nrecursive-include tests *.py\n"
  },
  {
    "path": "README.md",
    "content": "# OpenDSM: Tools for calculating metered energy savings\r\n\r\n[![PyPI Version](https://img.shields.io/pypi/v/opendsm.svg)](https://pypi.python.org/pypi/opendsm)\r\n[![Supported Versions](https://img.shields.io/pypi/pyversions/opendsm.svg)](https://github.com/opendsm/opendsm)\r\n[![License](https://img.shields.io/github/license/opendsm/opendsm.svg)](https://github.com/opendsm/opendsm)\r\n[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)\r\n\r\n---------------\r\n\r\n**OpenDSM (formerly OpenEEmeter)** — an open-source library used to measure the impacts \r\nof demand-side programs by using historical data to fit models and then create \r\npredictions (counterfactuals) to compare to post-intervention, observed energy usage.\r\n\r\n## Background - Why use OpenDSM\r\n\r\nEnergy efficiency programs have traditionally focused on addressing long-term load growth \r\nand reducing customer energy bills rather than serving as reliable grid resources. \r\nHowever, as utilities work to decarbonize power generation, buildings, and transportation, \r\ndemand-side programs (e.g. energy efficiency, load shifting, electrification, and demand \r\nresponse programs) must evolve into dependable, scalable grid assets. Ultimately, \r\ndecarbonizing the power grid will require both supply and demand-side solutions. While \r\nsupply-side production is easily quantified, measuring the impacts of demand-side programs \r\nhas historically been challenging due to inconsistent and opaque measurement methodologies.\r\n\r\nOpenDSM solves these problems with accurate, efficient, and transparent models designed to \r\nmeasure demand-side program impacts. OpenDSM gives all stakeholders full visibility and \r\nconfidence in the results.\r\n\r\nOpenDSM builds upon the shoulders of OpenEEmeter and the [CalTRACK Methods](https://caltrack.org/) which themselves\r\nare built upon the foundational work of the Princeton Scorekeeping Method ([PRISM 1986](https://www.marean.mycpanel.princeton.edu/images/prism_intro.pdf)) \r\nfor the daily and billing models and Lawrence Berkeley National Laboratory's Time-of-Week \r\nand Temperature Model ([TOWT 2011](https://eta-publications.lbl.gov/sites/default/files/LBNL-4944E.pdf)) for the hourly energy efficiency and demand response models.\r\nOpenDSM models have been proven to meet or exceed the predictive capablity of the \r\naforementioned models. These models adhere to a statistical approach, as opposed to an \r\nengineering approach, so that these models can be efficiently run on millions of meters at \r\na time, while still providing accurate predictions. \r\n\r\nUsing default settings in OpenDSM will provide accurate and stable model predictions \r\nsuitable for savings measurements from demand side interventions. Settings can be modified \r\nand sufficiency requirements can be bypassed for research and development purposes; however, \r\nthe outputs of such models are no longer OpenDSM compliant measurements as the modifications\r\nmean that these models are no longer verified and approved by the OpenDSM Working Group.\r\n\r\n## Installation\r\n\r\nOpenDSM is a python package and can be installed with pip.\r\n\r\n~~~~~~~~~~~~~~~\r\n$ pip install opendsm\r\n~~~~~~~~~~~~~~~\r\n\r\n## Features\r\n\r\n- Models:\r\n\r\n  - Energy Efficiency Daily Model\r\n  - Energy Efficiency Billing (Monthly) Model\r\n  - Energy Efficiency Non-Solar Hourly Model\r\n  - Energy Efficiency Solar Hourly Model\r\n  - Demand Response Hourly Model\r\n\r\n- Flexible sources of temperature data. See [EEweather](https://github.com/opendsm/eeweather).\r\n- Data sufficiency checking\r\n- Model serialization\r\n- First-class warnings reporting\r\n\r\n## [Documentation](https://opendsm.energy/)\r\n\r\nDocumenation for this library can be found [here](https://opendsm.energy/).\r\n\r\n## Future Development\r\n\r\nThe OpenDSM project growth goals fall into two categories:\r\n\r\n1. Community goals - we want help our community thrive and continue to grow.\r\n2. Technical goals - we want to keep building the library in new ways that make it\r\n   as easy as possible to use.\r\n\r\n### Community goals\r\n\r\n1. Improve repository structure, architecture, and API\r\n\r\nThe first step of being able to contribute to a project is to understand how the repository\r\nis laid out and how OpenDSM is architected. We have made giant steps in this area as of late, \r\nbut there is additional organizational work to be done. This will continue to be an ongoing\r\narea of work.\r\n\r\n2. Make it easier to contribute\r\n\r\nAs our user base grows, the need and desire for users to contribute back to the library\r\nalso grows, and we want to make this as seamless as possible. This means writing and\r\nmaintaining contribution guides, and creating checklists to guide users through the\r\nprocess.\r\n\r\n\r\n### Technical goals\r\n\r\n1. Update the Demand Response (DR) model\r\n\r\nIn the most recent release, the hourly energy efficiency (EE) model has been entirely\r\nchanged and updated. Much like the billing model is to the daily model, the DR model is a\r\nsubset of the EE hourly model. Many of the improvements seen in the EE hourly model could\r\nbe realized in the DR model if it were finalized. It is currently in a functional state\r\nwithin a branch, but its parameters have not been optimized rendering it unusable for\r\nmeasurements. In the meantime, the existing DR model is still available.\r\n\r\n2. Reassess existing sufficiency and disqualification criteria\r\n\r\nThe existing sufficiency and disqualification criteria exist as conservative estimates\r\nfrom OpenEEmeter and CalTRACK recommendations. There is almost certainly room for these\r\ncriteria to be revisited so that more meters would pass and be approved for measurement.\r\n\r\n3. Determine the sufficiency requirements of PV installation date in the hourly model\r\n\r\nThe hourly EE model currently has the capability of ingesting a PV installation date and\r\ngenerating an additional feature that can much better represent a meter who installs a\r\nsolar PV system mid-baseline year. However, this feature currently is classified as\r\nexperimental and not allowed for official measurement because we have not quantified how\r\nmuch data is required post-installation to be able to accurately predict the meter's\r\nbehavior in the reporting year.\r\n\r\n4. Improve the daily model\r\n\r\nThere are two potential areas of improvement of the daily model. First it could be extended\r\nto allow additional sources of information, but this must carefully be considered as the\r\nprimary usage of the daily model is to be able to disaggregate heating and cooling usage.\r\nThe second area of improvement would be to allow an additional break point within both the\r\ncooling and heating regions such that the model would be able to change slope. This should\r\nlikely still be limited such that the model's slope in each region is appropriately\r\nconstrained. A new smoothing function would also need to be developed.\r\n\r\n5. Integrate EEweather\r\n\r\nEEweather is commonly used to obtain weather information to be used within OpenDSM. If it \r\nwere more tightly coupled, it would streamline the most standard use of OpenDSM. As an \r\nexample this could simplify several of the data classes such that the aggregation of \r\nweather data would be done within EEweather instead of within data classes where it is a\r\nmore complex procedure\r\n\r\n6. Integrate GRIDmeter\r\n\r\nGRIDmeter is frequently used after DR/EEmeter in order to correct models for external\r\npopulation-level effects by using non-participant meters. Similarly to EEweather, this \r\nprocess could be streamlines and made more cohesive by fully integrating it into OpenDSM. \r\n\r\n7. Organize and revise existing test suite\r\n\r\nThe existing testing suite is the last remaining vestige of the library prior to the \r\nextensive reorganization and API changes made. It would be well served to update the\r\ntesting suite to make it easier for future contributors to know how and where they \r\nshould develop their tests for any new features or bugs found.\r\n\r\n8. Greater weather coverage\r\n\r\nThe weather station coverage in the EEweather package includes full coverage of US and\r\nAustralia, but with some technical work, it could be expanded to include greater, or\r\neven worldwide coverage.\r\n\r\n## License\r\n\r\nThis project is licensed under [Apache 2.0](LICENSE).\r\n\r\n## Other resources\r\n\r\n- [CONTRIBUTING](https://github.com/opendsm/opendsm/blob/master/CONTRIBUTING.md): How to contribute to the project.\r\n- [MAINTAINERS](https://github.com/opendsm/opendsm/blob/master/MAINTAINERS.md): An ordered list of project maintainers.\r\n- [CHARTER](https://github.com/opendsm/opendsm/blob/master/CHARTER.md): Open source project charter.\r\n- [CODE OF CONDUCT](https://github.com/opendsm/opendsm/blob/master/CODE_OF_CONDUCT.md): Code of conduct for contributors."
  },
  {
    "path": "bump_version.sh",
    "content": "#!/bin/bash\n\nset -e  # fail script on any error, show commands\n\nOLD_VERSION=$1  # e.g., 0.0.0\nNEW_VERSION=$2  # e.g., 0.0.1\nNEW_VERSION_LENGTH=$(printf \"%s\" \"$NEW_VERSION\" | wc -c)\nDASHES=$(printf \"%${NEW_VERSION_LENGTH}s\" | sed 's/ /-/g')\n\necho \"git checkout master\"\necho \"git pull\"\necho \"git checkout -b release/v${NEW_VERSION}\"\necho \"\"\necho \"sed -i -e 's/${OLD_VERSION}/${NEW_VERSION}/g' opendsm/__version__.py\"\necho \"sed -i -e '/Development/,/-----------/ c\\\\\nDevelopment\\\\\n-----------\\\\\n\\\\\n* Placeholder\\\\\n\\\\\n${NEW_VERSION}\\\\\n${DASHES}\\\\\n' CHANGELOG.md\"\necho \"rm -f opendsm/__version__.py-e\"\necho \"rm -f CHANGELOG.md-e\"\necho \"\"\necho \"git commit -am \\\"Bump version\\\" -s\"\necho \"git push -u origin release/v${NEW_VERSION}\"\n"
  },
  {
    "path": "data/attribution.txt",
    "content": "This data is derived from NREL's ComStock datasets (2023/comstock_amy2018_release_2)\n\nThe ComStock database, part of the Open Energy Data Initiative, carries with it a Creative Commons Attribution 3.0 United States License \n\nData includes information from the ComStock™ dataset developed by the National Renewable Energy Laboratory (NREL) with funding from the U.S. Department of Energy (DOE).\n\nCitation:\nParker, Andrew, et al. 2023. ComStock Reference Documentation. Golden, CO: National Renewable Energy Laboratory. NREL/TP-5500-83819. https://www.nrel.gov/docs/fy23osti/83819.pdf"
  },
  {
    "path": "data/features.csv",
    "content": "id,summer_usage,winter_usage,annual_usage\n108585,28424.455657276467,51265.146636654434,110748.6876074653\n108587,17291.77416917098,56050.39252082105,102265.57253033188\n108596,45599.89343243372,71591.15134416716,161796.56455473122\n108597,27858.563654001788,82835.7574326724,155789.54100966017\n108603,97793.92145670383,276022.5002888874,528779.375284755\n108651,26160.530202715727,65061.904466105465,129230.03731760167\n108652,17552.40115501821,37869.85146561201,77774.65212119538\n108655,75288.9697483809,170230.63370364887,326396.2163047299\n108657,30980.13281801004,107870.74777130758,192407.84824440558\n108686,204864.9369271906,636312.814306768,1217549.5227783667\n108693,87931.49171389916,133539.41024547044,311735.0087644528\n108704,70104.62995720006,332442.1135049312,583015.7190204449\n108710,928444.8606734825,1401593.2730090427,3409275.840730432\n108755,4578.994754768554,11293.42045303014,22698.127519789177\n108762,77702.74459206148,196618.16560514158,385986.2084317622\n108774,296544.8136611258,597320.1174702568,1275838.6052017317\n108775,253317.3316363064,267008.93643020507,731209.7498958404\n108789,751423.2701812203,753035.205485595,2228706.855953003\n108791,229352.75901112528,315200.2131346425,747272.6587924648\n108794,4566435.61970002,5031306.731039737,13493030.335085198\n108802,6456.829795661001,16088.638750352176,31543.307951441275\n108813,2164826.0496044573,2485732.390407298,6443783.41046498\n108819,96738.45226799279,238236.79392594352,458941.65329019574\n108826,175792.146601842,392430.40497245453,798595.0522383602\n108838,88345.54081954804,81888.2212437981,245419.94884663622\n108841,756493.9058922682,2625463.4448502515,4882942.8515290115\n108845,80193.80045846401,146492.99803848972,304747.75451275456\n108860,2891331.789714103,4489880.26975092,10627990.863640891\n108880,1529881.781145284,2755883.4423649716,6145479.0333141815\n108881,12036.3093042342,43865.57711884676,79839.00373574029\n108894,763718.0397815474,1075727.8790622756,2627020.3550714934\n108900,9982.317573662625,75082.3869098902,121112.89423403774\n108909,16414.576951083276,46416.42373200657,87554.8672887863\n108910,37372.410668344615,50905.52998611898,124846.6515128921\n108918,302428.0249960874,830313.6328558978,1675215.1820036839\n108935,16156.707215387223,79035.373745478,133559.96341950464\n108936,61445.73792056389,125102.7427156253,254706.56644278282\n108971,11437.283863123388,53333.77118995469,88699.17991168608\n108977,11151.606930669763,26491.374536162482,53087.79290406818\n108986,58682.64759426187,82932.34868510069,198890.87919706744\n108995,26619.22424145556,103069.21611792114,181269.21946519392\n109042,434614.2535863034,787655.885022473,1747936.877742305\n109045,304580.5049353946,641980.4499847923,1358028.8142834431\n109071,187694.34615476077,561413.1607904257,1062617.5019655507\n109092,71036.92448869723,152159.29360684875,307504.62535771384\n109095,22310.88375471595,49137.64120994376,95810.01786983013\n109110,60074.40292020607,217167.0225529279,385414.7769141169\n109115,15735.139682764182,39068.07624005466,74715.2986222392\n109116,31762.331595920183,69894.74133948928,138200.95159939904\n109121,11286.936338302492,40035.352541114386,69168.12009995454\n109141,243694.6172802105,680282.7759652381,1284251.8732881937\n109146,423542.8075956412,525223.3007873202,1347276.5443359194\n109147,36431.37610492465,80668.8537418069,161575.41089002092\n109156,86794.51309663536,132988.46722084488,301801.9244147117\n109157,57193.28142244244,151181.74791478834,280480.7917514395\n109164,58426.109148332194,78207.48254947808,190807.82403267146\n109165,73537.33311098159,127520.85423452435,276756.77500059805\n109191,17077.632311320547,137198.73289311072,216129.76532859076\n109197,245274.18543939633,647378.841969841,1230584.6290563946\n109204,30900.81391869883,71011.80002638175,143656.80035822548\n109223,409490.0883282482,1080398.1599387976,2095000.2793083421\n109227,755047.573482862,1150228.4608183617,2663810.6889429083\n109233,25513.780322368915,44005.402951011965,96447.13701977482\n109239,583968.9867058506,1586847.8904296295,2959754.9023031695\n109272,5695.427619677451,19138.11776681321,35291.78906359293\n109307,26594.213863059023,48538.21988840076,103925.07171656172\n109313,15749.127886598299,33715.40393945337,67963.1175337252\n109315,1716478.7218046603,3715491.511817724,7792067.509881166\n109323,3830.9334131309392,17254.046871847335,30186.16179792244\n109339,9799.340090536245,32240.039322096807,58398.94295280102\n109349,970547.2213669692,1366529.8733000734,3273283.400869062\n109382,56824.25237535828,115431.10136221701,240864.54289703898\n109391,39590.111199957435,95103.3965919478,188011.6375191657\n109399,70152.40984649709,110673.80633547135,246026.63559706975\n109401,3455736.2359923096,5216362.598000214,12219075.629654864\n109423,25656.37906509535,28907.55108554573,71762.44576007321\n109429,98480.9154553615,141659.8295606337,332920.37947642285\n109435,69128.3654216813,159679.77246504772,318771.8309556672\n109436,103139.11791720455,192519.08848764532,414106.8997169414\n109442,43359.94431738937,103849.8878685267,205244.8344006705\n109460,12307.16838968357,35819.052538083204,68205.4226424396\n109466,673895.9801360178,1877963.7237054573,3786397.527951601\n109485,668660.8508147063,2171777.1925018146,4006412.4569959133\n109487,19800.76797205962,74051.8484055479,136739.00990953733\n109496,155049.85461633038,311898.6793311906,654732.276571237\n109505,73860.31117305795,122739.4150254291,264775.32797184907\n109507,23869.99150737605,47129.520574426744,97808.12258815706\n109510,13856.19729096719,26705.27559237736,55636.37104486032\n109525,2336167.247165519,1241697.861994978,4894787.990245841\n109536,26992.674559205443,27138.197189776645,70725.706440575\n109548,105142.43978074087,222831.28099693923,444604.8356986926\n109555,10535.022260011949,44953.78946891175,77726.60101625822\n109557,34288.96159086555,41622.27936767878,108438.5579744472\n109563,340811.6429228909,792652.1780628187,1620661.17308453\n109582,87269.63756213585,93895.78635911668,249444.24081210146\n109614,240599.47478949552,890225.3754690214,1601883.4573801653\n109624,25661.797306788692,65867.40412495454,130878.41398509966\n109639,10954.38038739349,29890.15344602007,57998.36204012297\n109642,78584.96996832202,131875.77797237426,290454.7844857356\n109684,23587.3188082779,121241.54501360582,205952.49211596762\n109706,30358.000430104094,69297.43956387136,138941.6789405493\n109711,11189.770807313815,26334.292168421613,51869.571615127454\n109722,26080.27626451635,71318.43304303032,135289.36044660505\n109725,66994.32072934101,124785.70111374487,265178.3762755822\n109736,29130.64776446218,93790.0196481206,175911.52609930845\n109751,292536.15758783696,686683.9405579577,1378884.804452448\n109754,508220.0525213966,666509.9022386434,1709262.2512774887\n109756,35584.11581950422,55603.39915300441,126952.39150268698\n109761,5265.674417434389,24762.492580383696,41882.9683470156\n109773,202492.03540065375,804976.8934805684,1455303.2181775598\n109784,575800.4724498701,1392435.9849551367,2767356.7715669777\n109791,9783.81939838401,18757.071754255612,40467.28937055511\n109802,82048.2491070004,193420.37798852412,385261.6411201933\n109820,29029.71276803281,192175.3397550924,309884.58347554656\n109824,105792.76645980746,237558.63040659693,480309.8999190554\n109842,100139.98970948963,188595.61634942016,405331.8247959583\n109852,361535.9513174802,806945.4782259733,1633115.2332891012\n109904,551731.210922437,1563319.6544400838,3000610.9925168473\n109908,670447.3309562012,711181.7697402253,1983592.1538174802\n109909,323458.5081197222,476859.8735356384,1114551.33974711\n109910,346222.33012437826,916191.0424487699,1755320.661840558\n109911,560053.9243353424,823254.4279016029,1935636.9343242464\n109919,50728.89259956678,152912.46974169064,282414.3552747856\n109932,558637.0474154386,1067563.9298259404,2343455.0735655758\n109940,294243.04554658633,414048.23639617406,1032372.8267706224\n109954,90044.55222101921,133180.61678245143,312583.944572065\n109961,36424.12958942056,44698.12660073071,112782.9179429349\n109984,34107.79721751108,100293.47649559965,191233.46072087053\n109985,8907.46887030376,15605.538190305577,34616.086138354534\n109989,76476.59719438956,118085.41378354725,270584.482760269\n109993,532325.7664545825,1377773.453172427,2784628.261163854\n110013,36375.71159903216,92446.2220853534,186672.47966372033\n110015,28248.35688911519,105718.12065523748,181809.62330008642\n110026,15583.83265919984,40932.38234870323,79244.64071729332\n110045,30491.03298352555,73951.44534566389,138756.15067484396\n110054,92738.52221405825,154311.7062306218,340297.4184239004\n110061,54442.11899377964,97060.54696864125,209767.60812669288\n110100,383040.8188285828,479404.1637704654,1227026.9965745122\n110174,30230.75607768168,72386.85090191642,137752.78144193068\n110201,84914.66690886009,149142.9144303539,318872.995047395\n110207,517949.56789658533,1301463.574858435,2546585.513058053\n110210,5110.684314971875,16413.051559766485,29131.08271906451\n110276,11753.007993439673,32378.078662080818,62004.6394302097\n110285,524503.9370868615,811152.2650682701,1965709.2867120313\n110298,20399.787992415007,77405.11922151176,139214.8486459565\n110306,20468.986260878668,64729.56287322833,117883.95827478485\n110320,79342.84040450842,177441.3298930096,360879.99515555834\n110337,94471.78159735083,249061.4301485885,485230.5040969614\n110341,34534.669215650545,68674.62448968357,143748.1933456712\n110347,89623.77630826482,278272.5281454649,536832.5729610744\n110369,284746.7612462691,967647.7746448761,1733551.2763582494\n110405,152550.1491070675,335559.91963363934,675194.7868459879\n110407,364817.8922956823,758103.722184105,1571185.1534312647\n110412,220544.94772166578,247030.43467318028,645673.7847356452\n110420,30637.06019118702,55857.003046563404,120290.05435621277\n110423,75102.79974373739,233932.6215720956,446869.5106070981\n110430,1889286.854275218,1302858.3835225396,4480945.092347421\n110464,617098.8000944471,776754.2409303589,1955497.6919738376\n110479,21485.148320530316,39345.225135032764,86409.29427336587\n110482,489735.5653727088,1228612.2150427394,2492741.4571270654\n110501,72437.50258495432,97936.154867367,240656.93078306055\n110518,13118.975080968714,31649.345377455895,62799.112187044266\n110526,36841.167418686295,111947.59565917306,208552.13823977776\n110531,27962.290604397043,74304.29961191995,142330.0641124844\n110544,5173.531478869909,27843.909881848747,46534.252779124676\n110572,712662.9998589829,2267358.891224124,4236473.133063468\n110593,11937.19218596727,59030.713219658835,98145.01215322239\n110602,74927.29857677293,172724.7581601253,342175.35418456607\n110607,953438.5513859702,3297661.153131462,6262351.444412022\n110635,183562.59226941824,234601.61999496506,601552.050961664\n110655,189505.4925081774,388487.61509072105,849713.2713734366\n110661,119329.97896271403,119649.59417708343,338862.05809151183\n110674,56086.70062215068,89254.0901302451,203696.19861983182\n110700,304580.168501417,484169.8935653748,1092523.3983917297\n110734,42965.708543707005,79650.67311863053,172271.11321235326\n110736,9802.531774794126,51570.75603164245,88483.06203533156\n110747,100028.47528693151,201223.52348102594,422428.3319839319\n110749,2522852.7367831203,5664422.868418666,12031770.865676697\n110755,8814.216023447792,47995.70175386554,77394.32476640174\n110759,20010.520087434208,81732.75157639604,141472.24969867812\n110760,17362.592355269542,43851.476657769155,87010.31654709778\n110767,71335.21563228121,175850.69893782545,343335.9787734037\n110770,20863.47280936251,46030.98166682045,92751.65254639764\n110776,2095427.329363593,1934667.7848874857,5731023.165555406\n110786,130366.89653812838,264536.7716323912,553380.4975434691\n110806,5668.315135877892,17831.089301665776,33320.6336616209\n110811,2813323.561633772,3253551.8911795258,8892869.09756994\n110819,585111.7052975297,1705187.710199005,3266418.1197835845\n110843,298015.73167628684,1071945.5215950618,1940300.336690293\n110855,355282.4067141081,407712.90539134294,1062328.8494881475\n110858,422888.3465659715,462164.3382850601,1266121.119755021\n110866,15279.09799559048,62473.07538082051,108335.37310408297\n110879,279648.9242639902,295527.8056433971,830932.5932352988\n110896,1336830.762110125,2986251.057895063,6179835.92490392\n110916,76269.44739899004,202022.83043911736,391818.5344349639\n110923,6028.783758711779,18189.37806434113,33239.06129098698\n110926,19167.202502570646,98880.94157994307,166513.13968675392\n110941,18154.498095969644,43100.26422646143,83095.96940547602\n110953,539802.5577255497,1402521.3189941589,2794459.487446213\n110960,56743.74018267591,99642.36164880775,217749.57012295135\n110961,10941.844405752596,29524.330123595177,57406.45912654209\n110979,26041.27940796158,59591.28840657948,116661.48701753834\n110987,57699.4762801046,153559.35104009995,291610.462122672\n111005,12884.229174070613,31780.34954128141,61064.4235835546\n111010,200784.23965535432,438083.5968551758,870532.1123546408\n111025,90129.79118725288,115147.69644202694,291566.4354261071\n111032,32419.424007988873,98081.00433518052,184487.66946431925\n111039,35317.91459122786,64400.32817481204,135265.822219945\n111045,13497.296456924638,29162.78715628362,59510.77131447202\n111058,80596.34685886477,191915.49644962835,375600.184114142\n111086,26412.18009706588,98788.259926076,171813.32423010457\n111104,345634.51585793175,1006252.8157856314,1951632.3123276285\n111124,26527.390086960815,34827.537842767015,87003.03773756308\n111141,418873.34721236466,803386.569726896,1792545.4116305215\n111144,170533.80759572104,400033.43126611615,814727.7817054617\n111154,32871.2605307436,75051.1275681339,152848.8683234495\n111173,173287.07846811556,248864.7632139494,578343.4958544201\n111178,35910.32142093095,82981.12591516614,165488.47736297973\n111215,497025.8340097769,1326426.9032200237,2584909.1275081593\n111216,503075.27660192386,1263034.8388707137,2484764.042686949\n111221,168759.7788449644,245663.55578691728,578966.9528507788\n111229,3850.4512301615164,20914.829223969344,34588.152605327065\n111240,59399.22068416753,146851.82714069056,286860.05005838873\n111246,132714.48521682987,405989.2206000731,748409.0505340759\n111259,22506.58565912663,70330.61668645518,128372.83327774024\n111275,892436.4661145017,1812701.7633754977,3765556.718710012\n111276,23644.024940528805,50560.5528655665,102359.88036184068\n111287,182995.70994377547,230169.1711207894,583639.9298599799\n111294,1304757.99732442,3416376.6476700674,6775889.211359421\n111295,5084.644962281408,18159.00857975745,33102.9049627304\n111316,162484.50307331127,351530.38657656766,726386.2966236182\n111343,117111.29003077964,307488.72827454715,560345.7846546848\n111359,19118.93115204286,34168.313118065686,74760.59046896908\n111366,164241.5523428373,210113.4500176746,524820.6264213927\n111367,10304.942723484866,44211.10007453441,77964.43380141209\n111388,2002353.286828448,5864363.421049746,11145181.319192998\n111396,67807.36294414499,165592.6622990459,325289.2987650689\n111405,17501.95665509452,53605.078881288144,103295.82585985793\n111410,29914.7656918735,135642.14744050105,226611.55751617588\n111421,346777.61914073525,499118.05437244114,1212571.532852346\n111433,4848.588631289309,10882.661958202054,22514.120417771264\n111435,876652.7680232688,1876413.877705445,4052198.617559261\n111441,113285.61672730769,217058.60344694438,467916.72420008556\n111442,30584.186122115116,99611.18258691524,179757.39324343775\n111463,22456.2799573561,31720.856564317488,75077.35846544779\n111473,19788.8298873669,35354.66618785411,76340.63432793497\n111482,30797.105014000565,70546.98928813198,142133.0623538295\n111497,450911.71062040725,972822.5799561235,2117532.525031495\n111498,29913.319213144438,96020.14264476205,178788.51959313924\n111500,400703.6802814891,680276.000712048,1541822.302270629\n111511,991422.8341799468,2084749.0834999357,4501415.280986286\n111524,39403.69975923851,202639.34162570897,349789.0988625699\n111529,3350.69990366796,12195.792408456166,21751.795669101113\n111541,35977.97701967293,121161.29488062132,215692.5036464156\n111542,175661.96572441366,264971.3301601198,610057.7748392848\n111548,8700.913218660347,23746.86649606908,45418.92171194243\n111587,20515.057857954387,60089.040503820084,115483.44166337274\n111591,38507.32034261938,122146.52581876879,230935.30995684737\n111598,319280.6836213173,478244.11904296704,1156717.2576331832\n111605,106442.10640165502,165214.28825483244,380664.37571382243\n111617,125745.21462790422,330437.8808772595,658876.6676591157\n111618,2470564.965981888,2547232.2684425637,7192255.721788004\n111622,482995.3532154501,627441.2952775286,1570855.3044816775\n111629,25998.669966966612,62825.271205994424,123479.03025784745\n111634,263120.4210975702,435193.4758903056,985471.9629143046\n111640,4109.493547192861,11325.991056242148,22060.31318010378\n111642,12570.816374741267,52227.89791616331,91273.38560743374\n111644,6599.440839407982,19420.335429257044,36627.355398599735\n111649,108677.6531766686,216123.05425841577,456476.6646574858\n111652,39545.052165668625,61014.992724809315,140170.27918600323\n111658,3553.3509977409753,15654.225159850208,26914.55144835026\n111666,11828.734379583588,26718.257841697625,53572.79658407054\n111667,41818.09831847896,55828.03825669244,134877.20910026558\n111679,10458.557058829147,45917.88444242998,82473.43893609353\n111688,477588.5928734846,1970411.4084508687,3229697.568202638\n111702,13276.123365053658,54983.64888117858,97545.67654151365\n111705,103985.47567326522,143552.7492059367,350916.5127242091\n111710,13813.892346334573,53672.02327493114,95801.5543197516\n111715,130332.48716810292,149913.47134966915,394499.3175268114\n111717,430797.3279213949,580461.4341053859,1484491.0714287446\n111729,39109.50968443736,146315.85837437626,263290.3625645276\n111747,406977.2744278562,1175962.9894084884,2276578.6080523757\n111751,31177.54832018691,53578.69433241313,116853.0149212386\n111752,107750.80819095294,195163.2913704763,423774.7541290566\n111759,48940.96673883625,281252.9955535073,464517.4565628191\n111776,693870.6099562737,1659229.139798476,3343705.917489353\n111785,51801.38048010441,213198.72268038106,372468.63077485963\n111786,59309.22060147371,73359.43607460508,178279.8266770923\n111803,67120.38653327063,138875.94950140826,280679.3589975111\n111814,82587.46856323587,80997.79182977458,236122.7287531448\n111830,513897.6985957042,1355681.4776460903,2681258.7327787485\n111832,527779.5554865289,793209.2334681348,1874856.0397961452\n111833,14375.511683307297,23320.378164718302,53598.07388509589\n111835,45440.367363093545,93747.51741074954,191820.843111485\n111851,1439196.1101493808,1770371.7571443524,4569735.083958983\n111855,47349.33964111599,103582.30531909484,208693.4300160326\n111864,75310.1191883706,124256.24101637185,274151.9882834706\n111873,5341.042350579448,19161.463348306108,35987.240120315546\n111875,30154.994966032253,123629.17400068263,213231.03668991206\n111896,252739.0166226114,709683.2130254998,1345208.64882726\n111898,38714.234134170954,46110.82266402302,119809.79404532342\n111900,19754.848900335204,39629.723543878135,84359.2242135414\n111901,44326.68253626542,133273.6959077171,243796.9909593555\n111924,1170809.8186767288,2283596.0457740584,4916057.963905942\n111925,2628.3002039505886,13510.212347430797,23576.200555712585\n111926,619122.4612610543,1997286.8550329115,3591704.0288278116\n111927,18579.95983109279,29008.835920299945,66950.52425957429\n111942,37617.01293933301,67340.15238497617,144703.4503517694\n111943,27030.66453644585,37046.24087597697,90165.62557822594\n111957,175840.00972852387,442215.5041986842,898817.9768708372\n111966,12049.99377720108,36963.752247711214,68321.69305916394\n111995,218338.6368451753,840817.2580345063,1476027.4039820388\n111996,10702.009001786986,75913.99523288193,122213.71998292455\n111998,117310.92805119578,216123.8151270684,454568.1785051327\n111999,79388.03277900319,197872.4829543546,388971.0399256173\n112011,13561.637091878816,45651.526592765935,79985.20466641121\n112012,87358.51738332844,152740.7359643424,336212.08535585285\n112022,24800.14058760492,62053.579113834625,120937.49146303689\n112028,40366.33063257207,60469.58063888785,141744.9125214439\n112048,5567.277641730956,16118.37285619574,30189.488531007148\n112073,367106.06865410105,943630.5766531556,1852681.1656441875\n112090,4991.537506682168,17051.966223732932,31558.104180373142\n112092,63675.04661882535,114297.58634963383,248700.70005442365\n112093,17517.738853185878,35481.59502067117,74315.01647052988\n112104,33358.047830794225,34573.61563941735,96245.31009928152\n112107,38291.5378216192,102467.46787270707,198117.8477531593\n112111,30184.904949721065,67827.66686881987,139167.6160115375\n112122,23646.27162544345,82882.97966945788,150600.22592655904\n112130,25533.635179175057,62406.455264873206,122897.4365894357\n112135,15510.977626687614,39969.75593224391,76101.9806930237\n112141,69575.3182541423,215551.01203261167,382389.70507928776\n112146,2162139.3033410013,3910749.8070338313,8765277.500770226\n112159,63157.43545363745,108317.1011484848,235722.14687957102\n112189,565155.8249302659,1623920.2423410828,3123632.5093467683\n112193,394918.0986766618,783453.9389088909,1645784.8117441782\n112217,34529.98036162312,76725.94324789428,154350.5017645678\n112219,365187.7336651416,731704.958947959,1558909.6173903127\n112244,74519.98006020073,177053.59969274735,338921.0384805568\n112253,604854.5127073673,671125.2038290737,1809850.1198671302\n112261,52176.20437635328,75645.2129369367,177496.37252233704\n112263,577562.762012336,890641.3312422383,2061062.567310146\n112265,174408.1391585591,419218.56329641695,837802.0414393013\n112272,220219.48625518536,555826.3211977483,1041387.6801486795\n112289,498391.52923377487,609607.5851981124,1573560.8465014505\n112293,11766.558180000191,30921.001395887917,59543.382816768295\n112298,74178.48692031758,219500.51442832814,417532.9544794017\n112299,57344.61009014785,79700.69664916539,191313.0395153574\n112305,138831.50776026788,281283.9168955533,577221.0171863064\n112330,53545.97275468988,156865.9528312194,294650.46167336777\n112334,22361.577285127787,51600.13436663577,103971.52502271796\n112347,1638443.8070121456,2074486.278820531,5213515.2467879355\n112352,18023.50835218129,111653.28925221699,177550.54669210978\n112366,17715.2113921926,49433.76927477629,92069.41127847692\n112370,278912.6623628346,656007.0214709713,1347535.3979550437\n112378,351213.6517729347,621346.4254780734,1371267.0734597174\n112390,14928.867132621639,26233.70206763512,56494.18950597548\n112412,12720.734036940912,35538.26830432132,68191.96911596536\n112419,34361.06458524287,82766.6107101874,160594.97892736678\n112420,1649783.2575514652,2366603.1877525337,5783167.901836886\n112426,19959.163198738614,106165.7306700591,177951.9309561926\n112427,69571.75813977089,90563.50136399512,227929.0388103302\n112431,191638.26822974934,347240.793849402,743906.6235398473\n112456,10465.171732840183,31208.74180519711,58465.105878853996\n112463,16170.501622562399,47983.490141808754,91159.61590704862\n112470,64162.883947878705,140201.73816671944,281308.2278263124\n112483,14797.277432560879,29561.92979523865,62172.608371733964\n112491,34531.31648948359,81212.00749849518,165091.78363254777\n112498,269948.6789597357,526227.2115129272,1088718.2825500853\n112505,93894.05852802882,175783.86724262577,372152.1373077112\n112508,213020.596096017,485038.7346112244,1004442.5541354212\n112547,17553.07884115726,43225.838380602734,82277.14640678649\n112570,380670.7480317859,510047.20875040506,1258591.0016129692\n112571,69461.08437970799,151174.38551276023,296833.0452821262\n112585,21762.293694946264,130439.53998242175,212974.1821843383\n112613,221828.87382579525,323572.8956992301,774883.0549232597\n112614,65483.67384382545,331790.56550635427,557893.5637201652\n112616,21636.32566626148,56576.120784784885,107827.36800355616\n112623,10756.22411305592,26359.679639324266,49690.47920494591\n112624,22925.112865753883,109365.3157040597,190559.57114957453\n112625,59460.33690291659,158770.20665447318,300180.61451250606\n112629,25834.259783266036,38830.195765787445,89847.7613586917\n112630,90911.62205601917,158823.36999979147,355655.5728994832\n112650,281920.1523116252,553835.1670405237,1164181.2366016426\n112654,393123.1123374717,1065625.564573598,2081561.6609154432\n112675,13129.17883813403,62061.450953731095,111837.52183402912\n112677,85907.2242017839,157564.72536902002,340412.5468202107\n112685,521256.2212712633,543721.8290922284,1464823.6240824685\n112700,63334.42007849467,179147.2133451052,341047.7129808259\n112703,20764.637948719163,85523.7582946885,155029.7690094593\n112705,31913.399609998705,69735.63057029588,142097.17495426998\n112718,64412.67467348633,87568.94322620786,215796.15236484716\n112720,3949.629406366759,11450.836690503484,20944.961117654762\n112721,4816.137260708506,14796.239329468843,27495.529478736367\n112722,246349.57105715314,598351.4158009703,1188500.664858813\n112723,15982.125444459018,40793.91054691546,78266.68580411168\n112745,231145.70239373407,904141.3005179419,1668799.3740362893\n112757,14308.46422105328,37208.429330323954,70633.19037095048\n112769,70831.35732894189,106864.237358303,252187.76155900845\n112772,627993.205614924,1802819.6832279323,3432063.2014698917\n112780,53514.54789361286,135282.3198540381,275668.6334110627\n112781,16800.812169903587,84264.46232770164,140595.6625418855\n112792,31674.26819224571,82724.85567468968,165284.69533380182\n112816,362313.5020324643,296800.62921826215,950478.7955543229\n112832,29438.25607765984,59548.326815915636,123497.80617768424\n112833,148974.39317633832,190176.80983515413,480068.2196691965\n112836,311509.3284409301,1787070.502528639,2972650.599411234\n112841,242575.227606159,317604.8256844084,781321.2245256815\n112848,15212.790931572492,31407.776193044985,65755.152003797\n112849,16906.338123646958,52313.01998245885,96573.70516150763\n112875,263138.5079411826,581063.3551765559,1183083.2899416266\n112882,1429746.4738386397,1859426.764391258,4682234.191626798\n112896,163328.42908902335,375739.4666584308,752183.855728377\n112900,450219.7702837388,1608434.3427129192,2838622.7871817765\n112901,47008.372879490926,73053.50384661683,168086.3943975852\n112928,7031.3599835821715,16216.983555917532,31991.190051671518\n112940,112166.36501448113,210421.2414082404,447108.363530433\n112947,59313.38931012596,71576.1772997279,187355.1079720317\n112954,241446.909406415,511825.08797703614,1065229.410330006\n112955,20028.519354272023,78035.34709142482,147746.8444682298\n112958,1182392.4542561474,3976048.061888167,7642586.853905294\n112967,15411.437036872747,36934.158087390286,74300.08218062662\n112988,19259.912895648562,39013.79564724616,82138.79481187338\n113001,47714.51987952168,78545.66209800098,169348.36185945134\n113013,94564.78042397655,189177.96980543397,407341.63024201733\n113015,18256.120294194272,34611.948651025756,75592.53311544436\n113021,20908.758050007826,55287.89954196506,98894.0054734747\n113029,327736.65267346863,433413.3028546866,1084097.8957481324\n113032,10975.269452813305,36099.640117463256,65333.20781509167\n113063,1152552.5338178014,3177673.4775392385,6230696.016158069\n113073,1234890.5425897406,2878495.8805879736,5940427.875315274\n113078,15527.597266010085,36440.79621276029,73066.87736364579\n113092,1365691.4620961086,7698097.972680621,12811429.249182196\n113103,24209.859352456548,49232.62494932343,102192.8037991659\n113104,95640.74058191448,143463.03452767545,327131.5150251058\n113120,910915.7975054459,2612567.014641752,5162779.648097359\n113123,22763.947007220017,47533.938017632805,98343.6587037317\n113133,37650.89887470613,132059.2745597106,241142.86432036938\n113139,53963.77912519317,88316.63362429281,198027.22481930698\n113150,24626.396759649204,54533.4327089037,110589.20599307804\n113152,34758.14591979751,70172.77052126179,143615.771905965\n113159,8902.869118625866,30330.949712032212,54730.865101686955\n113180,1769156.356058764,3549048.615908933,7525742.893284405\n113204,58889.168359610114,145476.95470890196,278112.4099643131\n113229,71999.86422189657,140879.7947294617,298675.89064384275\n113235,143504.29028953047,399279.2626357004,785984.5115182763\n113248,36141.59729663083,70684.22600654181,145698.05373701887\n113278,71684.48416551671,114372.38132989382,255287.13699944393\n113300,250002.3495535182,257641.55765397372,708099.2438079225\n113302,13267.531230015578,46540.69968947608,82767.87399455905\n113325,40934.15219905191,97030.17395398716,186934.1874600735\n113328,66754.61164862258,157464.0111869535,316582.7365230017\n113340,30775.969145587795,109278.00054648049,191821.03796443064\n113364,541184.5926600297,1186502.1457374145,2396837.975507085\n113392,97239.48044710864,225886.81429290213,448821.71247247735\n113404,855939.3571822259,1138097.9248832962,2852955.2585536777\n113417,167511.4088781392,218971.1083190468,532338.8941830462\n113419,60182.19957314455,59268.04365918716,173409.0310777181\n113423,43426.47148077114,97462.3234618948,194567.8238444156\n113430,309004.8487623828,613830.5809828228,1308423.5385225448\n113442,65577.65647064839,182019.14450453888,339292.24501777766\n113463,24906.655072438585,64301.067727200774,122788.85731464151\n113464,3170461.836940632,5063389.352498017,11752876.290431112\n113466,77993.86101783291,144555.35754976916,309656.4115484103\n113480,22408.840680806188,49417.729301370615,99426.00247939109\n113497,117536.08019329993,400768.6037125032,735322.523540128\n113503,7719.029975780027,26729.41442541965,47677.507545267465\n113507,34507.777697182,68776.13735311961,144543.01067980894\n113521,388197.9574259587,918378.3937628889,1859986.8973656711\n113524,641285.7597288662,1646569.25544649,3373897.940408975\n113532,45358.069721922875,44110.70413116533,127544.62929426091\n113536,8868.607995055683,36118.933932741194,63888.49964285784\n113556,23990.934943423068,96835.63624882666,169537.61426253317\n113571,12353.151366674347,29990.33479570481,57902.30037083778\n113581,3262319.436696916,3447241.344196444,9306419.99425135\n113593,200215.8123761746,350687.14452960243,746342.0975995869\n113595,70043.3749418223,114092.10444472608,262673.28524750937\n113601,1170540.8431648186,2844749.9555417066,5991646.475012867\n113602,28042.65833684377,44121.45179905679,98753.77290964831\n113611,15879.17173341325,39506.34830344063,77567.02366353164\n113613,2065467.4063543743,4740310.15842399,9662451.207545284\n113632,47328.633776226765,75047.59839597617,170737.0261246871\n113636,30913.156682656165,115885.40439516603,201845.0059126083\n113646,40708.169689418384,62265.53198459036,142745.9030368167\n113652,6208.874501109413,25417.38158331932,44801.72477587125\n113653,7457.682961128948,12135.51305751549,27421.993908351862\n113660,87959.19420893595,149142.13405386516,329912.8759646117\n113665,41950.7790199502,88785.02036744067,179828.99524476397\n113675,117300.50918633415,213546.044544032,459908.0263507706\n113678,17187.665622025594,32991.961335125394,68678.9412124587\n113687,202865.688499409,518997.4836589845,1027403.7576585091\n113711,3887416.519570188,4756434.793040834,12075770.56456612\n113718,49986.34457827348,91871.73140090499,197649.73534567474\n113728,12162.246398226027,34816.86776189784,65671.66504207026\n113738,671635.6052886498,1768211.6187560898,3280569.5661604516\n113774,14986.914045257232,21055.77335107928,51616.07766208256\n113776,2296142.060568533,2876995.784099602,7300310.728111315\n113806,4579.136503323682,20592.978409884276,35859.68263258144\n113815,41077.11042280205,120383.50484516764,224982.99719135422\n113822,352235.3551735311,391076.6363189328,1038784.0449141392\n113858,527466.5903488669,1081467.5540878777,2275785.041408584\n113883,580797.1984588414,1641531.5488630733,3139626.960899942\n113887,35338.337426829705,72803.52950000757,152628.4858073564\n113888,29489.536911534913,134401.29695515573,232893.9619868069\n113909,33198.7645109693,79857.90533418318,157070.5762238006\n113931,8996.462516490106,21196.870649594548,41501.1049687046\n113942,511178.7079080558,1398606.8625522503,2744256.4362643147\n113955,62248.2364250463,162796.87134571522,318437.4451338439\n113960,258108.27818088682,390845.0821167001,929985.633466564\n113965,439921.1486114927,1039237.6777495985,2113430.317217298\n113967,5013.127754905106,27205.597185750146,45350.983955053\n113969,125090.07584809199,155098.62363429175,401526.3890571522\n113970,10598.979844784726,31400.209617715147,57898.11195372343\n113973,211825.9993939437,548111.5702065866,1063619.520466626\n114003,58955.240875230025,161029.02139184033,297261.2735937282\n114007,16428.065387759005,96853.47281073117,158575.27396492253\n114033,18589.355639229587,50235.30432468235,97308.68971956693\n114068,424689.7947217885,855963.7518347078,1817719.823882029\n114073,54067.97629863947,41716.179700234105,135909.22398076148\n114082,37659.1625467223,76712.50876092518,161432.24078124666\n114083,36572.33863221118,63100.44680899638,140187.17767009424\n114085,655451.3942534241,1274958.0967425746,2727894.912751532\n114087,103455.61722926889,234358.3757714745,473649.8487822521\n114092,27725.286977889587,57626.12980767349,127848.90585171126\n114113,18772.15901727437,54271.71978358635,101302.61140639153\n114120,38865.25001221386,91107.80097448215,186778.51480312741\n114130,122970.06736857614,258943.92902263542,519111.2345090627\n114131,41409.218886108545,141357.62124769078,250008.34217802214\n114134,9663.772388101017,35128.81750454614,63750.99833950703\n114138,27215.830923486392,61258.96106897447,123667.1448887613\n114149,364247.740784788,793361.5409054945,1596244.7475780773\n114158,156150.90163481617,287431.3143135069,634088.2384994689\n114159,214726.5704329236,570670.5554231651,1098673.8024830143\n114190,78869.84614122934,149333.01850014526,314017.99600721744\n114205,16429.640709519852,31938.188343924496,68060.10495889935\n114216,39984.86980627051,115386.27759315087,220957.96831187553\n114218,10941.819223366505,32172.940304870514,61348.88506084496\n114221,331980.9788898376,566637.2419590021,1292877.693060437\n114246,262153.9392901698,462432.12847326865,1032537.1393012332\n114256,63650.36583880453,96582.3458678381,225078.265032675\n114270,288968.5644870249,637684.0854922008,1302033.8968858188\n114277,40652.3960458404,73151.36585227707,157795.75684228796\n114283,32986.064519680614,59805.9917936823,124587.35022360351\n114286,14882.084675915165,62680.38521676696,108586.3603506005\n114288,645345.663191946,634402.7890829772,1763369.3009714682\n114301,83185.5949964008,175943.01633245315,358817.31803121825\n114306,239725.57475353772,809382.417102372,1542720.2997929263\n114310,10897.435706331387,33259.51801305185,62445.85491990867\n114333,29945.78308746388,54815.853700451065,117659.66216878075\n114341,749320.7050699176,1988527.1751212953,3981971.4584421893\n114353,39787.51343946377,96232.8413899655,188348.94654872426\n114367,76890.37182854707,195704.89226009187,390157.6076878134\n114389,417253.7550788466,1726086.8459438107,3104100.094893324\n114390,357509.14552780613,1546915.2210387336,2741426.060534397\n114398,766125.9747557058,810844.8812921355,2167115.615702707\n114415,47575.75839035692,78318.249197169,172559.10601992902\n114420,30036.16419370774,119784.93612280439,202974.8427568468\n114425,37188.05166189091,146671.21158722916,254963.60217217475\n114434,52989.34165699008,248638.28102062416,409194.8180903945\n114444,109634.79309874098,327300.01532573526,656713.3347781247\n114449,13501.6392166789,37167.955803085,70227.69750849993\n114466,49122.977344894556,109939.23289487373,224787.53100933327\n114467,35251.73484382628,67102.41955199534,142858.40467337528\n114468,46656.72918374311,50444.73176435935,140218.18532652612\n114469,15719.53681973057,29793.589101502508,63373.333357012205\n114473,19952.691179214627,29175.377818707268,66961.0527879301\n114475,34729.07089532851,76337.92773713832,152310.99277089874\n114476,3493962.615619672,3841152.423295976,10815835.842248203\n114493,28537.96926804202,43858.89163438149,99871.37301073452\n114494,49314.2318029429,63486.16753195109,158706.20624131127\n114499,174636.12724953098,235280.17658621108,579436.5616606018\n114507,20762.015134999987,100942.5699180147,173452.1058916544\n114517,10926.976902571698,43130.52546287937,75459.66482515115\n114523,105430.6823999882,175701.91984500233,406402.4608333007\n114533,31759.851364971993,86020.27082720592,162241.12421724334\n114544,84873.1435730543,143543.45617103254,322268.8980294648\n114554,386528.2810666642,1184899.3523565251,2212510.6730941418\n114564,38163.414734967955,98774.68863887458,193497.30551280116\n114565,3005287.542208833,3097840.1610493795,8875097.49013644\n114567,20789.395793130607,90574.78992133295,158527.49166381813\n114603,21504.752445715763,52336.63462184784,100511.12160139489\n114604,105573.50716505933,240776.12658084664,493370.4847076058\n114606,129007.88442977682,277185.13898566435,591942.6254566393\n114612,93388.37225860858,263180.6521755785,511054.94329207344\n114623,353670.5652153279,752711.1653927052,1582350.179194384\n114629,2582547.698733086,4328083.75831336,9974208.737731494\n114630,71726.60776532641,182245.24595872467,362647.78584169963\n114635,137454.92321105002,234534.58696444274,498876.4906841876\n114639,289826.72400327335,899511.0831301628,1670345.3870404898\n114641,88683.89541267385,113802.62308518999,285607.56748487044\n114660,15254.187378177814,66812.86470797013,114077.87435750038\n114665,10924.890725409974,17497.29468853874,39825.0157944892\n114680,44495.95501060537,78820.86024257937,172115.06985917143\n114681,59992.96428338195,103435.89529510005,227714.26327706256\n114717,29878.242300268077,51940.485388468114,113650.47276112202\n114724,3444257.207666212,3551849.9300913205,10218196.376350682\n114726,102717.21827187235,225166.89346749766,455078.91387209145\n114730,255779.53956544882,238181.26755664684,693670.1424014193\n114739,343866.1368334677,1031355.1278534827,1974082.4109826363\n114745,39404.31586969948,65373.05037485104,144757.71192853045\n114747,347475.2164494998,641270.710278328,1370955.1063040642\n114761,53730.12000413841,264575.87286876363,422062.4735727721\n114780,53967.35247784642,170415.13402772864,318027.178986641\n114787,470924.7691920668,724294.1521755575,1682891.3577305684\n114809,16814.50125265199,29950.079937903865,64557.819447309026\n114810,13861.347824727642,43760.1656038877,83049.40958337454\n114821,345491.79095599963,463833.1116362799,1108011.350665915\n114824,11470.208497742333,37667.29616068836,70345.25123582233\n114832,76359.91402703007,163539.54628875075,332728.8191601919\n114842,264321.15344583883,700351.6260950334,1400521.7962587026\n114848,293529.25082625536,407857.75989517337,969361.7555964559\n114859,23024.928049884355,32066.1437221302,76694.63238942833\n114875,29752.570515932257,94628.2914168743,179185.23210001743\n114883,81710.13119356036,202600.07973472154,389444.06865774246\n114887,16839.96112248694,65027.67713935425,112956.25711901311\n114926,102215.71884700512,247730.5017016019,495647.83875321766\n114927,44044.757474708815,59999.22990415297,144260.7736456383\n114947,243444.4559692494,535999.2952580925,1094869.32784124\n114967,22940.15225714591,43783.91425634264,92566.6952465864\n114988,3028059.06522358,3724057.3874897356,9716768.574865388\n114996,449601.80726625916,555117.8699754621,1443939.2998995187\n114998,79691.07252908024,119981.8802776584,286310.191124575\n115030,19040.784194444153,63505.89822086827,116789.84017432536\n115045,60335.84789380208,133921.46765786334,271712.03374460764\n115048,72083.48789399884,238760.28546518218,436689.4999011357\n115055,24148.893046913607,36369.52701789313,84016.48220350119\n115067,43849.32463663303,82207.29338754126,173375.73890350232\n115079,32450.261949992073,97114.3052081489,182505.45025979122\n115083,77954.65101269646,407713.72724731057,708045.1579752164\n115107,35478.70450193004,95323.25600537739,183646.95807066723\n115120,6219.781461942532,22416.711324987125,39923.31967559545\n115125,25077.18752720122,52480.013038688696,105190.26637696163\n115143,375599.93104720145,1007269.1966718455,1993780.3955527022\n115156,1242612.0980417228,3186779.0994119053,6456899.055040679\n115163,37728.94101561148,74914.6609005184,156945.96426600503\n115164,444089.225258564,942263.5991214123,1937841.1809364418\n115180,484508.5758901382,1160236.1415210844,2455188.5259896116\n115185,24345.821919242473,63680.53974527797,121595.25064059848\n115188,79749.69566100632,162968.76796106598,341721.28815933387\n115221,36532.099970235155,50593.274434336156,121039.437922392\n115240,30496.093301088622,67623.08053461525,135280.4625862004\n115265,15042.08074719214,40356.358543628674,77506.68223448592\n115309,4038.9111190415815,23281.532854383135,38277.497809923334\n115311,15481.487949725208,48205.94781810621,88263.93373769891\n115335,120570.80269727045,118476.56671075783,344310.5100878075\n115336,53716.33204235331,189732.6517659393,337444.87322011136\n115343,27882.606889303293,75131.37250018703,141625.8587245514\n115363,250229.8471925134,578961.253694123,1151511.9110307922\n115377,108533.1659552162,158482.60572038125,371771.968895843\n115392,23081.945294256682,90987.22587622449,158515.0728317401\n115394,83556.87081347014,108916.68633741216,274141.3135964881\n115409,15704.563255343708,38268.666845082225,73782.03259375227\n115412,38367.98227170951,65467.12632879253,143213.12371443352\n115424,32255.490446392218,45063.3297600498,104683.1761188473\n115432,425041.66791047604,989866.9122583006,1987160.995585341\n115434,111031.28718114467,136838.68602224003,354045.8780492026\n115453,37894.64802551324,94990.10147116723,185117.3139580593\n115468,187939.18754669194,246544.9142910233,619840.7530490265\n115482,52990.24258518645,116026.29913201375,251718.51058492254\n115505,35119.46147396441,58765.46375315684,132548.8135582693\n115512,43893.83692695324,99190.00410636378,211864.20451269235\n115513,27217.150987299563,58161.80384534891,119411.91612262292\n115514,37895.80872014427,70418.08342239322,153186.19590087165\n115535,285872.4188350542,793044.1576450786,1575960.0941722533\n115541,66469.74399191882,148744.6426175449,292203.24675431964\n115545,52463.04834924408,136732.97306935428,268719.6716545045\n115548,783350.4049110487,2205929.7226760406,4323116.934361301\n115549,17075.120942198606,50401.23642924349,97239.1080744873\n115562,163622.7026029438,181335.3592586125,479894.89384665084\n115567,47933.94717904216,48715.049902990024,136573.97266956407\n115593,48135.5736355219,62922.962306526075,155498.28360584684\n115603,31708.623656304102,91862.96407534703,172606.25990236906\n115629,316140.7133425535,702222.5951812064,1453272.8230357193\n115659,28068.413115968982,95254.40976867192,174009.86557067442\n115662,26684.093266025036,70930.22109122078,138320.9125049227\n115675,1568993.7590255933,3333130.6958145783,6975231.106269034\n115738,3105705.176303411,4368603.91639195,10368836.80838648\n115739,39082.417720037854,64143.46941577526,141484.77922541523\n115757,13632.029425924138,45660.38918856103,85925.66902041389\n115762,69538.19916891347,108693.1740907679,250012.7475290831\n115806,52505.77907454699,102077.91809392262,215873.45265200455\n115827,25206.63611453284,62354.30634115176,123012.14751383296\n115834,679242.6401107885,1368617.5791114855,2883406.4079259373\n115863,57629.74576776024,107094.49764747168,232255.93565753807\n115888,40237.85483108881,112738.77334892256,211269.58021241188\n115895,502279.4784074278,1864235.9915082797,3289298.1284484062\n115931,39453.6193846622,55192.23046063478,132415.45195379358\n115934,14387.842257403614,29821.53841295086,60080.2646844315\n115952,15212.88085230243,57192.518615371984,102076.48058415588\n115958,492211.1927664596,1023878.4344155493,2173752.837025146\n115965,323067.99620558636,736042.1450964537,1445768.8022944315\n115976,38478.94129356891,78329.32332672637,163079.68385396246\n115979,106429.26787130619,267322.4629340573,530101.2049291383\n115983,378367.4310513531,1037839.4464688668,1982915.8339604863\n115984,18209.702194781526,29377.807915064823,67536.09712461388\n115986,20781.44032066496,94416.22315854092,159082.0641080279\n115995,293874.91376645997,669917.0990416443,1348359.5268520399\n116023,54041.0218717128,80092.92873447259,187911.24310496694\n116024,37603.00401216632,85718.54756913733,170055.19403594575\n116029,3291.7690046026482,13911.41705366147,23382.40274855306\n116039,3722.8447735487507,14241.694034293865,25106.569457225065\n116045,29140.330161963917,129192.39000041195,217552.05994730222\n116050,55628.72876154454,55326.42043639839,152474.86293963797\n116053,73685.97498512334,181151.02544137093,362402.256091296\n116060,43671.90930777114,73011.61648776781,165991.99747319258\n116071,36363.8070532489,61532.11065608061,133234.17178255174\n116082,35479.18267652636,77953.79368350572,160234.40027186513\n116095,28594.65423154426,100135.50766501234,184156.6390190187\n116133,31784.704792948545,76461.78914026672,154527.33501318237\n116138,62136.73570140362,185300.68059206556,342767.13626054005\n116142,17069.567042706436,38391.90502939453,80005.82637226405\n116145,428140.2812776008,968197.3625843767,2070693.7203639776\n116155,114649.2171428736,164835.63291413418,392709.4291374461\n116159,416856.9372453639,460578.97456471995,1223705.1333268748\n116183,770764.9573081353,1902741.6669166286,3942846.6510222405\n116213,252629.08914926442,806860.5458317719,1517067.0960292644\n116215,48194.48789554212,85923.22742543467,187863.04295020428\n116221,367953.8409078946,377270.228785595,1037729.7889314143\n116225,235358.1100051695,455113.1307109588,998935.0373841221\n116246,24127.913560181467,77038.80725239946,144190.58659235574\n116254,58683.14444746128,75447.67827654704,185694.95803665143\n116260,18131.624496357115,49098.09305222943,95098.86339466072\n116279,16508.593962732037,29711.666109663438,63790.42803778837\n116295,39249.79689110782,125479.78769022331,235910.322992381\n116296,64056.243496647076,130976.09482096128,266100.2580326644\n116314,6863.458691195537,27739.196605286237,49732.91733344768\n116315,145403.14117903454,338195.82763000595,662724.5458428866\n116317,965387.8317525797,2259042.3289862936,4541458.120207125\n116322,12170.974748760875,27248.133390115916,54847.95386192309\n116323,11735.294633372863,42935.404855211746,76758.40091120653\n116328,84537.40098121409,211017.82114319105,422202.92131716723\n116339,39648.08957427163,61605.90970249211,143967.67630636203\n116346,2050727.2238817844,3495598.8273093184,8476624.092751203\n116396,1570898.8546728762,3669632.6561698103,7733008.159126512\n116400,11871.692007562924,28940.6689168875,58423.48530906585\n116415,108377.9309455957,148003.9933205083,344448.3118355859\n116420,4707.775278550557,22312.14415050905,38056.66836117946\n116425,37525.985377198245,72141.75582361667,151002.64016641912\n116445,92158.66288386828,191938.5788882705,388744.785062863\n116458,123129.46609317327,302434.7331942404,615708.1006278337\n116460,22202.830603314327,104707.62882471125,180161.55613726142\n116469,19060.907889590955,83620.31501155766,146378.41054257902\n116471,29188.098445717835,84073.00275785702,158109.93512199342\n116478,883836.0468667867,833575.8466305902,2458530.4197783424\n116485,355447.5300521577,774278.972673272,1607834.711311391\n116496,31861.86097714286,55716.790580987436,119988.4324354151\n116497,43501.169645951035,78685.64008818123,169676.8280065081\n116503,8582.609738432655,71589.32425749692,113854.7946358723\n116504,10818.823693057288,64735.12780797626,105994.6595252981\n116515,10262.35710903959,31264.85106009484,57732.260504754086\n116539,28769.54329937068,127357.61388154766,213324.94841090543\n116540,3095253.48782974,3530714.1267966,9755020.337739877\n116543,27054.932100091322,62569.45186517456,123857.05904671336\n116544,53391.54791009656,122377.66010535865,243738.2379428266\n116561,38917.81787731192,58871.863080691684,134102.4507633133\n116583,10859.254057792747,40867.77386697314,70564.29127205836\n116605,41682.697797589244,71266.98443801595,161765.8565252052\n116615,128064.00014461571,243015.4383178266,517487.93385481794\n116629,40904.00765404888,116095.08261254724,214925.27646465146\n116649,19448.791187727147,22119.553928106863,53786.283369216115\n116655,352585.08984768856,397329.8229954773,1040256.127128731\n116658,97186.86396832445,157120.8985782975,349024.2308764827\n116663,12073.392294090118,25868.529309995425,52170.49174345596\n116679,9874.554683305756,37525.69262796944,69138.31678963616\n116683,164071.8631058241,272686.0813308624,606425.3734210874\n116687,4430.207438209474,18142.988038163036,32415.86869103273\n116694,12121.640119927533,46680.436020513036,82824.28176260214\n116724,14058.018382735376,43132.13545136942,77503.51425256461\n116748,446322.7525617149,448510.56659087195,1231117.210487205\n116750,33834.9063444742,158550.23836034007,271299.9232790347\n116772,106618.77908426497,129077.90193468911,327528.9218492983\n116777,510597.7378630341,1764137.6742197024,3404816.662018564\n116779,14698.937428231015,52851.30685875353,90131.66949666454\n116787,33086.40089183318,48913.03048151301,117649.59139161128\n116791,590258.822303472,940905.8296752227,2190927.671559562\n116795,132235.38697055329,434318.91212116415,812717.7567507646\n116818,318693.09945602255,593092.2128524566,1286086.5348658052\n116819,35129.44946668679,61677.67334875237,138173.06901593663\n116822,670969.7730583221,1779233.174600565,3508217.01642977\n116829,17318.8049066899,25097.51203703379,57812.020575540875\n116833,6223.716220126347,15243.768673756695,29856.158065777938\n116839,17122.014874623368,40117.50624345156,79559.34081578741\n116852,24839.24237982584,58448.85128731359,118476.72884107527\n116859,209291.01376820842,934510.064461999,1679890.8943574529\n116889,66940.90277353127,121889.5580764691,266760.4718481568\n116897,388707.46637194796,924233.2701573279,1889594.851383616\n116934,79553.60793525675,156448.94223481786,337756.20761691383\n116954,59020.17959260583,284020.8991269446,485875.28063393896\n116956,60174.385286033765,104618.31541290303,228479.14344194866\n116964,55608.77679775217,84344.49620637631,194047.66974549124\n116971,34793.35472102881,60506.235801365314,130831.40747519484\n116979,11777.31759403369,54405.842929464925,91455.0462852334\n116981,60549.50398602005,112366.9107963853,239153.84066581755\n116984,271219.12173825584,455931.60754882125,1009780.3284019771\n116986,5171.623317197125,8148.042768961046,17709.170070347594\n116992,74986.7071588967,189909.4328474711,355430.14597458916\n117001,44854.99776889046,70859.85918706722,160680.15751939637\n117002,443665.9911955773,1053422.572479336,2135269.939899598\n117015,104694.48374416957,147596.23086482394,357237.3147466306\n117032,58666.149875999785,221062.7546577375,395486.76836916944\n117038,110779.42764962783,242791.87579491243,491878.1161161815\n117049,76911.50343570368,264770.6377349372,480752.81140496966\n117082,16100.568667835145,53343.16271175121,94491.02082878808\n117106,185215.75552019602,355772.7064474518,763645.4827600489\n117107,1736912.7313835612,2029496.4819990753,5348371.295166626\n117120,89137.96256044722,104899.89195039764,273394.42952482525\n117121,365973.2496739139,694061.2572720222,1525308.0333944117\n117133,29803.524484309437,71842.57287624828,137706.8730618843\n117139,126781.8898176475,444370.4921202184,829692.5752151457\n117152,21433.066126869388,56568.652235455724,108966.27299657962\n117160,76186.4633797337,125279.50071662263,273755.1756086529\n117162,75020.98822328202,133259.50089599704,284193.8288648797\n117165,30387.459332615235,79128.34576218882,150441.94247366043\n117169,98938.3951626843,283065.7199380915,544498.0824795107\n117173,11273.565145058408,51388.03221111995,88213.1684572451\n117177,232538.46661971658,495285.2912755972,1080777.5112346914\n117180,68110.987724869,119773.03217950574,267863.18537858233\n117218,1696000.4022639082,5144079.38574241,9749378.278650515\n117229,41943.84458559717,100211.04296007445,194967.12404501054\n117230,262644.0121355028,551528.1672894829,1170564.3983383803\n117256,38322.169799763884,73050.78695285427,156987.06137942517\n117266,2882.21203707536,15265.604774692285,25555.732964951603\n117273,313654.703255574,752040.2052551426,1483015.2884071027\n117274,4853384.7732268935,5761354.77727558,15365716.364577249\n117280,30210.124495848315,95099.84419042205,177419.82081175383\n117287,148492.80500616843,435126.0826222068,829860.737047083\n117310,169920.40241110467,517495.73407066625,973887.7717887915\n117332,399225.8328745774,889590.1952697025,1810001.4924533723\n117357,22915.68887825649,49321.05716951146,99275.27668532298\n117373,11708.465550381017,22825.19216915753,47702.01868219522\n117379,436473.1898028604,748288.3634910473,1682191.7876417253\n117384,95087.10359094322,89548.45700495713,260073.90005842404\n117413,4995.346083310625,23331.317590050283,40750.92506630502\n117418,244981.24240824874,287537.53906896594,737445.236483708\n117429,12881.224073877494,18473.543509882496,43546.044406896566\n117431,429934.6262301503,529191.916354316,1325183.8571913086\n117436,81580.81826462265,164028.8788774144,342670.05485855194\n117466,236409.9493985797,538049.5862703361,1094220.3824477862\n117467,580710.1432162867,1357448.9845894428,2694671.952589077\n117473,89068.12974367409,88827.92846923908,258762.37112286867\n117477,363533.97383919556,653898.1365380927,1453454.5022664892\n117490,20156.315593055006,37685.745147950096,80807.83962977\n117504,800911.7658935612,1966867.8194112196,4039079.30820382\n117516,58013.3177657294,255440.41466495843,449490.0153081092\n117521,41345.12384090319,142200.2224063985,261064.78827961427\n117546,163938.02458102859,490402.3467529736,919900.8987997189\n117554,48517.61757264195,159811.07993908483,309634.8739796326\n117573,12154.293368785568,32341.493609912202,61737.85231930211\n117577,42922.62275191272,102875.08132426105,204988.71589054097\n117578,285355.16038231004,867588.470042805,1653762.1399154738\n117583,17876.095515814628,58976.63249869335,107810.06513064173\n117586,37958.13033879376,85424.2814013058,168205.14661796682\n117589,18585.35678214513,55059.29401981546,102837.73300327036\n117592,20351.657863293192,30505.25210154354,70097.22624757326\n117597,439956.23338198185,768205.386083217,1780338.5666163615\n117605,219508.5157720938,470867.1549014672,966692.9061572503\n117623,38216.45134960121,142319.4084577111,259070.63582213057\n117640,1425989.9957638604,4300600.316256196,8253709.058079285\n117657,1371495.0473258952,3023736.6206714837,6317583.134163052\n117661,10870.619170547374,60932.42479384507,99701.85879521981\n117678,18441.418537374655,63136.18981903298,113371.87993218206\n117679,397762.97837151034,557122.6755702489,1320103.6218367307\n117723,75135.01569435754,72567.52241891973,213035.49686817115\n117742,586282.813795793,1362803.0795676135,2824595.3327109087\n117749,140069.4539071428,449098.0525789808,849832.4712275486\n117750,12862.958841094698,53427.092973836654,92751.16289932482\n117751,19342.18737186315,84884.29303844772,145266.1565703479\n117775,56219.412958828114,93872.22878907245,210687.07627494945\n117777,677415.4964568692,1439888.171380253,2963700.02684121\n117783,267938.1702652004,834008.9474991813,1561565.4682642263\n117793,21404.44708724475,41239.43050599685,86105.76395157329\n117796,30472.812082281118,87585.79058392016,160605.52660644328\n117803,47228.47651554851,118391.47027527817,221632.8446839715\n117815,27552.053797741944,160657.6555065257,260905.48479766506\n117837,96747.70477615381,107587.22374189722,297886.6719248288\n117858,56681.640262237015,105133.34839212781,224880.44635631845\n117870,161761.30325061668,505407.14088883204,965423.8873744224\n117881,7551.533082302644,24966.924716541314,46521.19012056131\n117890,9948.513629963709,29604.28241383054,54124.317132959026\n117922,66282.05478101327,175526.35201043272,332025.38535995653\n117923,73330.61787650193,263287.58827281883,462037.53391116986\n117927,67335.46771664143,98360.8011062968,233382.48071660488\n117940,16857.823399337107,68130.22236448222,118539.42835816598\n117950,6974.387344477821,30425.758073283887,52605.798617119144\n117967,271231.83189390437,641642.9957344395,1292180.7380973762\n117982,5006284.082231659,5545188.508781849,15548128.282174602\n117992,161170.3371383797,369798.55755390076,741469.0276157964\n117993,14200.288950174547,86517.99273065866,141383.2930743175\n117998,41748.026168806755,128275.96771651834,252348.5639541351\n118008,392585.17779567285,309747.68825004285,936564.2952037529\n118010,281803.7715061732,526001.264879802,1121609.7961090554\n118023,27987.853626758966,121134.76626129904,206291.06421464897\n118028,276553.7278961778,937796.4120157377,1733597.0182947498\n118033,44184.8408761242,107894.46169673586,212791.82614828623\n118034,27446.892151062133,48006.86651174356,106441.56749358294\n118039,14771.045525614962,26725.33044509841,57385.162030459935\n118040,72708.95782731057,146276.4668271805,302067.9075302768\n118046,43120.21735966017,84822.68467377541,175238.32261793833\n118067,56153.57024537039,142008.5005077422,279831.7251264164\n118085,18560.633438533452,48484.327035602306,91004.36048638247\n118104,29151.464603895885,69442.13914878838,136672.57202663523\n118125,805863.3137711863,664926.3840360285,2095812.5608624152\n118127,13040.25155673423,50545.77626593649,87673.45874503386\n118129,32175.601080471573,75355.18404608566,149416.60104126867\n118146,62651.65697629009,77170.36948235832,196463.80737656797\n118156,19567.04469180988,39819.83758117648,81777.69400824267\n118158,25841.96878943311,62850.83761426964,124794.77899401015\n118163,45870.64045632698,73483.00956223598,166075.09681877494\n118175,39838.71791626817,102288.03711200548,204867.2431737536\n118220,152351.5229827428,264969.76067208336,570399.6128240363\n118247,496158.3801261639,622931.740005816,1626682.1858435175\n118256,23190.23094085263,69743.31902996374,128814.82467826353\n118261,101409.60464489584,222401.80588559745,468701.1969089339\n118274,15907.058841302078,49117.967335602334,92359.57773608767\n118282,195460.8391666715,522779.8381159753,1029236.4016394857\n118293,109482.66346609924,123661.03134028357,319460.14436956815\n118307,347201.2194628429,956478.7166595679,1864980.8365874756\n118329,1503686.8876232735,2558877.5000783186,5768016.326334888\n118337,571489.2300695239,2242008.3949313117,3930709.5597648164\n118340,172815.5043432494,188769.15277444865,507793.9250269141\n118345,66181.23886017223,255721.511968315,446487.8510052461\n118346,28234.847006337503,100763.42253942453,174511.77355156335\n118353,15899.860175865728,29100.78300460779,61633.80732414109\n118354,7042.50308697354,25858.04480748868,46458.859500104525\n118356,90339.8765320416,154322.44545892437,339367.98441406386\n118379,29317.809612648496,36241.060982897696,92307.94761973873\n118380,757812.4906955119,1019139.576359531,2531339.754808139\n118381,27360.01094569765,40173.360062364125,96311.58517122692\n118421,7512.9346499929725,35795.77923660132,60201.88436879067\n118434,59251.51882458163,243615.55685654507,471390.4544762042\n118449,1788965.2054079534,5023531.608705243,9863616.27865926\n118454,60267.73886984578,127563.49228677627,256676.52392278647\n118459,63762.51193195763,98781.44572794672,225543.61795332836\n118464,927948.3011513151,2362434.207933617,4638600.755709934\n118472,9912.288397366556,23980.615792403267,46270.09947823512\n118482,60901.78634587283,156688.52046194984,320442.4153026387\n118484,1063058.3502620563,3916979.757806301,7098353.767228568\n118485,39046.16831204058,88863.68385244168,176300.00204568915\n118514,608753.9083700327,1413226.3192831264,2861527.1962763458\n118517,87483.52277148006,191528.1363516509,388540.26036133035\n118525,37828.03076662539,86886.1955503889,174733.97611935547\n118526,3031594.2980380454,3970553.426734645,10005861.370692682\n118533,10357.802184065908,19312.00205716737,42184.94990141653\n118554,32335.022094689753,114494.51353474482,205291.20034189348\n118576,90901.57173256538,118233.40012854003,302940.9905455819\n118579,94794.07149314471,170876.97158878247,369733.1976234688\n118591,143338.7016348781,208794.40072075513,503935.78511535865\n118594,42742.27466135858,105364.3235145258,205174.68628997117\n118596,52228.40990258647,93157.4747643562,200151.29584783793\n118604,18017.122301639298,72972.6542819641,121301.48978653864\n118610,108974.98290331791,230625.79157228384,483302.8073292209\n118619,1144891.712204565,3256240.833990283,6744903.350162538\n118636,648698.7495324628,1017814.8166446739,2368105.8085631654\n118642,6908.406913798848,26404.692420973908,47695.45549943327\n118648,85211.252030183,130211.2429095481,294559.5597355692\n118653,52704.62348734502,193780.20651717426,372235.5824452322\n118657,23714.333598507343,47450.376965807896,98799.65045945937\n118661,40103.89176361155,100417.40539552543,201531.8610958457\n118666,34676.10232335244,72909.69578416168,148786.1656565215\n118676,1228165.2178207058,1083495.5433610987,3256528.9416514765\n118698,14270.02422418485,16943.70397398627,42945.32022419589\n118709,158848.1030523681,333198.8783627741,672457.2881954534\n118718,5215.11018512159,29511.940891723803,49886.84121293235\n118727,9726.525425431286,23839.104243534173,45809.28719486023\n118733,5570.295339637193,11367.214764887483,23493.114348399606\n118736,712394.3367366878,1315731.4243346553,3026388.870400222\n118739,332060.58023770165,923683.7152767262,1802215.7445220305\n118740,10239.101599072961,23724.04104100477,47312.442008166\n118770,61459.23910722583,176019.4549899948,322894.4924864864\n118818,8242100.543649058,8829594.477438148,24076595.95737516\n118820,14865.124482073506,28315.856282099277,59686.82662777293\n118821,1681454.2585815443,2466525.2638067105,6016921.740652664\n118829,406236.02968110365,852974.1137028985,1739365.0649944227\n118839,6071.548306767515,20697.244268979448,37902.91782281028\n118845,1086699.7736115109,2935655.9649618855,5730302.0456201695\n118846,6976.767418635654,31141.230395868628,53164.15513259257\n118850,15532.557681839111,56546.68260914167,102090.9596060857\n118859,34950.979873334676,54459.78213364413,122569.42708530501\n118865,168422.67411111068,662899.9203549568,1216356.1201102831\n118872,578440.7555808364,1035679.7449182771,2268231.830066246\n118919,41485.362318841646,45732.069799917575,122902.75078961466\n118927,38720.02489491079,97012.22627543742,194530.48649930992\n118931,36620.447876769074,106943.77085578423,203034.73104584333\n118940,63787.70933460406,231353.77910939595,427513.2311801486\n118942,25415.686298020577,84330.18698647857,158426.17066918346\n118975,19073.606955986947,70050.22663961616,124472.96853439433\n118976,69269.28527377716,261039.95679334053,457384.61793123203\n118977,286569.7488286942,672265.331256681,1362307.1204512045\n118981,23634.203238831746,108630.89600466486,188405.12122794575\n119034,24822.423878815094,43971.553995953516,97836.93172164154\n119039,9294.76679862474,14518.25542210734,33761.71776657118\n119048,42663.18824875375,77073.51606714613,163926.55792988912\n119055,53494.45878759924,169789.60555044285,309231.1888900599\n119059,42879.942986047856,88677.84694197783,179087.8486449669\n119060,49732.970015102175,96435.33892019362,199475.85166915716\n119067,39019.49769127023,61110.65622631788,138698.08912133973\n119068,17110.202528468824,40699.31394203821,78611.61646616578\n119078,336273.1607606859,1030903.0095016249,2004330.8792193239\n119105,77367.88915977236,109839.75555723038,266516.86558946164\n119121,18830.701603108708,64118.268281530705,117775.17343866029\n119135,13301.185443059796,54403.84288683167,92916.26111016601\n119137,10014.366041506768,53176.90618327603,88832.43504892144\n119152,41554.675328024,83841.77396953589,178173.78148139897\n119172,139088.41533823454,433142.7169656851,842223.9334755093\n119177,51629.335907158005,103217.657853048,212311.12902982038\n119187,1414206.9147511774,3581405.213775077,7219799.556100465\n119195,14235.820459181048,22838.54992102209,51443.25248223639\n119202,457084.71727260516,462718.3853929506,1309201.1412337148\n119204,107474.2311144864,100077.62186285776,303676.9213505285\n119210,985484.9112422955,2728737.3555630525,5169007.067843007\n119212,70973.9170974022,141623.97214604277,303226.5164050085\n119214,123098.53386327933,367556.1877000671,730647.2580394843\n119226,21693.661469620998,43490.58513285947,90883.02059952787\n119237,22374.12091720363,60191.117892365466,112857.9749083678\n119273,52676.47989069812,90575.14155813759,197612.96296539705\n119275,1845264.1325190791,2500577.9852800504,6227518.938498687\n119298,3157317.2356132767,3450964.1131362515,9732375.168295842\n119305,53744.38874561677,58664.0816823926,160558.4365641821\n119310,14431.432431233281,26738.188015007345,57796.82189595264\n119333,84291.27108766697,271304.3664012891,494825.5731309134\n119353,43419.06116747588,51872.66384212729,134302.53811049453\n119359,91606.35187045537,160480.32260782737,351334.13071868254\n119360,4213.9771293893045,20284.333561244606,34796.376459569554\n119361,307279.43074703787,438600.94168951595,1086532.3450191263\n119395,58888.61757445448,92632.39330305682,214134.1179826397\n119408,46720.79092268051,123797.45385264653,241428.55534570868\n119418,291395.54682400357,938284.6857762533,1730370.2558939233\n119435,698988.5426357188,1524477.2910917571,3161530.8638870106\n119446,25150.814169741843,73636.77070099708,136690.61146563405\n119449,90646.5636026015,214799.71196021663,429808.071467252\n119452,43200.91951622735,80473.35364472707,173666.611750293\n119473,23065.177114393435,38160.157271873475,85365.80491589083\n119474,358974.5170639174,918403.7564843879,1838730.2085679478\n119476,4497.760175108449,16829.95690150579,30482.444021281415\n119477,6960.2495647367095,22653.63359983634,40958.3123146179\n119491,458795.79298320983,1591981.8358742418,2808035.3571961103\n119498,163758.59167253837,428876.6850989738,825683.4058780469\n119505,603705.2479142292,576523.7973993687,1682642.5019746362\n119512,693330.2713795218,1795932.3941945834,3563144.647989953\n119531,27958.65830284809,91117.4369432949,172035.525957215\n119532,135972.9781475961,337995.68901383516,680816.8238984684\n119543,1412846.7486691365,2528174.1930014864,5750055.71335779\n119549,3634970.0491938917,2313264.249133639,8150066.238237847\n119561,366072.7818397998,647281.6005993868,1392279.1521506272\n119569,644678.9562859199,1792768.5208293796,3568248.0354792876\n119584,79976.2985765985,162621.24409223607,341107.09345143137\n119586,873303.3412292475,1580150.3760859368,3545771.3112095743\n119593,13348.980264937181,41623.596044656595,79046.74669849288\n119611,68474.48433818022,125495.87134124832,270001.5873670937\n119614,372404.9744534981,780615.8483253628,1645822.5331085161\n119616,1773520.5901285498,3063144.8505715583,6910456.45258175\n119619,314558.0019703788,413578.7725615556,1024156.3399137283\n119623,158793.67635771743,516180.34355887177,926184.5005723605\n119627,226944.84732726778,541723.4091301021,1080578.2583942316\n119629,333854.43074059946,571366.9162658892,1270695.7517559486\n119630,375597.0303446304,579609.9847685891,1328321.071927453\n119643,691641.0805283712,1844934.493473471,3611928.4470249107\n119650,347027.5933574433,655052.0463380411,1419912.4376055533\n119651,16331.535764000362,52154.11528694555,97445.51113549361\n119655,513615.6834166381,1200161.0682484733,2451353.496249599\n119665,191226.85258929856,279913.4727739228,665533.5022894773\n119672,27266.6447499634,79201.97676401219,158337.31423273223\n119673,221309.56227211325,446028.24079706345,986936.9357724495\n119676,205924.57962340003,634054.4960893767,1242867.6849238402\n119677,6419.147012139234,16881.512796008526,32920.28234782057\n119678,222111.7340152119,451203.31799629156,928229.2146070891\n119682,17009.754556446416,40721.55338735328,80235.34498587438\n119697,411611.0312212292,1607326.2325548925,2819182.773783819\n119709,49281.83482995964,164659.87918825724,301179.2866683117\n119712,30659.705592807702,67129.57360798401,134937.56025543832\n119729,313204.2574904174,491091.0096335559,1135806.5036238758\n119731,24730.140021212184,69142.23136153795,128226.88693350194\n119733,96568.19154059351,325497.6925430901,589088.2052005328\n119748,287005.79661396553,594633.8534307532,1235041.4850446847\n119760,94152.57542559659,228479.9323339843,445191.6737300495\n119763,32417.54369226322,76188.42694970718,150663.57211149653\n119766,12349.598453835959,46258.61735903757,79059.4579671466\n119787,10358.89766533901,65768.16851468945,106019.63566763689\n119789,278680.0137965755,916638.9588338481,1612303.8843192672\n119795,123848.5614177686,136582.74314708175,364581.4989894682\n119811,2192881.0682082805,3509220.2927232236,7840566.006514523\n119813,3096384.4206636595,3023668.829656366,9111127.262139756\n119817,290212.04595571116,900166.8377626546,1678140.4229514194\n119820,190724.04081272578,443363.4891214761,885668.8271289143\n119825,23959.812829701455,52737.15910126784,107940.73758481035\n119835,20283.329237123693,35742.72398191002,79089.00004842327\n119840,21782.408757950077,28052.28196150814,70095.21640249016\n119858,19790.89474711518,27730.75570508562,64282.03136055862\n119876,30014.068274703717,92537.51009096373,165857.15577845054\n119892,67519.01511882036,125128.57553196409,258614.1531847007\n119893,57931.52319537348,151931.5882826479,292604.74341664824\n119919,53788.611481058746,108292.0557608085,231016.77329284872\n119928,34955.247511389476,97043.07865714074,180121.62004458363\n119940,20813.294246536894,77123.75011634802,137733.28015067684\n119963,83925.84075266881,158745.99717071952,336955.53413807915\n119968,32519.433805360597,84072.43364975724,160792.87961156998\n119969,3811.1883230054464,21400.03362568015,35953.57498623378\n119975,27443.32038634672,52795.225918790224,107685.89962614252\n119994,54051.15241082046,98576.23048564991,215374.65348188917\n119997,31883.535753967433,82126.46873612827,156019.01718716003\n120000,31785.104540788532,51536.12085147938,114796.1229406682\n120005,125289.16123995498,369839.3139642939,712916.0111372669\n120009,322665.1741460468,253008.6932456189,828634.5866351423\n120011,34808.222258381014,102042.80237234618,191165.7266062617\n120012,3977048.1382103865,4117596.4985265746,11114620.410301788\n120019,40700.35892331102,57886.79584169908,134030.93264384146\n120051,30712.77905161411,76798.00153468178,151633.42656808335\n120053,615584.5066770063,1461890.5149395866,3033381.392612338\n120054,73027.33007398571,204996.01931970817,382426.4338084245\n120062,43212.705448820896,69066.10593246706,156143.38979694125\n120077,232411.01333496187,427738.14069708576,917144.1894426208\n120078,418491.33361760963,538726.3510137709,1382753.5018424\n120094,89428.95535597451,93624.22782931132,254186.6481303476\n120112,49044.46881748361,117462.88275649426,225915.70466960347\n120116,251208.72462741853,494514.3254182089,1058210.2879731902\n120127,995618.6310892544,2334651.879124587,5046321.087582439\n120134,7300.541959934777,20975.982191683685,40637.33108015875\n120145,273802.3256356762,639647.6157122902,1279249.6041415136\n120162,33410.490133286155,113320.04239227888,202504.09296679532\n120169,114771.09726215359,229183.78130488054,480110.0499632441\n120170,92272.65572876776,92806.60611825211,263151.86104440235\n120175,20075.11542493879,52077.9001422543,101502.6101046777\n120188,16734.397706478605,40476.3030249313,80587.05154380883\n120195,266967.68538958416,550597.8583073446,1165671.4831942334\n120204,69088.29556374099,243351.8114604967,434485.74005231576\n120207,81193.38397879469,139381.8130895358,306721.5606094892\n120232,584813.2835002627,2357290.109467329,4270557.058215292\n120236,24383.025901110454,56620.84695942119,115396.4891305994\n120241,43650.14589718192,94817.88964625982,196341.65854980715\n120259,23747.965231579397,48699.11970486398,100150.33206109138\n120260,17763.301165549485,41952.628235995704,84218.82234739847\n120280,73013.20194578743,188941.39289253758,361778.585768923\n120282,17832.467825360407,40198.75285451988,78601.09873119686\n120291,41135.51018092782,76751.46364971889,166264.52739236155\n120293,428768.95399744145,750747.9807504322,1689362.0194921056\n120306,436314.79918163334,467323.7177695596,1260057.9246726208\n120347,4541.61765803043,20525.622086173684,35870.223741375085\n120356,5978.538515534582,17885.246961361645,33225.450984212686\n120358,705970.711191042,910940.0256063105,2276119.3419539034\n120376,12937.066096320403,38770.498582065055,73129.48807815564\n120385,44827.94308749061,79838.09742976663,174402.78772177984\n120400,18646.073035925183,52121.46442842043,100355.15337005854\n120405,112941.42248626532,295615.27396168053,579760.6090940554\n120416,17408.227973037654,26513.74441510092,59708.835723770164\n120423,86771.67414679358,146967.2067629914,328538.5379273278\n120432,106034.79442790797,120577.03055635167,317882.5148454417\n120439,22400.24986335348,84943.13642240438,147636.53918250336\n120457,172092.27130872483,451078.30327235523,889567.8147807815\n120463,60868.61136581419,154046.04111189308,309186.19202053174\n120468,128851.30814892412,258310.8860532187,543130.8894768077\n120491,6176.792819872522,24357.33288321203,42771.22783063559\n120497,78926.04650887095,224301.8483805028,420235.3278344365\n120505,516398.48241827334,802104.3213008357,1924880.2028773753\n120519,320579.3584443663,813081.3932024629,1642477.1115619931\n120531,11324.139469378644,38942.54426911911,69694.52977399927\n120533,27646.37578787039,133987.5665697706,229794.67850183515\n120536,81898.09892975165,148112.3972913163,322478.67107496865\n120543,8281.01533124414,68970.45218797184,106983.92977974119\n120554,151444.57018132135,309232.0975532367,670657.7753961222\n120559,45001.66582960176,117314.23001122136,225002.90237912937\n120572,54877.14291201224,108937.84604677658,230080.86753022537\n120589,688194.105709531,2028600.5644309314,3973782.6051139394\n120594,47894.83499498869,110703.23667894847,219370.54146979726\n120604,4125077.000823737,5881874.768551871,13825216.590496399\n120605,3935356.5499479985,7157388.665087123,15525690.72303461\n120622,511078.9976555785,431124.6000493323,1368038.9328595619\n120625,9689.470845460912,21935.69943778982,44228.022458361724\n120626,273581.3043221118,518126.51596218377,1208623.5224256413\n120641,18499.043914736314,53917.87575476884,101749.7837041977\n120650,62096.28178946365,213569.72919398945,389206.92386613257\n120653,115364.50901445247,225612.76211368982,483088.04286456027\n120677,11215.947014479572,42839.738981096605,76555.78497851019\n120682,856190.329581329,743694.8596017831,2259086.4211058207\n120688,29907.536273629597,89512.99010316233,168474.9638819571\n120693,20691.22542634982,119533.49868672615,197067.69312279677\n120694,97172.63970974881,191616.27630261972,399284.64542949287\n120695,48822.15389547036,159036.5105115691,293716.5123476049\n120700,104822.89609040217,175299.18343623498,380495.0549824524\n120711,402739.5055119016,1325656.6951408416,2428084.1400755667\n120729,9401.21496087133,37540.381997962795,66831.32232062574\n120743,476475.8638956163,451330.3027469464,1334255.0899476046\n120765,14360.652567946974,43618.15537159933,81608.45898332848\n120766,56157.97641570163,183825.69598829834,328865.8309673909\n120768,29456.81502231525,116769.88591856367,210422.0960491583\n120792,28339.78621725597,106090.25070875515,190244.8803674428\n120797,41101.46433091583,75914.03008570232,162955.1500921655\n120804,100428.0880149833,163369.9459089519,374057.04298015137\n120809,11288.896089106142,38917.07921259075,71080.0457630016\n120831,14417.58369777078,56803.72809568443,100834.96751312044\n120836,269094.37448379345,489257.47010418976,1062462.8737854906\n120846,25730.339869628104,52890.52445764077,110291.55290906779\n120855,57910.699443056845,112401.53793053947,234845.750348421\n120859,202468.7274894278,272722.4003704954,666793.6161201051\n120894,99285.28522167078,240029.43186908984,466248.9253526672\n120910,62574.53554133475,207467.45848874783,386369.388345108\n120923,50876.70639865878,114115.67401767315,226789.50220359437\n120925,99961.01430277461,224398.1467682371,450803.31063309143\n120928,24133.67822204774,63697.56339826902,121701.12208470746\n120929,1716814.4595135113,6446686.749749257,11845063.158158176\n120934,16718.36031794079,50600.65398744241,89820.98504944748\n120947,750433.5776405917,974666.6123765846,2521685.3693608386\n"
  },
  {
    "path": "data/month_loadshape.csv",
    "content": "id,1,2,3,4,5,6,7,8,9,10,11,12\n108585,178.96607321422954,156.2157615515202,109.81367662374429,96.78573476528025,63.69382278253658,65.04275238545634,53.43386545628892,56.690940887562434,63.39514100167378,61.0215462299067,120.66127955775407,126.63265024304529\n108587,45.26015310479342,35.81842707333236,25.759601879539815,19.360345079301688,7.464581094642364,8.987339995038587,9.260272346848954,9.491315130413001,8.319603079937785,10.691725291766012,26.97049369197886,29.067160705115022\n108596,15.892253754250216,13.022825847897925,9.631313194707861,8.563864824918113,3.4799873024671144,2.9106983768529386,3.5674868268306175,3.658949657810009,3.0438900169364955,4.082820165350333,9.667833103480646,10.205378907379709\n108597,15.427013673706822,13.130643792476492,9.659251213328709,8.427097808154256,4.086910875966775,4.114092566892657,4.542721759393891,4.6281731083476245,4.0770022321923785,4.8902039763390155,10.25387612696432,10.556433724630283\n108603,249.681020822873,214.8480189626379,157.241478201407,143.01226624289558,100.35588641574111,102.994771658663,116.55260360667859,117.12868174933817,98.16387160681766,107.09670912269227,176.87412751707294,182.26114624335202\n108651,130.83588840801025,101.25703430795514,65.65553949535212,61.52439805062091,44.04467598827515,44.74636124976597,51.86710026713611,51.1586779538316,41.61520647782832,41.923475287929456,79.1953075414117,79.13530569251535\n108652,543.7675219285371,453.59574953112417,325.8701489480818,279.66237711394666,175.8507836041784,169.67677744268622,172.78435289726707,172.00641330049183,172.75509276674606,193.570755206167,377.5469137497105,378.79258195050414\n108655,69.87420629574278,57.50177605566508,39.36673767106564,37.90632996884626,34.002568972143095,35.31714254782649,39.863896654603174,39.64347316497806,33.26575024307318,30.748488575442146,45.92868371501271,46.75584738189856\n108657,41.968679388483494,29.74165909666744,23.79534588739902,21.238904110310607,21.683511839339154,21.98169742273194,22.029826502695826,22.797852183810832,21.158416092164426,19.739587845478674,24.575532654946265,25.08530556001923\n108686,45.07142291752183,30.388696810574807,22.810471445327885,16.272581896298867,9.469338469573454,10.472118285313314,10.486878589089947,10.879957174917042,9.772168206158796,9.159255911586357,21.807181313506327,24.099834853297164\n108693,13.251558189518068,11.434878859969249,7.267053363527069,6.786008690026881,4.98181391310713,5.435615594136687,5.95197853572163,6.011706815582664,5.130512268865592,4.799844697254966,8.228486504612924,8.392147768066339\n108704,46.452053769008074,39.23003738480325,28.25582099716143,26.175950097927288,18.36283427233849,19.174664232153834,20.419747794549803,20.73467365331337,18.340413838054566,18.818183671089436,31.3060532455212,31.762400784901377\n108710,41.432275761567894,34.75547796565035,26.21300098209253,23.554831973463028,13.953079488505422,14.193945667730341,15.578649102864446,15.856769904005677,13.94799066014542,14.82604854213329,27.181258107049533,28.314636357070707\n108755,5.022839793522658,4.15694482982086,3.4438346010010537,2.7868727591592357,1.3482466004029847,1.5151347320139485,1.5884384653106964,1.597466156476849,1.5524787626508019,1.6852896660634793,3.1439241959599986,3.3593216396753562\n108762,41.959829376724855,36.87353133581682,27.852005152197684,23.832109413906437,10.357396516557055,8.873309364191543,10.519274176431058,10.490470953162797,9.131418212372948,13.225219761956927,29.10089241147699,31.163627708060098\n108774,42.37359919343361,37.365416922268935,29.59799556805179,25.11668033726992,13.200847352799162,12.949570336740834,14.346157918566359,14.570654945912722,12.869572708313493,14.898138388119982,28.88486864219093,30.893524195522357\n108775,43.21790228006449,35.81638541577673,25.120609101877623,24.567792126153595,21.40745382294679,22.027424680442863,24.084467284789568,24.406121112472626,21.3871169154993,20.672745627374592,28.517886507473502,29.0392372649811\n108789,227.00097826385866,184.99142716712657,148.3303319062691,126.86655722412495,157.76723988692763,159.04747405195664,129.5057810699606,132.34361173901138,151.61278340323022,143.08508413375174,167.34210364723043,168.06057728917347\n108791,38.821696363020386,31.65356541286105,22.35410641883391,18.81939243988925,6.177445495835048,5.724945033883314,7.342751990164167,7.431478362370447,5.8724993960852965,8.416987874735282,22.871260024968038,24.546300764705563\n108794,28.31487436469394,23.97551660046474,16.76663357039204,13.721816823589949,7.5026457527699035,8.342580254525823,8.409529637804162,8.607560908806331,7.592090377931859,7.975575987867535,17.852183992189588,18.36851779710743\n108802,341.88704043471904,256.5887242200901,214.98361560951017,167.32518467955572,158.77761219827505,172.08874307764484,134.9571828095784,129.24931487120327,158.5287837443021,170.85249015491408,249.57952949567638,243.50409008905183\n108813,822.4007690957357,744.4094962163736,565.5114971624004,506.59608164290336,273.50290946498336,263.4127880003464,207.43077568210222,207.7904354966048,263.3504406004495,360.2312334137273,627.6964250438074,624.3948719553221\n108819,12.315276260195898,9.051160421375798,6.7645711600857865,5.191508534917699,3.733364252985234,4.130625112308846,4.227008046795126,4.200734165173223,3.9293083437823575,3.6024462996032702,7.123492668841995,7.385308128600682\n108826,61.4265938438641,52.58548323961705,38.88442184092506,34.009383211965634,13.836847060330483,12.509265519409192,13.40776744996435,13.761811581893541,12.493907466561556,19.923919408157104,41.09586496437113,42.596176245304214\n108838,289.8684735127909,238.15392394415292,170.59681265887696,154.7065209261971,92.16881149763988,94.52638788693395,108.79054136161841,105.88559645262696,91.80120549176243,104.41020780191563,198.11085224800237,203.73078526760125\n108841,93.57794216267217,78.65157855374726,49.18364943192709,44.218123905758,22.54355153338599,24.496513112414625,26.07629717019728,26.296482040801983,22.81179569453296,25.807792586548345,55.59616414042189,57.111466910562044\n108845,21.275937171369304,18.793447301942056,13.977816218081614,12.667539604292637,5.862147622717496,5.619057175355693,5.983157905737322,6.081693598861576,5.6293753562790645,7.904824046236533,14.811721284849973,15.159009760413552\n108860,46.91003621408396,40.382784019805484,30.341451782907708,24.46137729265566,8.949498173795627,9.737946504422762,10.063112213510783,9.952051787780166,8.940531551543213,12.056700299640243,30.035742197780866,30.142766830329535\n108880,70.1380750417333,54.387774352186746,37.49907574568266,35.400053473037126,26.348640704190696,26.81568685817371,29.121751457966912,29.98739803634775,25.852218289022883,26.179348541140044,40.82516193029071,40.3050651959515\n108881,202.97786869871143,175.92000253407028,123.76160058710506,112.083347900251,98.41820849936553,107.9809423194591,115.5556499167397,115.75574855177831,102.24520720713089,91.70856134843336,137.29871821467802,146.19760807302916\n108894,929.3883085640599,837.3727864649109,752.7055767443018,684.6827903239482,370.5156069464358,341.76962751735584,320.93168490290225,341.45645586777994,356.566281310237,520.7021608460298,751.100000564676,725.3769686047575\n108900,31.748749744794946,28.095111728871384,22.801166780235587,19.099228910397873,7.684404778134029,6.014029078706148,7.208222156848042,7.4698551790458225,6.319690968730774,8.674305748677892,20.581938232964315,22.48906987350079\n108909,49.4876913230803,41.9182237672553,30.007523169870105,30.361763981029547,29.199909945720176,29.68605459017065,32.19120325846197,32.52117612853163,28.624752247787196,27.389983188826374,33.99600289652981,34.519738764191096\n108910,67.54564726573719,53.09973363829653,32.09370785088437,31.291281112031857,30.451497768518923,32.138635062644944,33.92228550892898,33.96143412394171,30.549216569381006,25.489764158624766,37.95665837606838,40.58771226648184\n108918,423.8229043396233,368.47122733912846,265.51904130246936,249.2297926043129,212.65454083778843,222.61264680969413,223.6660442360941,225.44653793622211,214.27483707399966,223.63662177227377,307.6084583402569,313.70943688053086\n108935,206.9996979192357,142.20603558014844,106.21314578679437,79.14240028627698,65.4558789822619,75.7283697060417,67.77128958372792,65.59493134660724,65.32686818727073,63.11827044329758,129.32736217070197,128.22266791733773\n108936,131.1945784928851,95.46011909359184,60.286272875068576,43.438476020615596,15.224609251971653,18.770841846009244,19.090472167707787,19.391762508082746,16.089348994754996,21.909518363587246,68.37222343241638,72.02944296750482\n108971,608.6053912894009,453.2940727124098,366.790015103764,287.75179083280904,189.7924346629373,208.8236290438621,192.68132719959027,190.38855385690113,195.06037226702745,239.03451899989628,427.1418439863816,440.1583727189356\n108977,40.23490481553482,34.49234018044263,26.790956163202644,23.05668604924733,9.804531347374354,9.178078017326914,10.606683483235956,10.77480318151037,9.186952101966021,12.315862934675161,26.861059142294764,28.67800441082394\n108986,111.5940288225223,100.06593726097357,80.61453481939533,71.90117171308312,47.52401461576482,45.22835062672804,43.217183092972,42.98157672451101,44.87721482545178,51.943115091423664,85.70567843944431,87.64352584130862\n108995,35.01823224804117,28.53525626704645,20.01574512441656,19.462182643499222,17.587528911788578,18.56723863610576,20.59422344101644,20.820469986967765,18.282869946849242,16.50285670670446,23.31987655418737,23.76495769644171\n109042,126.24729010636085,103.13390013862951,67.66167331927248,59.85671684334409,27.79887336707474,28.085582585178454,29.728857476144533,30.57650899106419,26.670081909354444,33.75060857541302,72.37055842847487,75.21989193615018\n109045,6.981630922138259,6.016334496616799,4.315416015538102,4.423192695041058,3.836750674753954,4.239308293997808,5.372929864048441,5.398976948574024,4.449199422104603,3.3355931021746175,5.175580916271314,5.349429471066629\n109071,179.42635057084627,164.23884788686726,126.6563845045771,104.00588199241047,51.65712961043103,50.19149536342755,45.920826148796806,46.33806809657614,49.015222120718775,71.39176774909049,140.20198055000353,140.17596233641646\n109092,40.764093204063954,30.342106447971567,23.84956571811187,19.14685316735726,9.655261733005666,10.859308427040263,12.019599560287691,10.69568430834371,9.707986653574098,13.87277724003933,27.582552821670916,28.609066084520823\n109095,27.1294733113215,22.543099431680993,16.266819565131865,14.852143371295709,11.67164718959367,12.646779168220547,14.243203898911894,14.503576452420177,12.571898032164958,10.928053498221212,17.000504294140473,18.066315418434613\n109110,105.33124161519522,87.62324022263036,61.93106584059978,53.91167760013331,23.40197283224884,23.69491809940283,25.698245377784616,26.00673835884627,22.992554114410314,31.64185201244525,68.98196439045773,69.68280902249967\n109115,1019.4354658037824,765.4742214350595,631.331848608608,480.9627911059916,255.28737262644458,268.361159847209,232.38655314896522,226.6913666816789,247.0680450208238,336.7114674531828,680.3389871464352,678.3012864466618\n109116,241.96946875311355,191.20923193521654,123.93221179224078,105.01464816240413,93.02317906879325,102.97081382657575,85.97249814446046,87.31101706445536,92.8982745682886,74.61967016113198,141.85171945085128,155.34420012840354\n109121,12.38391763353423,9.377734228432319,6.795933197645812,5.3964207118965675,4.3330792397063185,4.780924526452641,4.897888059697689,4.795512765634928,4.447279747046973,3.910604832742288,7.4416752829342965,7.838437187914891\n109141,44.233454203349226,38.3309536035477,30.958861321200793,26.155286426241627,13.109061765580757,12.625275054311174,14.30554608858451,14.632432121003268,12.80370012393169,14.949122502715733,28.82161903261853,30.737070779091738\n109146,1198.814440240996,982.8884696844015,675.880824299714,563.7031592964239,337.9643095465832,345.4956130131,330.61626452401185,328.5911898579386,342.05239418429505,395.99159511100623,784.4188850387508,821.958015010663\n109147,1512.114384296411,1267.1556686157326,905.401681751937,806.6628942176804,632.4794616287669,630.8354701016231,567.6525269026912,597.1964126788088,622.6488979679993,648.5982080708786,1065.7487425724344,1082.2147984280625\n109156,479.9394531695234,332.7510781499916,265.1747074087778,205.6552999887147,135.0393206692196,142.18755337277753,119.39531179764121,120.12490018692745,135.81854848115862,162.4703004816692,305.3200829834208,318.98563043801624\n109157,19.19482954465636,16.24885006515705,11.1152796959126,9.805449925357001,3.5138578810234105,3.360373061378091,3.9670073866965176,4.104783246864928,3.4751334264024396,4.648064726078829,11.98327207804241,12.503211211934092\n109164,21.410292799916434,15.06441925175937,10.88156301445294,9.755341729733512,8.58906692269508,9.89695330684497,10.171543279884176,9.911814168902213,8.986312423911896,8.016824448474132,12.58193881994804,11.757188311423091\n109165,344.7565421444574,288.6247828784521,220.25325229303863,196.6071943514019,119.03489909001487,104.93947927624174,84.24304728390113,90.72801595902759,111.30211486586276,138.51606603510646,230.2173021968222,237.67873469565924\n109191,32.15855399453603,26.41169752078135,21.375792331876557,17.556314922628516,10.37324290068966,11.382494513800765,11.189441773981967,11.58910783584722,10.734199404297412,11.646036649529833,22.04463092436284,23.52737687727525\n109197,32.32332549134983,27.282809169591367,19.064245476499238,18.018324041634717,13.717990948320294,14.678111984276544,16.56156279357358,16.738933738654712,14.620597788430581,12.578319341625306,20.625710416117347,21.8412582461046\n109204,762.0792192380007,662.8327120209343,507.2403183117557,477.4796630665131,468.0750081623933,492.8020944147596,496.15821027407475,497.78914808777165,465.8779600538877,435.001334801404,571.2932411269087,585.5982650386335\n109223,172.0712538703042,148.51877319581757,135.7296009721906,119.07448865815044,98.14304936517651,105.14197733590889,107.28811396485709,105.69579451174592,101.551637720265,108.75952808831056,143.61079475700842,144.3219637061481\n109227,9.84538577609473,8.106539911286852,5.917129523649598,5.190733276088817,1.5156790648657787,1.068900522637582,1.401869104875662,1.4900643583418658,1.2360853473229818,1.98254770014623,5.6936398213322015,6.086090210378659\n109233,92.34308153760779,78.34450979714063,55.81099397043124,46.85875777383708,16.80319446616752,16.781474013283553,18.79442634039984,18.734941045028616,16.384541466363153,26.485572339223083,61.348728228071025,64.08182138240755\n109239,241.8342376751688,219.33673687397342,155.9584673723448,135.909136538777,88.86151364118388,89.88101176624858,103.71563973494848,106.99888884630606,90.99980399696277,98.65751454029736,175.6571184310285,189.30345181556052\n109272,36.08058319911061,29.946804252846164,19.80839563647121,18.02462141564175,5.329440817114314,5.260306031957895,5.683098588047012,5.863553617061324,5.248024488579078,8.98871601869586,22.11760281145389,21.696899681693928\n109307,16.815630006237697,13.691909494224268,9.78480359271,8.754807603444826,3.662034117641264,3.5221473096782248,4.5509364052563726,4.634151486344946,3.722697670668581,4.032309547405957,10.148523646592652,10.678794781458262\n109313,81.95738655803069,64.82718997837078,44.81160271374328,39.639118477773636,21.954256474965167,22.817729933285342,25.94396913505608,26.577116075228115,21.987170394255994,24.107660837232377,48.38026950089115,49.027873162021265\n109315,50.941079870458395,41.51620402621311,28.884763595505465,26.424618368151652,14.183120208576394,14.516788227563067,16.402896312674528,16.285463221280384,14.20755390624153,15.616634090053964,34.753425961519625,35.608482244208304\n109323,270.0490672199972,232.34636075671642,175.30150670540147,152.84512240562879,85.09793492965007,75.45275214295489,61.54956913264398,64.48846313966304,78.84258246342591,97.56196080090993,187.56613216557247,193.8332509507588\n109339,51.01363061999207,35.4043099271529,28.507860993001152,21.74419903260909,13.432006909056646,14.90843856414479,14.801808802472625,15.486713016268801,14.015567108418749,15.021394075463563,27.85242296835013,29.637507901873338\n109349,24.10918969444418,19.509821249653605,14.344207169113197,11.75293946594563,6.412939272094867,6.6415176922757295,7.001583034474043,7.159348941324706,6.607624022924996,7.317420927076562,13.639592976469428,15.06660777790675\n109382,13.415136020212959,10.418906415537792,8.137845323824585,6.267560271490825,5.018954228723029,5.8241180376102495,5.969392244061868,5.799169611669325,5.368508673484169,4.693462357774674,8.66095751235579,9.048150831623465\n109391,40.38100935034795,31.825075119847085,25.179677000753845,20.2364272757016,15.347100619802847,17.371829404814797,17.64383927314346,17.337357024052448,15.906413002351481,14.878372551474351,27.03367120156129,28.195654164764427\n109399,208.128476921902,173.52953707973742,109.58227413693145,101.80970193190866,104.26203145629417,114.41106233699263,125.61631842542516,125.67517704957119,105.77076866662173,89.09045607701809,128.60891175695596,134.10653148359268\n109401,51.08240204862408,41.073923302618994,28.02201596545221,25.222738690445976,17.153336231054862,18.441695330139755,21.242365877620955,21.413973168575083,17.91161465970329,17.154138553094498,31.47888813913674,32.43246624573218\n109423,192.06331301648387,157.85371122362884,130.5075877179912,123.49019663431427,138.49835566692332,146.14497585121,145.6126173457063,147.46641203855899,138.35161957112075,123.6761713022941,142.7305715770218,148.42140849873897\n109429,329.37097811865135,304.7163452901351,248.53628455724976,242.10153169280053,272.4504833061156,287.3968460023817,318.94066254227494,321.8075796593046,278.0467021487452,240.84847872554468,259.5546298070024,264.6170986751357\n109435,39.845965559577905,35.41136442850187,23.322981596082954,21.568674712519563,12.260579385048088,12.651167976435413,13.608671514908519,13.515012842284863,11.952480445455372,13.748587766671012,27.845784293923472,28.896910628041812\n109436,197.56404571584724,171.17617001636998,105.10679345777197,107.51847629624875,140.29449427597658,156.44148894737276,161.70860012517912,157.23280427956834,133.87843839232556,102.54305151533633,123.03662293109939,131.5943035173245\n109442,117.25402946097543,103.08595260045063,80.49407703320819,73.35639670530773,49.51057548195555,50.673639223621095,55.91973593959683,55.919336697961704,50.63557132147978,55.061768080702166,87.56550707891898,91.22767627717299\n109460,70.76392697292428,56.3074870060711,38.44151648273874,36.08270542867454,27.15713211645847,30.018885587267167,33.78780811552674,33.82006375982651,28.92314988328204,24.81484880748912,43.15894937527962,44.01917976409282\n109466,1518.8890245869102,1292.9794601872486,876.0584718470129,784.3480024752608,653.3463645241546,729.7860206742567,786.4830006921294,801.0199035739633,675.4624618733924,585.8291472321005,1012.1208785305685,1050.4806836595726\n109485,38.21626338143168,29.489581182767044,19.20286822508125,17.98389181589844,9.760855315799148,10.10687747701168,11.81294884312359,11.825651104761366,9.749257790684188,10.096236929035546,23.01787235533712,23.25766415279694\n109487,835.3559975900869,758.336282821699,611.8813510470635,527.5544262880056,377.72648050586844,351.9205656174529,310.2531825065441,325.9479632638262,367.647742336424,415.50530874147586,635.821205641092,666.4663591063751\n109496,71.57814400929371,59.69281036207438,41.609647527914625,38.28315608233371,25.33687235080674,25.59787034964425,30.13003878829489,30.287887143964902,25.27696488788542,26.462734606809075,47.58998429909929,48.91986891913226\n109505,16.47320738613747,14.140429916499432,10.648445249721885,9.131231105215608,3.9884281212562325,3.6845601236820693,4.024119522681747,4.179546295229136,3.7691636668965924,5.031487088210602,10.406849168809774,11.31173327953149\n109507,584.9632000431567,492.4122527735873,369.79893079388756,318.5940896165986,197.10025730741614,189.51605099788418,154.61279462187196,165.18331666216292,193.77477094709118,215.75109686240967,396.4932814646571,408.72669394566935\n109510,558.0322847713899,489.41626060015886,386.61209885224815,339.6482362614625,208.6040801021996,184.40090433537233,180.1685213064971,175.01047500895305,187.92214177002703,251.5342396323695,427.59269216474235,437.9609782534836\n109525,320.5035106608027,255.07184497233106,204.6364482872942,174.7464961737302,161.31599072822837,164.85962096420465,131.1087722376105,133.81033981628352,156.9030660209957,150.21370009623294,232.21114104949214,233.45842398085247\n109536,17.18873826172539,14.80526746666961,11.29363934630211,10.250627633912025,6.6383816294521685,6.909632814195546,7.6488302134088855,7.836254017438695,6.929597259109615,6.528286714280859,10.909352914911905,11.764690903186185\n109548,105.00588156233165,88.93719489132384,62.340328509682635,57.02695396092033,38.8329014945319,40.57462756791083,47.073576502337524,47.1599057368864,39.91744098574628,40.425007281421216,69.91531589032395,73.63741939173251\n109555,129.748412145243,112.93970614270367,80.80646626923983,72.9277823741887,48.295500588764654,52.668748857817455,55.8149881909171,55.990109527763394,47.14633713333823,52.72026340315174,92.80945716507111,97.64458200051308\n109557,30.21050917215564,28.4477383150106,24.61124534950408,21.24738954254324,9.279528965704722,6.615135955506931,7.101098411311273,7.267775899074403,6.354415249139011,12.32543207175601,24.114886274738108,25.643991937960298\n109563,5.222544324980028,3.840190968356763,3.2029151878650377,2.3683009227905374,1.5640984891751633,1.8901975065916394,1.9957398474019181,1.9549274058326471,1.7639676367843,1.7523782308438627,3.20676556043047,3.484089049538521\n109582,28.97658416913791,25.387810529912393,16.245418659695176,15.103629485425895,11.939025904036635,13.11480926377549,13.791761636399958,13.60461949082113,11.86446117102592,10.126579474473603,17.569454817129863,19.083241347472168\n109614,32.45211104011086,25.55764056484339,18.869464916016046,14.356182599290651,9.246509812207963,9.837108052928532,10.08515142495364,10.418974625392595,9.46344031156909,9.172354307342784,18.34274942486553,20.0487291525107\n109624,648.9441031553869,597.7713660968194,441.21628955032827,375.5062947315892,198.19795675572607,209.6815465342212,193.28730630081256,175.7930466939639,193.8740677743235,253.31681534539075,503.05331988586414,506.9960769604707\n109639,62.23480238385686,44.944381391017735,32.273793297417335,28.000188003045373,25.726505857996667,27.060199135974532,27.27206018985194,27.8928551685523,25.081846616343906,22.421611036584114,36.57248134211139,39.029938357124514\n109642,28.55703246423471,23.635248656928905,16.224078514034595,14.733351213095307,11.09461299601657,12.136120623104173,14.617872125521384,14.51178158221634,12.04437293450831,9.846468726741177,17.816493778445782,19.067573446837972\n109684,16.963262965789372,14.610738191091121,10.83920424311502,9.86997649761377,6.282706623793417,6.582228625343323,6.941287923582608,7.125043993159004,6.319851866711465,6.894579309228841,11.691737523439375,11.791114000055458\n109706,70.73242308645409,58.99416255676532,41.16224249887515,37.69525658049311,26.37749827840852,28.500149378099078,30.270640770710944,31.002526501646905,27.499833609608782,26.277480340259018,44.161329694665724,45.02630104085022\n109711,59.70909147892588,46.6031460232431,30.555324517239,27.757445082114646,21.316281475980023,22.16727359150383,23.636328079423567,24.242318541779763,21.40579257955969,19.920052573143543,32.66661842059474,34.30760438572104\n109722,18.36470550381819,14.553788549038158,9.43602078923481,9.044122802831582,4.664221019427409,4.786884699755536,5.698884329133212,5.7938031040379325,4.880362767478192,4.8641113043128215,11.200989369656813,11.373060196635262\n109725,24.041935359764363,19.664697908932958,13.45730000256133,12.550008052878697,10.440348289260763,11.619987165335765,13.093054637091377,13.402871087871056,11.739917321974106,9.54060133149515,12.70857143240184,13.899497446961279\n109736,49.38960521616385,40.16656576943252,29.99717642507998,26.40127044070669,8.737690633434864,7.6275901531193036,8.029392200754582,8.233763183419876,7.407583819003036,14.034256412522971,31.651056286866382,30.69725362853676\n109751,86.53442153762393,58.35045315250828,44.80294367240861,36.452916865517295,31.098227354283633,32.69425007109793,32.47894717653796,33.68617840523169,30.593552790441088,28.690299938857752,44.8761450789125,47.00676477081682\n109754,232.06730602453754,202.59921530533535,145.60512009216464,131.54121834065873,115.5996569475732,120.61970122304928,139.40263197886645,136.7849482365807,115.64900803297539,112.99465791854053,180.4946524445107,189.31328849610236\n109756,41.21399713598289,31.805013721400865,23.184293278072865,19.068142496020126,17.310989193299818,18.428883529185796,18.668760226796277,19.143312979281877,17.4483340539684,15.980187249980183,24.22838357600013,25.317223601639878\n109761,1784.1075456778385,1571.6815712813093,1274.3159416249596,1194.6393257692266,937.3489810888524,974.3665964641106,1019.0628749821883,1009.7205723468817,944.9624381209941,996.1825095738559,1435.769966525652,1441.6396021063163\n109773,12.040709054289007,9.112456894122435,7.951498114106926,6.0991289451462425,2.224016507466894,2.4097053360787655,2.6244862617479487,2.710188783925687,2.5660375088121503,2.7429793047864766,6.513568889901805,6.982925465820858\n109784,466.759245588175,282.60319911194875,218.28518706781534,147.0506387421428,93.41126092698194,103.4095203467489,95.61996955133237,91.49309202878024,90.29589073793402,106.46007323437028,255.60730317737273,262.6659570232634\n109791,14.5595584563126,12.758142995027491,8.218062258686539,7.2508879262427826,3.131966885922173,3.1142872332407743,3.5172664987869524,3.5542567514602927,3.1886666450263004,3.6216432119412345,8.30342304338527,9.214773046645657\n109802,19.08762919270261,16.27147802005083,11.535318720179337,9.87323110955815,3.7016895858085754,3.706684368032847,4.180916106879958,4.3161940490922515,3.8119887949248468,4.8960226398992654,11.752594347629424,12.55095104310976\n109820,69.08834674350834,57.92793109529831,45.354992274071755,41.3987089845188,27.976783105917317,29.321031103195146,32.63174030993021,33.30330560620519,28.812229861269337,28.97132996250961,46.27432722042665,47.28031549445376\n109824,93.22731653432187,83.81109647024549,55.98982269078286,48.93185248699885,20.55294619541506,19.908249497126473,20.281587482056132,20.613917523547137,19.313825713749853,31.692352141749677,64.15049249953013,66.11857490394539\n109842,57.76338764848125,47.05858326112149,31.274188286116622,31.365404323622897,33.27808191991519,35.414430812021195,38.2065511919344,38.087728057769326,33.82978491365295,28.524543930172133,36.723952171327895,37.684580657781375\n109852,31.045806789871172,26.718946343936626,18.331620981455746,15.373761013683946,6.138367631855401,6.533062776532333,6.8390743023167495,6.703399413220787,5.9641684896163065,8.16445629117647,19.613418335872097,19.993754358869193\n109904,7.3535772285719005,6.249562255951309,4.660620717838557,4.150389138179382,2.0748583710213433,1.9839724852772065,2.3471736207290594,2.3729397356156343,2.0540590078044354,2.177615481279423,4.7639431001439005,5.081594717537828\n109908,185.43399129893606,163.3165723444619,107.12612157690526,96.18897958778834,68.89257253676176,68.31667364936894,63.431473926622786,63.70886090655442,65.19948148513377,69.04210143943085,130.4203990547044,136.75964470457265\n109909,53.82039181699313,42.58009798239294,29.731040829232757,26.237744283352313,10.619332439491677,10.295974837715976,11.730774623699402,12.04026041559039,10.05037519764256,12.83576762972347,30.729588234393173,31.872328830605912\n109910,80.93493055693672,63.43657658375791,42.61090139333928,37.08279020304251,15.996537493164336,17.340286534310085,19.092373264081864,20.006861154695375,16.555030660174605,21.026318648210353,45.82976906800092,45.627962396284865\n109911,1230.3237564883443,1072.628062022672,817.1052247254162,751.9459106608384,589.3421109227828,599.3007755848565,607.1217690673403,646.9369921391999,568.0615463469269,653.2165839833036,966.858658843516,982.3114028194395\n109919,1437.6445528385962,1337.7799469763324,1217.6800406172945,1172.9522001927166,1161.6401636575463,1199.9209283408009,1195.9605579572903,1211.901932715805,1164.6803529909891,1163.9529398028471,1291.5777122193797,1266.963439271521\n109932,22.956411461518098,17.16768696999552,11.078706917729072,8.478584844732133,3.7284691285629528,4.123582269927842,4.378163992611942,4.39614916094719,3.961847546166554,4.475970547882624,11.481124727443774,12.602085731279029\n109940,43.518828692997204,35.70682979351017,23.995751019561833,22.81859478654034,20.064260608701588,20.979244272293496,22.891106353220394,22.847935211877505,20.31612379371317,18.53444004796639,28.41236149573483,29.504767852404413\n109954,10.07055315371737,8.458794384137613,5.735919763271935,5.074570560314403,1.6162065612993033,1.5136203087727391,1.6841004405440643,1.7131341120644945,1.5144807626297458,2.5713389021673017,6.208683571784588,6.2702830730079935\n109961,802.7858403752871,631.7929997662628,478.8790956613784,374.8203693426334,218.74101559738847,230.98150395921067,198.52722980412113,209.9214372410357,219.16765900374384,285.4170482847004,537.549971469978,529.4975036121045\n109984,32.26255209361728,27.234932193208216,16.685094412353454,15.30415964558181,10.761312684903066,11.856314851519892,13.089668927599842,13.11969381154782,11.257339896681462,9.990737377242567,18.678097906239717,20.068255809895213\n109985,483.3337759370385,396.6060038552752,280.72600188332655,244.3947749046863,148.68849186358085,144.330055300239,131.198358731216,135.52132029717097,148.79584349299472,147.39705613202023,300.8146792342042,319.4785309170281\n109989,45.98273327406304,39.643769343687346,28.412491045548915,26.26192453024936,21.93427625097244,22.593842852487093,25.475489516896996,25.093720360442504,21.73213911997525,20.71880878899233,31.812215212938618,33.51695239595605\n109993,191.87571846485068,166.68532520083198,117.85154901459168,102.92234772638041,62.94394699309654,71.33403058666778,81.58090147793999,81.80229299245696,68.32518792616081,62.22879462751565,129.1577542623388,139.03449174005326\n110013,173.86424393082498,144.973379071877,104.28577818777042,90.40849635323684,45.88780237032487,41.02942107112625,40.0430072575968,36.52442940164633,43.09545020501686,54.024970866628266,116.6751903633087,120.94806240602932\n110015,20.658572565343817,17.464569427172258,13.185282747736725,11.721389946862274,5.386979321987256,5.086356288311474,5.921727121294374,5.9220327973129745,5.134121827130931,6.382339248739226,13.940625003258475,14.569988569606318\n110026,21.534904387882868,16.293198060200833,12.681192605169846,9.943145993720476,6.303832281680209,6.986050131803418,7.090331914855048,7.45197584499501,6.963944084910562,6.1468742458595305,12.139669300675921,13.870190603889746\n110045,84.65092837756524,71.66518167662377,52.71507816144537,46.35176184757836,24.64878825251811,24.978223733335664,29.287577799191737,29.409310896939473,24.629481251499847,27.928143884427765,57.56324183646621,60.87050542002583\n110054,69.60703915315156,57.27991013225499,44.312416288019,38.72937807928599,36.99552283063524,39.96771976439133,39.79112382719208,40.7317904683642,36.060292606413846,33.39559161165809,48.88947330718422,52.89714434696427\n110061,28.622284019476304,25.71782494051643,16.861106510616715,16.154941260320463,10.924985832293569,11.662585807724504,12.548801229565548,12.612940325147807,11.297690757719307,11.021290143814085,18.91817122141536,20.032229303842563\n110100,122.4997871240597,96.40014273761442,64.48220207333503,58.077309469213105,22.03868427304656,21.797843146141027,25.194167362058316,25.80477849205288,21.7105867959679,28.075502156684138,71.16961477381216,72.41545244115387\n110174,64.90727621263274,47.72574243823865,35.35236726493478,28.172093253440572,20.285517263858786,22.217930467156965,22.164972662736364,22.865528534493034,20.365668223315303,20.5176243008261,39.366102220779645,42.3325700901384\n110201,320.7390983910637,268.20084209443917,171.63386909939456,150.80360924133342,97.46533606498117,107.17321280154246,118.97636461150582,117.160341389845,97.52440794926498,104.66431948432238,211.0650729251564,222.06241599452503\n110207,646.916808907908,564.8380727932878,433.09279954051226,389.37279941857344,332.823950559378,308.1032781791193,271.5070624345937,284.99594555236104,329.7649208305379,325.43967236384253,486.41663102574705,496.0391276919705\n110210,2448.405917511004,1581.28984753854,1318.3978071827044,1291.119422341008,1375.0577405831568,1514.386897428908,1646.8926456771674,1625.3067388317272,1446.6120992730398,1292.6761730135945,1473.5738799072938,1459.80209318871\n110276,688.9416713803521,558.3924597145304,407.58729916542717,382.86181569171816,180.0940229571993,180.98062068592117,199.02111081489087,198.82734551875336,174.20265627269785,232.0762314476547,464.49717840401627,458.42654509952365\n110285,30.248870117535485,25.172905638714013,15.857551140903103,15.00623792234696,9.586804224982398,10.201764659176549,11.006135203432365,10.985734710508831,9.656227530872691,9.96396419131446,18.76569930885951,19.0818792324502\n110298,1013.9499216558685,722.2595673681142,415.3189279834071,315.92676776482773,159.90056995715594,189.476590503979,145.44722813227827,153.26858595300854,165.16789170995327,169.7136930374062,476.3314095443638,521.1231486499837\n110306,118.61864267918567,97.45701332917324,66.28662836491448,61.53969064135254,53.70648878402494,57.394583364168476,63.6127448362905,64.23473465972111,54.47130688497182,48.17341738463245,74.99018197303097,76.92878498524051\n110320,6.411346875738037,5.672401899218195,3.8982006042632924,3.274876367907807,1.1364755959529287,1.208248536230305,1.3007442770834134,1.3287681529866338,1.2452063604039112,1.3955921660322033,3.647175726598143,4.07773032166596\n110337,29.7911991810571,26.38346089118859,20.776489248765294,18.33890726130773,13.295635199879307,13.581752619202433,14.804143603311708,15.138513109213415,13.370137941173336,13.79368342965972,20.863226207959944,21.977316745563936\n110341,31.031686141949077,26.11714288149486,16.83513438220797,15.403658731087047,10.15348817183007,11.297281457859778,12.6670118070324,12.618940755046614,10.849103561044007,10.100384406342386,19.795650642634516,20.539795097567566\n110347,1328.6916556919687,1257.2177414950756,1108.3355339099282,1073.7161086534154,1036.5305975685746,1066.9084213190022,1067.8767835744593,1072.1541685278742,1020.6894390498925,1021.7697890632702,1160.5566352272078,1158.2210152063103\n110369,16.798082228668257,12.482774873866529,9.77051630567958,7.396027002759407,2.277820315950735,2.5289338451216037,2.5441451342158103,2.6964558935058074,2.490409884556298,3.5000921958557165,9.869992114023583,10.488154598302211\n110405,9.464003652415236,6.2187066934832815,4.664979432601014,4.001225805437777,3.978677427835204,4.34484188778058,4.469676663807049,4.547644717592964,4.227848342935927,3.8703138965817616,4.779918241915045,5.123404247696186\n110407,94.49027341202316,84.99776070767973,66.0289238019953,56.27840619497072,22.60852385478751,20.93730111897898,22.244688554439897,22.866144871771876,21.042211749774495,34.818587660111895,69.58464041993989,72.35696154964141\n110412,24.593767586950744,20.066039651598306,13.95533424247577,12.59617973969539,6.117042090487706,5.813637340069312,6.883786016212347,6.89901502732453,5.7571304456997066,6.975786017478848,15.667488241238624,16.124409617685803\n110420,72.07681244807961,55.17235194333654,34.988428049603975,28.347424220615387,21.172259281541315,23.843705192473955,24.532218316181865,25.017186277787275,21.428971698907908,18.822417441455357,39.64470038407329,42.915634770072714\n110423,188.7072732302646,166.1082473256905,121.48166709653721,118.07133371092667,122.19356976967941,139.01261497280524,149.24637811098154,149.05393365432266,131.7116979327192,107.47780244147668,141.01278746205472,143.85324476594042\n110430,36.11457292636581,30.717274008884537,20.7149865171262,19.271473964509525,14.081635474139627,14.648846569441313,15.782594081590283,16.22890356422217,14.533637929178337,13.404031167438124,21.095337977494562,23.03522001987093\n110464,118.7617256957827,103.31843111178192,77.42928132861766,71.0281159345561,58.933256047693376,65.77436962055052,67.27785458944187,68.26638473490317,59.75610056276317,56.18172801801104,82.10163474645002,84.69299211249867\n110479,94.65157098616817,73.68585179842944,59.08990196086852,48.498765220275146,34.27618605490376,37.798190854318825,38.31570901429602,37.48353365400307,34.81711002270071,36.683436532350505,65.17672717163624,66.20719015132254\n110482,72.05202643565009,63.025172525320016,41.14684193037313,38.90932576183484,26.649807767055755,26.867183603787478,28.251541807630904,28.536654696680213,25.5298724759223,26.95447427310295,44.634882951171036,46.403904250341675\n110501,18.40860351294464,15.167603479406777,10.590655591560283,9.07577162621167,4.7880429855784,4.9371931347009905,5.230865524174648,5.383188733669514,4.98672961440468,5.553676923926798,10.748297469125273,11.732464803852569\n110518,78.98847127336575,65.46369538405513,47.715412342602995,42.165782825725564,24.225640353428762,25.279211210283616,29.472017531705635,29.277990038115135,24.21083708494105,27.167094109771888,46.92064585429585,54.972364183397055\n110526,29.204534145388216,25.17914131217686,18.66247127057796,16.752119494544537,10.296167136897111,10.89745065508744,12.662510307147816,12.710598929345164,10.811138824400368,10.285406744991683,19.939003813577585,21.198250324451713\n110531,31.96323066887562,26.445546072730735,20.85494842472427,16.107833141545306,10.248740557001165,11.881113641882868,12.008315381402387,12.441581672705851,10.81229879334851,11.230905498298497,23.04580909416973,24.974341561254768\n110544,19.433405351887668,16.511004175390934,13.023353737338649,10.536708233280196,3.726740100268928,3.8079112997923517,4.217353261376555,4.417023121860054,3.9869960267439852,5.226636518168742,11.946672836156868,13.051242324121892\n110572,34.16692624319309,26.62935053388384,21.750609037788315,17.2592537894315,9.286713540362015,10.652599000161976,10.79241297150924,10.806242769741024,9.685427638899014,11.56480545060877,22.750087211155673,22.987664057720554\n110593,57.69013164264826,48.02464104278873,33.63946057851585,32.40556937248161,28.558464428968975,29.45967313177216,32.555930072785095,32.61782785931085,28.255988423144263,26.54772770811684,39.171491108911255,40.03108395315992\n110602,36.15583535392542,28.657732359333625,19.493189080307868,18.333670138106385,12.387440352594629,12.472085602322581,13.861039287664157,13.987390638250822,12.194177493077644,12.569876930310672,21.77730792834237,22.166295657966604\n110607,111.23546319388743,90.35546869338874,60.40734506062655,52.84398219785956,22.260770982792952,22.817022985741055,24.509666885562115,25.172098490444036,21.801118853137286,30.232375684563227,67.91279999611898,68.51684930790535\n110635,569.4325225247393,492.57444430512857,371.04268800058367,329.75981906904366,235.83528844280937,214.60482459840952,175.40347596245687,187.13165136963966,225.05722965258093,250.7596171289064,411.83354471759657,418.8375054294242\n110655,365.31016442138747,303.1615405423161,220.30540667416818,189.44753967377187,102.04948511880826,93.60056580184306,86.62365518204778,89.11874828205353,96.93529820914166,112.07537962466984,239.62138864482048,249.9546103507552\n110661,25.645425167393675,23.304465769032976,16.001706334620373,15.851730792112209,11.81764744004532,11.645922309207602,12.336727580257197,12.588103174677066,11.389321281091977,12.43965164223847,18.06317245024136,18.724989063504754\n110674,427.422908047152,375.8742577019933,275.42502691890894,241.90078554168727,136.3667643241504,127.71393816331143,105.1189130547653,110.05569317348547,127.53082544731234,159.031569920477,312.3508663301547,317.0327410389123\n110700,58.40226322365256,47.12165618005962,29.049001309692787,27.916198073976556,18.457929569802015,20.381635541136642,21.657491125483983,21.373156560632424,18.858554724662213,18.01138921849081,36.187165361410706,35.47271026231585\n110734,37.80064225786659,32.4183924945869,24.334164470230643,22.09531291300492,13.843989240440756,13.17132294864052,13.72695920893214,13.694423099106304,12.472762572239,16.79071526657036,27.857911407386954,28.415681673317742\n110736,1525.4003117232833,1346.0528439885134,957.5995888024161,871.953357141201,640.6836044573264,608.8516083773258,546.2713245651424,575.6047383523409,615.8746846699697,730.0647356611139,1128.611680759199,1160.5453744471736\n110747,70.47073350503959,59.322487607425685,41.59056650183438,35.326525789053264,11.563916694564387,10.976598702656858,12.018009323277418,12.411275290942207,10.772732674419254,18.72680713438353,44.16998829217929,46.30767391133264\n110749,16.202232729831834,14.277005688637097,10.047424602600382,8.930967198802112,3.246548754565825,2.9581235857191706,3.3515200736498274,3.399772327101175,3.122783934714973,4.796656990487154,10.440313138682281,11.256381134882448\n110755,274.4270434559156,232.04276040850013,150.35698229280044,129.51217658518297,86.63908329232763,89.87287779665046,81.03269004493097,79.15890303775794,85.2544002392881,91.87871197646533,194.07998691784826,198.299457074297\n110759,2602.877820086111,2153.6541494183343,1473.7342145159291,1310.7928499066736,745.0180816394945,728.3709925648768,636.0420568025389,654.2817442799002,719.3406446894475,919.276950325758,1704.845894023248,1684.2442080642738\n110760,14.738943041568646,12.752568352078402,9.133940817687927,9.226001747496575,8.31129902464399,8.580813964231554,9.395210954368618,9.58664192675466,8.648202068275522,8.099637240514479,10.28555161799888,10.600055271887182\n110767,275.4132088794049,262.03667517630106,193.33070357080035,168.19237428243275,115.89260735725817,112.7145990590176,99.48984466631285,102.2607944942914,117.89406456190457,112.56567432544657,215.29816257702115,223.4030174744618\n110770,55.99268176546599,43.750195875682,31.565675865606188,23.81521497644125,10.456490665956284,12.305899243782678,12.608156289338407,12.940326642191284,11.26341425429349,13.631431537098322,33.18730242614772,35.225470795470386\n110776,49.54032299414971,40.82614662216459,27.255580824192943,25.15849768648254,14.602432410610449,15.681623692220011,17.182116788295463,17.439229563050564,15.063589237762272,15.478480665186792,31.71184966342139,31.690233983256817\n110786,123.37306015852361,96.87842936111075,78.4562818657128,63.88850687339729,33.91378289082818,34.78254782247621,32.5447600381098,32.715084382081486,35.19671733594321,44.20821807933103,85.13993860883902,87.1950307138702\n110806,308.48205706715896,309.61694740995233,243.52791802304856,218.47290221713678,119.83443905711141,109.78277916411106,94.017437375265,98.965166114089,110.84078750256677,154.34333452650708,265.1721648023484,271.2586493020869\n110811,33.0466081263027,27.574901177563095,20.219897512618505,17.009578494864744,12.689662121305641,14.094099101739877,14.360190846574058,14.631525288595464,13.033161351523596,12.376406977963944,21.59502636946526,23.18360357848889\n110819,25.32643363653677,18.22668603829008,14.920268181465191,11.6860197050885,7.76413480227917,8.150090777168408,8.200898092764758,8.459680682169209,7.911495641096287,8.000854363268196,14.143388350665386,15.406881512734511\n110843,48.021139556906256,38.23477234289354,27.563783180862774,22.63413362678372,12.970270258942127,14.672486375818345,14.832192353025935,15.27080991427144,13.585348310972396,14.266365784392057,29.369391688136663,30.640949804172944\n110855,8.619471599947829,7.542254125657249,5.457135468298443,5.017779345707563,3.238727580077239,3.285360618006219,3.692332446486158,3.8209517674702083,3.33674872766374,3.269653794734268,5.0984362038901985,5.591191327370393\n110858,40.23628276238755,36.593048666476356,31.3866683111192,31.517713781859584,34.66806120194729,35.39033807633947,37.754286024305515,38.603088864038554,34.976806642491674,32.644617866295604,31.281428623094673,30.95251367626498\n110866,19.00487921539529,16.069604549688556,12.569549392227241,10.381461447376855,5.412494093601674,5.846332516448513,6.017652159296008,6.191488042020393,5.652267546732234,6.643756637756781,12.366289079144561,13.453419525726165\n110879,396.03062783959086,355.4285755276713,253.79043263012997,220.11325775215724,135.3945998247786,136.46252322417914,105.55174135717473,105.07073919293246,128.11816279465862,152.23883163583287,287.1714959642343,290.61888400278036\n110896,42.11617060262276,29.23105084374319,23.10194447743454,15.723772476235323,6.915589444853302,7.726645858226839,7.7354327543925505,8.009189163971003,7.115369636343991,8.923375639346657,22.362916273356994,24.010940336344873\n110916,6.875388865431513,5.178207360075857,3.565471020617578,2.689586044160411,0.9403573786176894,1.0890691587270236,1.1392677370231572,1.1913830847087319,1.0744930540981468,1.1967451740086137,3.4825578941169946,3.7754445000607686\n110923,185.46209538248928,124.91538844866649,105.17232302035784,82.62016415035647,60.41959095076126,67.2384896884125,55.942754901008506,54.67847568840696,62.608664538416505,64.09354634582571,113.97828053519126,118.86976207666793\n110926,40.537714121269005,36.14078680116816,27.023094343468856,22.292818819591655,9.485988763227532,10.219976448921512,10.667460886432291,10.496475132424914,9.456899683182204,12.958965529226706,27.83067692696822,28.945412931096484\n110941,160.34927004715027,125.83036678824655,95.06736741857397,85.1870489055063,50.82598129902324,55.46383644511719,56.83862945419325,58.051017908358794,51.489782215400254,57.113011177261406,101.34836853666202,100.40550784057731\n110953,145.83224266723843,123.23200610286597,81.68323713339542,73.13093774011205,30.161317343782045,32.314285284786656,34.645576246834146,34.6821051383898,30.16959886797293,41.875937382995325,91.75919974950567,91.55910225523482\n110960,71.28434878619673,60.3786114614881,42.23441654400512,36.42921689914364,22.805972580048763,24.85051952169241,27.4765873511658,27.84925588093486,24.034878620456464,24.47617361388661,46.74779800195854,48.75154479608901\n110961,33.47341118371243,26.908846635956813,18.618218253735158,15.389064861255623,8.091317568177196,8.67823471851335,9.027366442086212,9.397206344432941,8.505645991689569,9.32611530571442,18.808905908327212,19.877766033588305\n110979,21.204379210598812,15.77650513478261,12.272310876164097,11.582026249603192,12.242328109014561,13.273247326895863,13.940531683093505,13.422731597746099,12.221150247029534,11.299444815110526,13.227769692794128,13.721718954236318\n110987,104.12446555936876,84.42703793728549,56.705654357462656,50.878080976338595,31.899841168473124,33.9602902454765,36.55566376156549,37.27113139736992,32.41482457900185,33.11840747105035,61.703780624798455,62.549192940344845\n111005,170.71851345803603,149.70065053954028,117.1802234393107,112.18189466996026,111.28035017575982,107.92318550705629,88.840667806538,96.91416214949277,108.80105346419293,98.56977674177169,127.89890831753188,126.81130541962533\n111010,44.762833306824874,40.97415807958326,33.335719947166595,32.18188061949593,31.275991216367192,31.948730805245653,33.77279227610324,34.69653911134858,31.67143895015664,29.9861239987521,31.313661929105628,32.531189439762784\n111025,33.617322799392326,28.690566878700082,20.43061595141213,17.620422280485787,13.495270138913707,14.865726231566978,15.30503161524332,15.399296836618072,13.82475887644158,12.852762068848973,22.282769738087378,24.664915015489203\n111032,30.442023447721514,18.75473218469376,10.775967804857508,8.689528825728846,6.144595605482759,7.237842984806934,7.507344698119621,7.340855258938084,6.458958795688754,5.177403158177264,13.97145567888007,13.409147822516665\n111039,253.82772976415546,218.98013926266108,157.35215372186127,136.7980059891574,77.2371156366281,80.13248455789645,89.40925768408717,89.17252665415606,77.48518698307603,95.12190495400556,175.0441830387087,183.2219347904034\n111045,8.427368135382489,5.735614681834072,4.262445784741538,2.9231520984333206,1.5120332534138445,1.6954940114547599,1.7257858839175826,1.8496638498755795,1.7080472566421327,1.6215926465948325,3.960870315583472,4.619529534448009\n111058,31.821634586312992,26.869421205817822,19.674674529392757,18.8160692650901,14.251997180602945,14.264474728810589,15.547515836964294,15.806231441093724,13.992082677989366,14.139056524935437,21.06731761969492,21.655377176157284\n111086,5.957001810157928,5.146784250169852,4.069784643293283,3.591140279601928,1.1334129927066916,0.7706553996830185,0.9400271365232007,0.9903009885997379,0.8850891542879842,1.3183977981637491,3.6442417093865718,4.026491873388206\n111104,13.818522037376292,9.329473383819776,7.6435562156731125,5.579811860919121,4.74570200235105,5.025979947892314,5.2586771928012475,5.259083100035044,4.751673974612278,4.396200550441395,7.695269943274964,8.366768028503298\n111124,1596.2376192144618,1383.9011469421148,1075.1441687259673,986.7012564181905,734.4457296523481,760.8740366189392,728.0218617544988,736.3079413952685,728.9564214332476,854.339517984986,1223.2923077633313,1226.3400349510373\n111141,33.91061631263039,30.55168910674166,21.642859250178653,19.212704336419534,13.676768369036198,14.641112480067171,15.228302643169908,15.285595611991752,13.829136428803768,13.28143618067294,22.7677731799234,24.62409050921347\n111144,29.03915750150027,24.009811445271662,15.83841111328699,14.186332387826903,5.601507692942879,6.119148734986675,6.531115288673398,6.597991042897833,5.927189357632293,7.565608420301102,17.475797879705375,17.22296891573435\n111154,181.8249501914654,152.2561084579604,120.67416929166448,109.17946001431027,112.27341402904842,120.98883730143648,105.29108612484406,104.5666715076031,117.34794185818569,95.46468685357976,132.6089972939274,134.86687765706887\n111173,178.91207767244106,156.74773091052157,106.97207649996543,99.74449354043395,57.308897677034814,51.87894639103791,47.046488134242324,48.634695752189906,55.490503879327164,70.16232430363853,133.3765207253726,135.2821595964676\n111178,90.84481136509672,74.19271685109996,56.653623865222755,47.50537764117684,50.56421994791687,59.38654532612995,58.92363563250559,59.97676036063195,50.40444670059904,42.852727536608874,67.02927527251292,71.59137734309995\n111215,342.72086485993447,263.70543932493433,213.83193400200614,170.4965564806239,142.10962450323532,143.06579655476125,113.04759008494172,116.38106976943635,137.0870527811288,141.42966416089553,235.66785389302538,240.2427618820396\n111216,510.8724908626031,438.8704298262535,334.80994154848474,291.24616290907727,183.10414291614362,187.70503512086754,163.9831310495609,164.01755185802492,186.7160417310689,191.6013823229414,353.6954996009896,363.56215926174974\n111221,110.18612930996333,92.96622295491267,68.8107674999948,60.13145858804237,21.073820745067284,19.0023337275691,20.443434439109406,20.61938365732707,19.140140025181818,34.76814387057084,75.58298970981862,76.03344091609542\n111229,48.91559301478215,39.46349397444578,26.69424123754592,23.62820817527211,10.08055497061838,9.733569517822056,11.535784296318225,11.49963981472157,9.491121148006318,12.344367459701362,30.212675169698944,31.189452278627495\n111240,42.772794723656645,35.36659032992562,26.21351966154886,23.211136466880646,10.199915375410603,10.055211535467944,12.083748620365906,12.177316968695619,9.944828953046361,12.281275776879504,27.906249916396,28.8071200164425\n111246,2176.71882277322,1828.2145398098676,1413.1179230337316,1180.7828074266565,575.9935216946067,566.7142372821827,592.2566544406392,596.766997903227,560.1841028845836,778.5429494880643,1582.0361971031464,1555.073049640642\n111259,21.037292733309112,17.846013527306496,13.821918928931176,12.008833342714347,4.33400238965887,3.3371112703377706,3.6493718948879335,3.8483333055122584,3.4410337154002906,5.299236514850922,11.8657447940807,13.078335089731093\n111275,214.21731608677217,210.74074469078394,154.9417525613781,134.5180951626037,94.69901347647796,92.96631839392451,85.02395813503051,88.69802880285418,98.3094137002393,88.06475947320557,165.04129139663127,175.7697108884472\n111276,85.9655078747677,70.60397403961501,52.01613085047055,44.32216291149297,34.185977610125384,37.05173405290556,37.60743809221109,38.37919929620401,34.0826409109446,33.35259217294227,55.57927710873742,58.79301537464638\n111287,35.6377287830571,31.727991707715702,26.327187826898403,26.68731278667603,28.91765512806925,29.406670504213743,31.209996877615385,31.933568953891754,29.05071389166511,27.620006108942004,28.275187052529756,27.73402305005137\n111294,51.63325836911331,41.52402657860232,33.38191092014025,29.254967337584322,32.53231582882725,37.18495079398905,37.912036222579644,36.30989797552771,33.3895983511722,28.450172952808483,36.84002416991235,37.27543422926576\n111295,1121.1529046435835,926.5126141098774,737.7816082216358,589.4416281120373,289.3073489499402,266.8949242855615,231.48842516457628,242.91745049220896,293.5716512750218,419.9372908738879,794.2171936117861,802.2484243455666\n111316,63.01043581174602,57.085729920127086,49.54222557729125,41.36197525539929,17.779590090768906,16.643942011017266,15.99991388034372,16.81982954673606,16.827903076336344,28.81468202551297,50.44887149743882,51.40660982616186\n111343,25.104916579413672,19.97858426501217,13.275926945496137,11.747353119809386,4.057166761733298,3.5677740129457245,4.431853831895605,4.4295873523941145,3.6327889772239113,5.266088437222229,15.043587765868615,15.417732764606951\n111359,202.90060700979956,182.8990280135886,139.11953894218988,131.42071095411725,112.9336175942414,117.35326343511817,121.36452581847983,122.3320046291753,112.46257057554824,113.60940724341442,151.7909933899201,155.86336114418853\n111366,42.18007813131159,36.28809363708033,27.539765425954464,22.110806182789187,9.42167099477015,10.490912493155014,10.67757403706024,10.679633403511058,9.398479395821752,11.681133033408791,26.74164834373079,26.98693592623936\n111367,15.669234530893519,13.619963533518856,9.703082057354369,8.58408012822977,4.125059448822023,3.9917197126687944,4.34369825271169,4.478670872559945,3.9851216213332497,4.852858523860437,9.834773190005889,10.655228300765474\n111388,18.689796202450687,16.11116880270686,11.862855842546281,10.0509766844928,3.6029494751116387,3.6094001455013704,3.992826929685615,4.080035248802984,3.6263464568375934,5.0495115776366895,11.947312097171471,12.776559759530677\n111396,85.85780813456485,70.57345688842257,48.19151221849806,40.877259647270016,16.405326172520926,17.368708683611636,19.496357361678857,19.615119007933853,16.822115793279266,22.183084278046017,52.68749613261213,54.427529931884905\n111405,37.1258312021978,30.222513820924217,20.260462560477926,19.20665571988997,11.760336926426607,11.424313135310214,13.208942632772862,13.225879109307764,11.135150815832798,12.022531035356021,23.408877834750218,24.45643302694858\n111410,286.677737875715,234.9961515442569,185.47636234428674,177.13565876430437,195.59992836800416,209.61698112672144,208.5981985601955,211.94486333788313,195.8975670054963,165.06985044414736,200.4448228795127,209.1388179281612\n111421,34.855213425774295,29.1393160436472,17.662196639546927,17.122319943132883,13.00298974355738,14.376911629861915,15.235627522014985,15.21836584929649,13.408390009718662,12.159791110085962,21.34187406331774,21.765413402487056\n111433,71.25103040377083,56.635036631406784,31.156533190260213,29.44437456056083,23.471802761457752,26.18099481833541,29.424365505174258,29.436412970588783,24.3764791712428,21.79774782811676,37.530988182373726,38.17362272912606\n111435,42.100237805626854,32.455825499272954,24.557259835423853,24.434300916757127,30.42296707776421,32.03143500762792,32.31233369035923,32.25557938629161,29.404854436454762,26.30996466922779,27.31424516035789,26.891744542871805\n111441,115.76395462341775,99.06101210309319,68.32350441988231,60.031564015331845,24.67760901815347,25.680981653782545,27.291623270949017,27.486432943193254,24.53655947452555,35.82676847498821,76.11910257369995,76.9724482644782\n111442,251.674458088101,185.21550685955359,141.61442660510093,113.84974838465708,97.32764689949684,102.44434228301245,94.01902778677622,91.38325797105136,97.52906286671178,92.31343511440839,163.92330266865,166.8007684031298\n111463,4.868301074404901,3.88763719207221,3.2182675864496906,2.6756638942744178,1.570197893900968,1.6125184164037552,1.6418501277750208,1.7461546836211672,1.6206941552775573,1.73894288578149,3.0165992356865794,3.328228919459605\n111473,7.768393383162369,6.625697773468888,4.957131825894779,4.5109281089163416,1.834315395706885,1.5670184809340109,1.851833974439795,1.8688462886440556,1.6502854726035214,2.0948204965055597,5.251468518108454,5.572308690692834\n111482,29.764467147308178,25.154544978824042,18.255743517308744,15.993070102280708,9.854994344704682,10.148836611723185,11.823203284758868,11.72682856777219,9.840296598992925,10.777883509402274,20.554484407095295,21.35454840567235\n111497,664.3807961872391,557.4655952448554,410.79987785912346,364.34797062426,245.43441436596694,237.41817371194728,231.2110225302751,241.3611574178992,245.0746606689933,252.01099179381757,442.33853020158114,453.06096368114265\n111498,36.44359486046962,32.331188460137994,25.074758661388884,21.5395552966843,8.672447097060012,6.29443119434754,7.586900758825474,7.8613233156139595,6.5821788575083255,10.920661113108844,24.059349618884614,26.02208783291592\n111500,11.836160236365863,9.987886258725977,7.03742586406652,6.6128495992286025,4.497928175055441,4.658374809096277,5.1532402280138845,5.205521087100148,4.681227986443226,4.413503135968661,7.44590641818871,7.875232005633619\n111511,22.960284758794234,18.38861958457289,11.924335086711048,11.002477431426552,7.416606626072473,7.839363430619127,8.264968703105268,8.637330568031466,7.678212366399854,7.247046693158517,12.77523923900946,13.523310794894186\n111524,346.01127686450207,288.5164823317688,236.9291843126741,190.2330465911876,138.6467523924335,147.63967336944998,150.63430564514704,138.90567515324096,143.01595025247752,162.19084256892148,276.31830175919424,276.4772963090723\n111529,36.45028896457534,30.692171154197283,22.184890085344662,19.423972012394398,9.238676066033381,8.733734003747713,10.175415487459048,10.285901093455402,8.815355048754279,10.39089631375907,23.17018420053594,24.743374735244487\n111541,1205.3881843851902,1053.0911235888352,815.8260396946121,718.3955980370532,462.96717100862406,457.1246447603705,380.7554866595817,390.54215633373474,460.9935444100645,481.6861193382711,858.32202414547,881.7445953883824\n111542,242.82898597418583,177.3747960100276,141.01157940097121,112.80109690712796,85.01225901109129,89.07707371644453,78.9057453581985,72.1609672076552,83.16780857464511,94.59989412253192,161.201044039785,164.14543827709426\n111548,462.1085270608114,415.1948532667606,316.55337759297424,283.4847057042977,216.88086264955274,227.09967654157015,284.72077642527603,290.9468300993187,226.72098210921257,211.76154811041653,352.5021431133635,367.75165624762514\n111587,1008.7820000975452,917.474508857132,759.0437160724583,728.3801531576613,770.9446006830394,829.1793748285143,870.9835735211972,869.9383408606906,803.208210840602,687.7869212687451,797.3822911668947,814.5703505078407\n111591,163.717185623155,140.93114246031826,110.89788494338717,99.28338607022168,84.24132746693684,89.07917273342385,87.94816688437655,89.42343399348708,86.12055938845987,86.51164135098976,116.33276017214976,121.73949053197732\n111598,11.85070737154754,7.763371416865544,6.2605821364886625,4.459532421250555,2.7058389449761275,3.3182644737405464,3.4900686648665022,3.430187308263021,3.039867444902398,3.1744313144953935,6.477415823284641,6.910555072855131\n111605,717.5842719643153,629.6801213279424,516.3676560067203,471.2859586815449,214.23777374944686,181.20881115135643,165.88047823584023,175.34167970515702,175.35848378604936,332.23779562705255,551.1438622301181,551.4601526077815\n111617,23.82359382858886,19.06217899509356,13.336597095936442,11.921783772900428,4.0687562937501776,3.4962446228176614,4.086994968092622,4.211917106061382,3.5859422686932128,5.400946051951066,14.036123288698958,14.445544825381335\n111618,54.51339981773734,44.4572007429747,30.437953495077753,27.18583225507827,6.477341318024307,6.8933821786300555,7.599975994468102,7.647620904697747,6.088580784384433,13.177675329457692,34.64096729403688,32.47149713890326\n111622,17.47071594262846,13.186937702355836,10.414630875795254,8.54756113955773,7.023962800688487,7.9612948299159285,8.130462493297976,8.069021886326738,7.334206192240525,6.73644195408393,11.067252196591632,11.198791522888177\n111629,187.17194093393047,161.87356358738253,123.15790277374403,107.9563507978032,72.24587016514198,65.36087060747181,53.1075866778296,56.75797708414836,65.33361590587855,77.49124099822515,129.2091882499582,135.9542068187486\n111634,266.16741819721193,206.1546585118868,171.99490489882118,156.19297729232392,187.18500471241796,191.27790866379658,154.09205226028803,149.33038223648796,184.29443697587365,171.9129963848151,200.83137016719402,190.54927933699284\n111640,547.6799287522664,508.53340146458754,413.1656671762755,386.02114229889025,317.7804971439671,322.034343870899,309.00035243461866,315.54011837350146,322.11392056276924,346.07330421834035,438.5292776338318,452.47833378601433\n111642,53.74860452519608,36.06454979324337,26.814268481933347,18.685536668559564,8.926318017435042,9.64426250278036,9.705750538548239,10.072846632705463,9.151682766250799,10.481939514570957,26.72894336422319,29.90449221127869\n111644,23.883646106357578,15.963149597568876,11.226899342825213,8.285645376296644,6.635253066502868,8.09568688366676,8.4329683884233,7.393993629912173,6.53712424560318,6.863423174323597,13.742800977914648,14.44375708756571\n111649,192.73250084936092,169.0587345961409,120.54520387637241,105.59029130821082,78.5137796198966,89.33285988409632,98.70820343474116,99.08167946553415,82.97748575541891,78.63535705882263,130.41856475809743,141.16907344416651\n111652,20.412522717458955,17.407344874695077,11.073139674528798,10.465548038348226,4.127848374280073,4.286874677487791,4.738791652199742,4.802648003198658,4.280764386955498,5.530797552611388,13.229600724072583,13.616578643224821\n111658,17.983802223665535,14.62044391893378,10.017124954621208,9.23298287369505,5.587273469061587,5.67907280437981,6.172077958565152,6.427068380190203,5.680144249764949,5.504004575651515,9.655552416620639,10.366890534562492\n111666,288.3074561804087,257.46877966747877,206.0276115102527,204.0415100969188,211.59454773111796,227.48224276133516,234.80530118611009,239.6367545995405,213.43781481043868,194.00844002109184,220.35329772123535,221.78444629861025\n111667,62.820851405633135,51.49475137180122,36.7070976884213,33.417089368042895,22.847500725490484,23.287473729506196,25.997997070784905,26.318735095018774,22.65171445179746,23.42379855852588,40.23939567941082,41.080972409468316\n111679,40.42403585292157,33.79360679398471,23.705294886874256,23.391511550039148,24.692321048525006,27.368422194670995,32.53517335467809,32.07296086959639,27.077669054322932,20.7380407511878,27.32723669118853,28.81100739426016\n111688,748.1995654929243,641.1273120974965,486.72718182050767,416.9905675839764,265.4691281573721,241.65029429919633,221.88616130575193,229.02685193039133,255.3649689394582,287.6541923815084,546.0971281929211,558.1242358056811\n111702,107.68991254822078,85.91110153866099,69.0412702215969,54.38668295548348,29.835852567267658,32.55596870529702,32.203417865941496,33.2381294547769,31.03190683739235,38.94395331741764,74.2122038806143,77.65468842028193\n111705,16.400017125629773,14.558225964171523,11.60356159994176,9.881121502237868,3.7887865620154275,3.0222981173088517,3.535267188544011,3.710704022911856,3.2048575798891905,4.263111511093477,10.062583229608832,11.15039262534847\n111710,41.16325310108722,36.09409677140245,27.243268057344242,25.453878860689162,22.37048133086481,23.34799035374175,26.20445284980827,26.236479085140115,23.070689125914797,20.22022984706387,28.277667658601292,30.504759160729478\n111715,437.36525118750905,378.3515735253645,279.0941289706035,244.35686302754593,160.15443839244432,157.11215885391084,145.56424437845106,147.59914744976507,156.15510180634544,182.04771603056977,320.8787095572234,326.2606146580423\n111717,191.96646617970234,153.62335697973552,114.67469683577416,89.2291138951309,51.6042624805968,56.697676027217675,56.99809940491882,57.83624963708954,52.08265173012141,60.607017971317774,121.02869524650957,128.59956692662112\n111729,9.60195859938286,7.911058409928142,6.081580670413454,5.566871621477792,4.798210808533457,5.035467775468101,5.148876614447517,5.342925288820911,4.938161987345122,4.664889567131106,5.761294252927859,5.977893870374547\n111747,72.89446740558635,60.72487348342473,43.073090847024105,39.07931685019867,26.68561637006057,28.995269556291902,33.4240924589661,33.913269389054335,28.65383413194167,25.819717659604493,43.294231431360366,46.05897479385787\n111751,132.89041003048922,116.83704834177513,90.4012288092185,83.09172957045172,69.59441121328437,73.05635360942348,78.75511606746238,79.41957522187765,71.5921234829741,68.0355966811384,97.69879309666898,101.94193967912848\n111752,393.2357161251791,327.763071975827,253.04833938972132,211.4910956931731,143.24151568923898,150.74009695063478,114.07636740125946,120.2099003845731,147.03557407615492,174.1140776565982,277.31927527873205,284.59574568783836\n111759,132.46265187789297,117.74784357526494,100.65182883176993,89.10108647318704,45.005274830604534,40.499899782962295,33.800858514638115,35.39651518628041,40.2666933632288,63.517283559337564,101.27094874484581,103.09968864209122\n111776,1775.4869591777772,1508.1807341711417,978.9377045563419,960.3010062834766,1190.2479120892194,1341.189807578784,1503.1305109653397,1329.9864716007378,1254.9339468395842,960.5980514199282,1210.0842647744466,1225.6398883033592\n111785,504.14083110269644,432.92779124917286,297.8328041906526,252.19165483826828,136.12266059197393,138.4330335995175,126.54912154225784,127.95926221450053,135.41980466664359,183.65884729721128,355.29141993709635,353.6038655133943\n111786,429.5639641154928,380.06920279303256,288.4687644868224,243.33713386404372,162.81081938773934,149.1357098470791,143.17932612880887,147.68257290732353,161.30858977599058,165.7483119239535,307.8685183239932,326.0352835718801\n111803,1487.3912900879982,1320.0506824487034,1080.6744445609354,1053.6564217705297,958.2387699312781,992.4294567995131,1083.8980156161176,1087.843086359932,969.0756617468746,926.166756579413,1174.7778611377973,1188.87654440796\n111814,12.81935083807136,10.332474920030398,8.373710416663108,7.28083790000137,5.793037167373362,6.264203075059758,6.351677248500724,6.367662230384697,5.883732511732982,5.599897210004575,8.783144421397017,8.834574948754637\n111830,17.88720340633048,15.239830422238516,11.875910318588081,10.463238417881671,4.752987735418329,4.224927667866581,4.840121279517821,4.892361434349588,4.2584238957720695,5.603143247101515,12.036314643862468,12.645435649222387\n111832,30.195712043907175,28.61537758791397,22.99145352494688,20.45725805197032,10.329368273354223,9.277418778950874,8.834923940242922,9.118882192948018,10.040654813923211,16.59781653683062,25.48898865368659,25.745647088189127\n111833,465.155643252577,462.33793239548805,353.9682446380597,298.17091186083894,177.16890994849982,179.96052568740413,152.56860601062283,157.11418904260793,180.22220466413074,221.16206607447953,388.37511307476063,392.7624253753044\n111835,32.05261360858134,27.920390094070626,18.359409244191497,16.86656892064557,12.812726929438975,14.040582152604483,14.638262778093338,14.349706969314209,12.466843616519387,11.634403236726623,20.446901108654913,21.26337417479241\n111851,126.274394573202,107.73379861167881,81.4772394593565,72.25827217306679,36.00322066974193,36.23152790872153,40.66873009069745,40.91385848685011,36.32955068095893,42.704965518081,86.26557865806775,90.26719771285123\n111855,44.235739148324235,32.90995224269359,25.261353430637918,20.22919167004379,12.570741713967578,13.90424938996834,14.051517671280394,13.760701338737864,12.616892965601364,12.925396410318502,28.42828853223791,27.873112652299568\n111864,27.261662434512747,19.952059186526142,15.255038519237404,13.993676004942934,11.639839936403673,12.135313402216147,12.112171563239514,12.442000743871214,11.269071705386668,11.530446285742162,17.074523234933544,17.17922903142693\n111873,22.929337128605074,19.13284310672608,14.272102627811522,13.381659330279208,11.30707113747324,12.08713568943179,13.277407506760996,13.599004912094312,12.04669740600747,10.62232642373065,14.047265015710828,14.616669063716504\n111875,125.66023522267864,100.94309146630724,87.08854357066114,90.81587024543147,118.98199035676967,125.95503427239609,126.06552094103331,126.56628868592485,116.20529304705867,97.66231247967374,91.8068491498896,93.24559860122348\n111896,216.61530066308003,189.9884961048934,135.25276826990654,121.75530116438314,75.97690112004143,72.17887518106761,60.62382768646722,62.644308078671074,71.91743892644436,88.94762566177644,159.20437781697657,160.37450673646805\n111898,86.19838298939295,68.51859254151344,43.85500515818474,38.455260319386184,20.55019409390065,21.656446242399884,23.36794419833869,23.848998052129442,20.63279186357249,21.626597270269894,47.346407046547576,50.74414018685382\n111900,71.84114251382958,60.118674551474875,42.47673397933728,40.356091160210404,33.53680799570681,35.78276197342792,37.467216900169404,37.88639105131573,34.18810203455831,31.448279828697157,47.91442851773462,49.55153278653654\n111901,230.42692223470078,202.45230673566445,163.52602223320534,148.74776230862125,106.98350325984765,107.48747281104217,118.63690045520082,120.7999360323297,106.17915572106186,115.45967255720244,173.95251836131337,179.98097912617274\n111924,370.7496290843005,334.52742900737746,250.29867114990924,234.0477995619997,210.51333731945115,222.29291849093028,259.7565224183618,260.8394597607808,220.27277657818064,198.72319243374568,274.55579712336,285.7791604110403\n111925,24.289073845817096,17.10971762243501,13.801194996618149,11.0247360555988,8.44385676564537,9.627416909068616,10.326580016824357,9.85593990201372,8.995723143238244,8.831931239756985,14.469238229811934,15.15926603175483\n111926,44.979987316649044,39.79994192274851,29.67573138253903,27.065374603737236,17.678656954942856,17.807274428487055,19.04798443168979,19.403502453794047,17.16592729131679,19.109370179037835,31.996423994463065,33.66123514666829\n111927,155.53262654994006,92.99506794202817,66.28834042794624,46.6062905045444,34.92789392218037,42.73138095857424,42.70173263394261,40.442145666752,34.00784761790193,36.13489089998911,85.93809346235719,90.59723275732743\n111942,24.44645217148935,20.268544779878923,13.828657027002405,12.811429157073086,7.7824903966245005,7.8884710095671595,8.897889078893664,8.996649155533255,7.823834981037809,8.23448097883614,15.054761517629926,15.974975683526134\n111943,408.1066747466134,309.24513756934664,252.69010774460932,208.30119498920658,119.7986533147841,130.38577176786526,122.74318201624737,122.8306826277309,125.72410789738642,154.34949155445344,292.8193269575413,297.52226180866217\n111957,39.75607894686701,31.556102790367078,21.21056074076644,17.36101134160018,11.325473865380848,12.665132950027344,13.432948770506384,13.698891507493348,12.018257566586492,10.908598071157996,23.035632501777403,24.266669608105293\n111966,340.2889055175202,276.92047170415253,222.88768417037767,182.57416254082418,79.30849425928533,72.84570888949042,63.49226458429915,65.09156468315423,75.52327225727127,123.04695633773518,245.4764328319839,253.98973004208926\n111995,42.23514873371077,32.42690924173255,22.136974527307714,19.59174872298597,5.313735550788537,4.408229369045237,5.196582965043353,5.3574750446244295,4.408534229540479,8.246806471546746,22.666985653733757,22.827862085544716\n111996,55.69033756264055,44.34150949974325,32.282096149817264,28.35047123976865,8.784387077041812,7.341812040314332,8.692406482339466,8.795281355388093,7.347742205530941,13.652991237456048,34.1165129181089,34.20235239534324\n111998,11.935115703555999,9.595764121654435,6.424372834254634,5.991540252379443,4.46279898262512,5.2275207688072225,6.403389005334433,6.524322243357602,5.467739977946978,4.00771346551808,6.402771155101619,6.934727462584922\n111999,28.022757177946666,22.322919809823823,14.284907539229648,13.536631205928625,10.509759316925736,11.928357768712372,13.305974347755221,13.243305373503308,11.142674093277284,9.603020784911292,17.416300338631135,17.66457118235519\n112011,33.943957115803684,28.30432716975246,20.28303481899408,18.420520044371,7.2217851320512025,6.647864780658279,7.245954892415047,7.320255303986881,6.633423561413903,10.327393758743131,22.723967698197832,22.539046406690492\n112012,26.265314425735617,22.64520304042371,17.808975927519313,16.36886729501561,11.998022494603442,12.778473433320015,14.180527533657596,14.39527717293906,12.759986111907372,11.763018482698584,17.896849833235244,18.76506377237998\n112022,29.52423545671553,23.128723784201927,15.758092119489461,14.276868897944132,7.766557673423256,7.98805556549911,8.802133695383526,9.07069866873908,7.987990696581032,8.470546652084002,16.4833141306523,17.039058302399084\n112028,12.178152764892692,9.309159686939315,6.994341380290416,5.423687188096689,2.381010658279192,2.599479024089822,2.647133968725795,2.801719707253017,2.4909138104265542,3.156917274116105,7.369812618882187,8.208129533764907\n112048,21.264915218527992,17.504502265500044,11.522371039705149,10.46861371371146,5.249101671426438,5.343604691161572,5.812635743285526,5.969432146852323,5.279615365533866,6.328614294008481,12.658145306745991,13.06241487385565\n112073,1553.8883976077866,1234.9694645714285,1009.8880625033186,840.1391872773094,528.7229508216593,536.8198414248521,425.77512406538847,441.2739536376384,531.4057154364331,637.8502941139313,1060.9966410515992,1117.6010540399627\n112090,1026.3874480216384,882.2152253374923,640.5587285405321,601.9788292371499,640.5725768581593,701.3210894901242,807.1182052191458,790.6175689157701,654.3881239099869,546.5571534756044,765.6340163675706,776.875007886766\n112092,228.5164775600802,193.6570837386885,158.56096569246984,150.29516713038856,145.46298305460144,152.47024478939022,157.07999350144848,158.7324692956443,145.63716485675155,140.87807050550632,172.07158230949338,176.17273002087168\n112093,159.848245316772,147.45994090564832,109.75249993487158,94.69685796966418,58.65454219737302,60.86979091090567,57.099180907834175,56.412465786822715,58.68735138756139,68.12360989123164,123.27204515681335,125.34594616792543\n112104,16.802982189036076,14.19901946403912,9.953242759617071,9.32554651795633,7.708498063213716,8.237722698343937,9.443125523765167,9.396823592433671,8.175246247231204,7.162066054467155,11.13880305333969,11.783733520034595\n112107,5.42808353343126,4.523051255774838,3.2496384913231138,3.056232842402455,2.220489193932489,2.3705743199643545,2.6664155728536008,2.7142839526329303,2.427262505267546,2.0948054372950096,3.346671670968859,3.559038792559629\n112111,18.568828213095653,14.014062766175467,10.954481671979485,8.197028082417596,4.771957401448655,5.571097189026009,5.674470196361965,5.543478999699582,5.034418425681252,5.226235111244628,11.882366143749291,12.104769561831842\n112122,62.29430689825382,49.382625539940385,32.344325001934074,28.035878214764796,13.844642641576698,14.631288281779957,15.819686908090173,16.052428616737103,13.99902919848938,15.65347886286332,35.77152029830237,37.61574273897665\n112130,7.489138097529481,6.38552104425567,4.815785632284263,4.271408470904926,1.8296601508081876,1.516601914794425,1.8091300518202045,1.852330772066932,1.632579548691879,2.009374581029181,4.6694387023144435,5.143792300759123\n112135,154.7517260950975,140.8613404320248,113.5976279124503,103.35393099103955,67.78009603229957,67.14507739926104,60.93559724950727,64.20763125381978,66.74232607532518,83.81761672862334,121.80873717569051,122.30002599348207\n112141,39.083782980024324,35.472592795004815,30.27865305478198,27.290429268658436,16.944600663795338,15.595806298143724,14.569175133308232,14.73704990307233,15.084756896142526,18.813378547472773,31.270058164738145,31.935007167165285\n112146,69.21599565902342,58.02756913033607,43.781688451996224,37.99575301040917,15.28202739304735,15.470019080783112,17.864362002273495,17.884995142786657,15.397525613030224,19.56653384994415,45.6482951943368,47.95498756148837\n112159,29.024125645653736,24.113795846893975,15.839504911952094,14.738867825938817,12.157393602142502,12.951635925396353,14.032607788393195,14.168786344799615,12.63513995599813,11.476704777980647,18.228190558459815,19.422490434517247\n112189,74.26812532056891,59.14406790114587,42.2831535554137,39.529854107908776,16.536187072565376,15.559103382749209,18.028914844794354,18.232901459862443,15.417345921827149,20.18076379729034,47.86719202620187,47.305379114675524\n112193,39.15594244788585,33.1561033942476,21.902385181186467,20.686248005784712,16.094172527222415,17.48695469048266,18.910854694058198,18.937661381514097,16.56468965739558,15.050811851263589,26.195216156449607,27.28718180555431\n112217,16.488073504325147,13.976767799955715,10.060686291095818,9.205731036666432,4.931758180850563,5.038300860373986,5.404986118372726,5.573372326310985,5.022169075776106,5.607502718865114,10.258544366707714,10.60281114861906\n112219,26.564958345188742,21.500673335433856,16.862040729743406,13.154848599083584,11.843856307595185,13.460831330683838,13.84271798627395,13.669152739260282,12.303982379694999,10.40222690407464,17.722849929801484,19.001912347619403\n112244,25.553693250476698,21.59592421781328,15.317473522473723,13.450855677332507,8.400501870612622,8.937870156888499,9.613521298059043,9.881043408442792,8.71934490653122,9.008295040105907,16.57625089520987,17.07324509149545\n112253,242.6346157341946,206.8082874968811,150.85996195507056,140.25348513016638,101.8613243459818,107.01410223511849,121.85439808934835,122.49239589729434,104.18092000729442,102.77857318246937,169.13728717225916,174.85584166359905\n112261,311.8135420120962,291.40745781317446,238.9511228024605,200.41089767163498,92.55809565569461,86.97364787770407,77.50530407426064,77.8249204661501,85.47064058822944,138.00215467818813,259.2346669035812,261.9870564843289\n112263,197.11372917060052,174.9990961453393,138.590050503104,124.51566064818748,95.43136729234706,101.51872999048406,113.77101509592504,114.00288944885973,98.12081516104028,91.05917472230068,151.4087161863436,158.36656836934637\n112265,311.88134406764567,277.4756601068493,229.03402909960042,209.03069627590585,172.995514185243,159.6160895128271,133.617080525966,142.84076531319315,165.76112726168319,164.73772261906853,237.22241365022273,240.45952241522247\n112272,120.5488330274285,97.37994333369244,62.77106175712829,54.51246313587566,22.808273312271716,23.7580700349577,26.970313713355637,27.0894199831517,22.228285529348504,30.241033618797022,73.90204735207675,73.85814581542398\n112289,235.6675109543034,206.33674952383876,168.18595863599765,166.2949005777247,181.75129724687196,195.61453475916508,219.06923469547652,218.4925051977257,190.71784500984424,164.4190992852442,178.2320763728259,180.37873465176443\n112293,106.79190825147927,80.4996947572565,62.864713553052084,50.5839982772907,35.21176142222693,38.33736385919892,35.647174249549266,35.209273514632905,35.378704645323204,37.055278106712485,70.89783611715457,71.1871153041246\n112298,111.2633311600629,97.71128782647293,73.32431266493668,64.40285945100408,35.071400016455776,33.88238705607575,38.9913837884412,38.796842349338434,33.554873533921686,39.42608210227947,78.715148351939,83.60974773949546\n112299,103.10451091193771,82.54743935022293,69.04556533038203,56.36355229431411,33.33323217012656,36.71647578059596,37.321401288894485,36.265674755795764,33.873416702473186,40.684499958891465,74.09183249222775,74.25856428039138\n112305,205.6394943747389,164.42727918831366,146.83077676960337,133.2697635468549,95.90520648490518,84.39912333050484,69.32273922465502,73.96710179537533,90.50480014284793,102.6934839304943,158.45260239716157,158.21035231221552\n112330,112.17074344134386,99.62054498244942,72.3641844801154,66.4244591890542,31.31445500682555,31.393118873152762,32.725943502731525,33.39284143547943,29.990209272096724,39.681989870273185,77.83627538328682,76.2614174073201\n112334,475.3711564266358,412.9025448796695,343.88993410519714,273.50486986868697,136.58606463463823,152.37490481265155,130.1616455165951,131.199970705937,142.79763957386558,187.15147157007928,366.9250364451936,377.1912804131643\n112347,61.601881408131746,56.50203565407862,44.217489224615484,38.390855992997324,20.72615836046649,19.205003790765105,22.106234320671255,22.277272100746266,19.47177758184539,24.61296965667398,46.34368659053169,49.56642344021427\n112352,91.95093962415449,84.283017509876,66.95769696223643,62.09210382391987,52.553394743562116,55.47098943525296,66.82088500668837,68.54911822636366,58.807382145836975,49.532905662740056,70.12287067563041,73.42878681092458\n112366,18.223143659976728,15.476828323681332,9.04087763470578,8.930709024485148,5.838546120185241,6.069981154610259,6.667575458080923,6.396371395540908,5.645187785493911,5.834415741635371,12.816509118570456,13.325174925476029\n112370,84.66734455663891,69.09211296567632,59.94224296884061,51.25155397734249,45.2897502952839,49.02357561195604,51.04707111413243,48.66057760250349,47.02671742929528,49.20559261764809,67.59701073048775,68.148049370605\n112378,25.630639850699108,20.67686049842452,14.691096562329989,13.58728772969996,5.613269702037779,4.986609617470513,5.535813016100773,5.667630594581215,5.009495431823529,6.884978022891555,15.992203453924766,16.220765885019393\n112390,14.67971250543805,12.181577474411093,8.770916287000308,7.659774003457923,3.8169450568896695,3.703240951689084,4.208586648211805,4.339672532225877,3.7871801083026337,4.023046575733025,8.694456180634118,9.422505437061945\n112412,580.2958622180528,423.31351134452734,325.5325590383829,268.0149611636576,179.8473708627464,201.42015520545542,166.61340503168827,154.3524829070467,186.289493780887,212.63297099295548,396.3469241003458,403.0739091296132\n112419,20.91305730030305,16.421359071854116,12.382368503201683,10.567185249422023,7.274628635414427,7.999031345174569,8.108074515583594,7.7917431600920635,7.18775012221061,7.8126269195761004,14.340940179891742,14.266119839246398\n112420,2618.00114398354,2145.6956209571736,1444.6280673750773,1403.7409294048016,1225.4623212077715,1339.2250210035281,1512.6615079569715,1519.8927902258772,1256.4091497960528,1103.5270199512008,1713.8446688942336,1691.1374980390444\n112426,25.941743698657984,22.42106245385669,17.13579667381153,14.824534248175592,7.271170982419144,6.331881554188813,7.566625201538013,7.648801949565665,6.438646303496257,8.127546648758369,17.03143994700185,18.089774813421396\n112427,251.2184065918113,195.79841519589687,147.31618623214456,118.44616832613175,82.41147314291787,94.69659328782366,80.84344839274344,80.05119836117807,86.58705950050388,88.87391179340675,177.31962484473326,178.50550307790962\n112431,245.98599675545236,203.09949356699738,136.74990563328677,130.2777319805967,121.18336635964052,133.57984421325088,142.9535786449142,143.92724508775888,122.42521900119581,106.8758689599517,160.89816277579766,163.68220041477932\n112456,383.2320301991288,322.05784004288387,224.3933389853,193.11857118718004,96.7333224847612,88.36635921120282,76.40100938868694,77.58956326649489,86.67598625274196,125.12768781337822,260.78422552368517,270.0446695094062\n112463,8.434519774533486,6.432389262344465,5.063237411119911,3.7082046637748545,1.5470907012636448,1.9880737609833947,2.145420803824298,2.146912390905853,1.9498260471173654,1.92595160538012,5.0185546250928645,5.34700327651327\n112470,48.06735346769377,42.848217238574016,34.48757201679958,34.91814362323426,38.82605416527801,39.57541005801969,43.46794121952821,43.47670823427504,38.041233697037036,34.37733789191504,36.768293772285745,36.891508374228756\n112483,97.95269185610898,83.87323472260525,67.903319648038,63.712911163008904,58.18767267449422,60.33070006247414,65.3765672528646,65.79738042720639,59.070932153422284,58.74038222036333,72.57003054261946,71.38653431417102\n112491,16.16837928147751,14.016496607479336,9.643763339146284,8.701097580917253,4.941298608103127,5.114602243723599,5.475675550861302,5.56121077934465,5.04672252896648,5.352698303577194,9.961891439355808,10.510583966100441\n112498,38.387400739872746,32.72935615268627,20.55937057582299,18.41151403733137,13.544813594597134,14.944291774611527,15.442699227349108,15.260619852502614,13.218135953514153,11.647335288752155,22.763163480766448,24.030717046442298\n112505,36.099436361546154,31.579504992635663,22.692778796647648,18.824175522091103,8.937519805623966,8.9170199187556,10.816844584926317,10.846882171175658,9.236154718991571,10.43344809465446,23.246011348965002,25.882347551828918\n112508,56.180714707089955,46.223604844183534,30.59916670646171,26.12238607372633,8.801426083037262,9.001205558348097,9.974001361464662,10.209083646260256,9.014958859723608,12.158985642635454,31.80176687529774,34.108575152847315\n112547,72.67728826702121,59.19887910259716,36.76767733749676,34.714889088913715,17.273027449737352,18.917191186676916,20.851715057783956,21.153529351052786,17.81555109100373,20.36375039885398,41.327060720620274,40.25580388930681\n112570,276.4108728211088,233.0266593682241,188.66775681076314,155.70860569271662,99.17659067699958,104.8205372958855,93.81614411985316,93.30772556729961,99.83167073390972,103.77103459163696,209.76340685459215,213.69902398793437\n112571,50.00905403699034,44.04042403327949,33.815723361082995,31.18001434436398,26.85414572481613,26.304182210494854,26.875274821370827,27.294097764041275,28.402844630524072,25.60382444890502,36.02360898107832,36.61690948574917\n112585,64.14238136402201,51.17547978890042,32.980651966919915,29.931683769424186,19.850770001233123,21.578246145435674,24.498532670275008,24.61208503606843,20.896874631996432,18.58283965664105,37.58292853989103,39.92525107488341\n112613,2123.1968908428453,1953.4509876117788,1714.2645746354256,1953.1085842854902,1568.345460064729,1674.2876932487973,1767.495894762489,1774.7627126651832,1618.5507488421342,1543.2199524981509,1815.910790615133,1808.2731035977977\n112614,29.205449645059474,23.04253783023423,17.23188234361734,15.145221112438135,6.128336315697667,6.30542886999567,6.689274757483794,6.85525776745877,6.144087790958337,8.008865424095584,17.828621374242946,18.08582106662863\n112616,24.055882085360956,19.629453442520045,14.753124835247549,12.622012655481052,4.5043987507757,4.4854766335382275,4.903217862354219,4.98706362507046,4.480527421587621,6.586279621691152,15.122570829864525,15.719359264175617\n112623,49.96819352008863,42.403231813079096,30.752035744622052,26.346471539842508,17.611523526241797,18.97299186028992,20.110719285443615,20.438807832285587,18.048402861609738,18.356507464051575,32.8846056165714,35.057631130309815\n112624,950.841537760148,818.1835626664542,592.4920663340482,524.4560871459988,425.3361109224853,427.3265457305459,367.95468799058796,379.3408901470876,426.59277168931254,439.2198290741585,693.7197824671537,708.1617529466404\n112625,40.7221512656981,32.8791714311801,26.403448708537017,19.86502668369622,6.615046855892869,7.124374311769294,7.09317182002765,7.40363820202154,6.769749489239054,11.148825318760041,25.35344002418718,26.785254516781944\n112629,42.50053029527325,31.790462286240942,23.25385006093954,19.310642879445286,14.398882784680728,16.069593224951415,16.15120154380911,15.691358537650661,14.138049361836751,14.4027718108869,27.875876861868438,27.813576443772916\n112630,32.4360345580161,26.399197719512607,18.40498432389329,15.71955260736302,3.8231089202550206,2.7471668235109594,3.0261457559322347,3.0918355554030112,2.8512104581546422,7.832098641122313,20.271901222164857,20.323769890943247\n112650,80.99276324932451,69.97686025131318,47.68119459618829,42.960447977109034,27.25967107415996,29.383312835813523,31.46997763112206,31.277927459276416,27.282077975608125,30.700369134330423,56.993091323785265,58.07801700557205\n112654,36.43876839912602,31.231764449358803,23.38575251661889,21.132244222804886,13.590613915397169,13.429993521685427,14.87470855788536,15.16172049465329,13.247189968502418,13.51034908333282,23.571227858341945,25.231586710498416\n112675,25.805961091810723,19.64017916515346,15.052080269841495,11.686926157029445,9.929344854476923,10.594183445656899,10.633273388145687,10.754684297130444,9.856399433984183,9.13336558186592,15.469976324920408,16.560194410390338\n112677,1293.1588307855552,1207.5437335217996,1263.6147506513332,1010.3801669254318,757.0713997808363,700.3814696917055,700.204990404317,686.8852074407597,714.5242479264157,940.0992277584076,1175.8144836159759,1176.6570431111738\n112685,24.34345847562225,19.992185340602514,13.765774008087016,11.791772828618582,3.5423905664253343,3.2069662676475565,3.5651979663447118,3.6990363830629023,3.195499962279532,5.744370892602271,14.657939854213073,14.888316980503733\n112700,22.434583089271964,17.82144947316379,12.127388667954516,10.817272330116355,4.906616990654301,4.9433684854527815,5.601523496426784,5.668192309298252,4.913325112138581,5.530593658842423,13.077252865955757,13.606158454615128\n112703,139.8346468336408,120.47321011775942,92.81742282968523,82.97127949087854,47.239725280047246,42.80185512128972,41.34053982941977,42.91233110892593,44.78297633673115,52.08566226730988,97.94820732139048,100.69918973453657\n112705,817.1426063136689,662.1292130851791,461.5081018056363,388.71828230573794,132.43456433790269,108.01635420069928,102.12693291407199,103.49484674471765,112.1596518753988,204.70465264884447,494.30747271118236,508.4196127499701\n112718,440.95490320416883,403.5478871701139,327.7898624405329,303.51141689234356,197.38117021254362,174.87863920403944,152.24807708832608,152.9720250954685,182.6558328312315,270.4144356478663,379.89220698507967,386.36979076560334\n112720,29.37967479434805,24.733429251520505,17.99183965633292,15.66767584745744,8.547455234069144,8.596307954255082,9.075315484129032,9.419687521533799,8.62625866588739,9.383925591602127,17.595615990038976,18.701234474812743\n112721,116.26225816789191,90.69921275589229,54.076735485795126,51.576972315789334,30.726009143405904,32.613280817409084,34.1458568652737,34.86809054648384,30.431268340431426,31.905571269846092,63.551546580791786,60.525120079169675\n112722,32.95231213675779,27.433551521946367,18.205139761665095,17.577065225850326,9.919825046877087,10.772046506591066,11.586857632175061,11.396061551554112,10.16727183754375,11.163111739978245,22.21173725243221,22.057616091181067\n112723,22.720675280364674,19.546952271943493,15.140300568628165,12.615527464582922,3.808099595218908,2.8390808858048424,3.6817949366817615,3.822289210418621,3.0213262938499383,5.280893457651957,14.082944773841106,15.310889057983212\n112745,47.62880009286709,39.81753203137825,27.388400368588037,25.36203034305144,13.718538960505553,14.793327761177178,15.717666804238705,15.826123567096948,13.979256738615152,15.952014606967092,31.559012151359905,30.885435690292255\n112757,30.091838775779742,24.116544429290066,16.675115629330666,14.35012885410545,6.153976180824417,6.436109150133399,7.450118954014404,7.639151486700232,6.400792312348514,7.215688251299128,17.38917106619589,18.299346505247634\n112769,594.9654654768891,498.32540150354043,361.42392474967767,321.31360563647394,225.48731565999157,224.89690676211552,229.87032266418547,236.32309228180688,236.76245350312084,225.18606153307545,397.25716331407705,410.0315126346308\n112772,45.66947314470968,34.84521544704321,21.40860120203007,17.853177073553102,9.152542315387933,9.978021380228752,10.518661866683038,10.878705718163586,9.59757138585157,10.368009600626202,23.131049372732104,24.850916320590827\n112780,294.3523495499386,258.13141805820806,193.5624836584866,178.3900098789119,147.46390887904727,153.4879492492751,172.1344036614413,168.80266158959276,148.27259609141578,141.81181882016475,220.20266917474413,232.9109803353113\n112781,426.01804038840055,363.04322112887616,270.0298388065718,236.99423806147513,143.33174221265162,128.02226151840733,104.0628852278448,109.55825107499489,131.2838363268969,163.39800134610354,300.27297053518424,307.97555154936026\n112792,218.33933176656595,186.3165270711442,138.79133544156085,123.49616490621742,77.04433496999052,81.20136951241071,90.89402205714052,79.49275235739267,78.07522665703694,83.94950998371507,149.82404397871576,156.32070871804677\n112816,2042.108233236089,1792.750643155031,1387.063938360379,1219.3065033132725,681.9083755209613,631.0112819207266,585.8944596575609,582.5171030448638,646.303999686703,852.0206728935342,1564.72356046743,1576.4431282807288\n112832,206.99837961733292,182.84735299411244,138.83479080507357,132.05566933796183,116.1978813469849,125.00960320334146,136.29243044843756,137.256130655862,121.032922588585,111.62453605687386,156.40410835408142,162.03768157006343\n112833,2195.912757234656,2063.2872211011545,1736.3730452933758,1636.2502306734614,1541.5908428815144,1630.92724149345,1687.6521846280325,1704.9344163883445,1604.2121224936454,1524.2896709512952,1844.377932832865,1899.3475291308564\n112836,383.2254169121422,333.5078500774442,243.55575696059023,214.89997858164307,132.96962059846848,122.17728004249004,104.06657669070633,107.31015450388712,120.59534805323788,150.0115369958064,280.17842371382477,285.9116461936775\n112841,161.34250938194157,128.0145747798789,91.69504090783865,85.96619969851692,51.77763118218689,52.66700653728877,58.64289775849505,57.75399330376882,50.90389650167703,56.264517615060186,113.45246873186717,110.27978337999093\n112848,40.58499875543706,32.63958391006444,22.64148363387358,21.716990578044175,17.423841786432444,18.250131510221625,19.961517506274745,20.3585358528147,17.984008660595222,17.352920078678167,25.28956078118705,25.42549315347035\n112849,31.690831853305973,22.1560905431573,14.898827131727487,12.892053472490986,10.855941037530878,12.439279277041836,12.552643881631644,12.295272482493587,10.937199634511899,9.548055693257183,18.020934278964035,17.417182459807236\n112875,30.778472416883737,25.827726259644443,17.944045289473372,14.855233888094261,7.0817155836587915,7.679396511068113,8.217165642938204,8.267819889173756,7.494772523600404,8.82572610001205,19.289435300866735,20.967083452549904\n112882,188.99116436449629,168.05511119617364,118.17829917996009,103.9599616604961,64.52556662678599,62.36748244053876,54.74116252828867,56.02403468900683,62.192190585929275,74.75052640312514,131.6622710892634,138.09006327563395\n112896,360.3237639149963,318.397420064235,230.49546220059656,199.87091813472486,111.43105764096961,106.55069104512333,87.01216368007864,90.11384776810024,106.74570876718911,137.79316891294587,261.78292636442285,264.86668874260397\n112900,1030.8390427072488,889.0074085663754,872.9106438710279,616.2551952036412,459.5697563336598,471.8243488167565,503.77134375514447,504.26693844841446,448.82324383558887,502.64195347884146,790.1785274308595,799.5812189791266\n112901,70.30089224248019,54.43860313152604,37.36155123669728,33.8937545369353,22.78815979982929,24.989354079581457,29.18306089286109,29.475608698104757,23.938140317016813,22.39279984956139,41.0690653988822,41.500922419568994\n112928,69.00135162062118,56.483645516540676,36.82686188115373,33.01619579662109,21.950641539584133,23.48628439780296,25.32773865451571,25.73262554106363,22.413734389955643,22.58909105318779,42.47121627222668,43.39516464783341\n112940,89.55903080123252,69.61977280334595,41.183630384310106,37.60151807913852,21.725814212026375,24.401118292458,27.465935039985716,27.93980463510831,22.494488430634696,22.38748141339686,52.46473450550249,52.0408840907847\n112947,28.14905779031443,23.225742884529534,15.123312418748228,14.466348093676247,13.057049412594829,13.979250316748761,15.953962068752675,16.193374154717226,13.975109856094585,11.874766716143013,15.240237855617629,16.768360797997975\n112954,262.1547378019123,235.66127700671746,200.71413396682013,200.89495343054284,240.908885277903,258.96902253766234,294.9837463084903,293.77070848692233,251.90597663372557,204.0470030472253,211.86968774815958,213.67299164545264\n112955,27.930408640924192,23.23918780691149,16.681300546972693,14.523413399605845,7.610564731579494,7.976231382498788,9.565188644213817,9.781887405302493,8.141053876010208,8.23263934130023,17.51618640975947,18.57077789768211\n112958,8.078778748366535,6.632057267920199,4.724375588059728,4.142889723240789,1.8655332166733158,1.825692342997186,2.215183581365125,2.2503901962484183,1.8634071361557387,1.9837268223947726,4.893303898934858,5.23483102325467\n112967,727.8698784165678,618.6792876587473,461.74442437546026,402.184139159693,207.48515554127903,194.58544432778217,199.401165381851,202.1651900376476,203.11780131863827,253.5474685941085,503.44083878693726,518.0411766449001\n112988,97.32390402872484,74.34530170786881,61.9019150332428,49.8415971994946,35.49732098076274,39.38058468774019,39.21667010249859,40.2037274171388,35.892805552117395,39.276239849163,63.05128132026735,66.25356660119994\n113001,674.2699323440904,562.9095139710239,502.4120045733351,421.0164528481888,243.7036815158071,249.83082616816603,199.14612667307622,187.57497054044433,241.23203966795467,306.1841515223136,523.1034427087891,524.1972495210084\n113013,164.5037487049229,140.43622462980852,104.84872105546052,99.81535617798085,103.32227875512692,113.16635368028638,132.8745345375587,127.91574035741192,107.05187593892255,85.47706179884658,120.71199274989335,125.87803256432811\n113015,23.376446745818093,18.540946687488358,13.360624316223296,10.808387925166649,7.133252696340975,7.831030777255361,8.38249584866374,8.44259677645487,7.621963705300974,6.889541333706413,13.710797751549153,14.566110123078317\n113021,45.68912480225791,32.30600260175729,25.156659380528485,21.00846465047145,18.11196779855373,19.50386788874961,20.439688314304593,19.744601289723022,17.783116440805056,18.87838054017621,29.61380661178402,30.40064285895495\n113029,11.206607801600251,9.83544704170255,7.213274335342528,6.346898926707462,2.558833767611384,2.2180906002347376,2.4374094840358578,2.467833338899673,2.308168085230281,3.4168555226255823,7.614403944549388,8.03116258667157\n113032,487.71385456779115,426.313425659754,347.25002043798685,350.2609573236986,656.1384306771738,837.3126344918147,822.9135596871055,859.245463192324,669.1331073737733,427.70696798736753,397.7593343784386,411.2492088719234\n113063,16.2061967825981,14.566976432529199,10.599461143242662,9.167536541891844,4.4565960381548875,3.937581509652809,4.835398545860514,4.822267242979587,4.050977864546313,4.865041057322962,10.999071987087389,12.10315615920636\n113073,22.820758500925958,19.582869830948447,13.963563072964028,12.583168574362844,5.964881216854048,6.152334455727867,6.57460948420867,6.678365320499828,6.0502485737472185,7.662596748047082,14.98922050516393,15.041522598946411\n113078,167.12306859005426,149.26736055632986,106.56073353417871,101.39534438787737,105.96597337333024,121.98760571027384,126.03281506842914,126.70131559518875,110.30157970691553,91.67878962935254,122.4552268041369,127.55095802084723\n113092,108.50159365322097,94.73233857557459,67.85684277578093,60.24239126145338,39.80309977979738,42.62889442811024,46.147463925280164,46.371505148673876,40.7283210689763,43.676312237994765,76.17416912789425,79.40882672090369\n113103,65.37136000538281,53.729631180027994,39.88157468364683,34.82790146127776,13.286028549687996,12.384612141984245,14.241079048282263,14.3553024234662,12.384557121149795,17.785226677409756,41.31147201786246,42.78090661899417\n113104,21.25219100978284,18.00268671488838,13.376816005769639,12.116216061966457,5.88378172561482,5.696001565301761,6.608449873238048,6.673990335624665,5.761622019369431,6.472561249070275,14.066427889473811,14.866706166834483\n113120,18.725579022615605,14.414924983853055,11.235190455287231,9.3649511965507,5.451610767256717,5.689886962298697,5.682343273690308,5.968563328838769,5.513061918530428,5.670169141545324,10.999334945597436,12.013604142642318\n113123,730.9648305299668,638.6218320961273,483.5537769211417,450.7631898784578,508.6204712507924,563.7021739090727,465.68243179575217,497.5979511191073,439.7915833974903,399.7850528398408,541.9048710612676,547.3233130340684\n113133,149.69556122775666,125.9846813481696,84.97420583060106,75.72059202167836,53.685311161855914,64.55448420975229,75.15713748842283,73.47236408801996,59.93921468372156,50.749507790071895,103.32502014868092,107.87355726227321\n113139,26.329455300253873,22.10334358313808,18.72165644087626,14.929573450928071,15.318916085502186,17.226447608515258,17.615998342346366,16.928535111445594,15.569523104431525,13.212806535112238,19.253696755812463,20.40449494154361\n113150,7.153654067477428,6.374345895944742,4.22285695016388,3.8083080016181654,1.7077629061131803,1.726167083683411,1.9736588685494112,2.0391818689804375,1.8595608788287403,1.8138106167274746,4.215303608339731,4.674025792893899\n113152,7.220910521153736,5.870579188545621,4.354516015342716,3.4199135129767866,2.0929431628944646,2.4012841870487276,2.48219332708913,2.5130664783110634,2.202725102346311,1.9940848847043893,4.5404629266641825,4.87965538478264\n113159,31.192108853652073,26.155722321261404,18.544193155714744,16.780754691446816,10.111559185897823,10.198657616387171,10.91530467954933,11.117225378907495,9.952191765844706,11.21258507142935,20.48816308207873,20.80208094123691\n113180,165.5729750757599,152.2841476616299,134.8762107126829,135.77190198449148,161.15716139328433,171.37913956736912,178.96188165470124,180.96093895060642,166.53255366211644,144.93378612765795,141.0579796216305,139.84087908169047\n113204,282.2393244841082,217.00578668847595,166.98345364358508,136.64510513885983,107.37080486336386,122.13072804015576,121.10671300223461,123.96889177383908,107.22894764887344,107.13055853103107,190.57340054505252,199.25257502023908\n113229,27.150680522414948,23.49736588118087,16.158046156954153,14.726568292891741,7.945552423271141,8.375939178453589,8.921826641187254,9.043242653827035,8.069372709105037,9.294352881154158,17.774238365731343,18.234567926068443\n113235,10.189239191685665,8.69684167615184,6.212046787238575,5.570477246805536,2.046255575115581,1.6038945125415092,1.728728811195706,1.7816029897871966,1.7067432977075814,3.0501892710384824,6.641593072821076,6.887498672891041\n113248,156.88192566532945,133.7445414431695,95.87878216115382,85.86751035674547,51.06275018564889,52.966098591530255,59.78250440717563,59.041290598498335,51.0943536373613,56.40705975543668,113.42207661697998,117.57941794272874\n113278,16.856027673559762,14.237220001885037,8.97070248937009,8.44704430183384,7.539192042961547,7.897877347380169,8.546993646588165,8.691241765141603,7.829408625656858,6.8989633189776045,9.39834958409755,10.073143665252271\n113300,21.291568671498705,13.180963427090873,10.483472217516914,8.10221599801687,6.871905550995861,7.262573225761577,7.508785080371582,7.524040691376434,6.9319055423831815,6.338561532239295,11.13121394895107,11.460244553967707\n113302,318.4668899175118,296.4360324473013,243.23873931776404,227.82115711673723,183.53694833784186,178.20743352602133,180.4209627453147,181.93605612374918,175.83467070769808,199.59776857134148,250.816801355649,261.31691960885433\n113325,446.38558147553096,387.21995513238704,290.1728717870642,252.5997293390969,179.1770314812912,175.01268146841056,181.52172017102345,182.00990359386276,181.93046057018884,182.50371480649636,331.80750433932803,336.3487479795732\n113328,416.40180382636777,367.1848668065227,258.7037007031186,221.05136385767963,146.4675394508333,157.84576913969096,142.0471301061919,141.2357673074034,139.7642200752551,150.06455747915973,292.6271626580498,299.22605188669866\n113340,37.64932473749745,32.15181930555544,20.669569905004415,18.467172706130977,5.472795029742903,5.41071325101917,6.080238295366648,6.033266444951755,5.406459864406795,9.118419602706949,23.164109653120665,24.152127956214233\n113364,251.31213871073987,224.7466170804292,185.34292134030682,161.87921211288423,88.65618689331164,80.47867245815954,60.75837098626748,62.92215835755779,77.72447447460996,110.86053758598743,201.50720461374152,202.90772968903758\n113392,21.30386147732638,18.657634857838104,14.346365195237793,13.449702558248477,10.417673368334588,10.727443247572426,11.604078346922227,11.91897462540766,10.918736586343885,10.137001996005825,13.410746724033059,14.609259780391364\n113404,52.66984365106462,41.65050808142278,28.548328805431698,24.513523374413893,10.319950011306297,10.368955661767103,12.510931523267018,12.610981355616781,10.075192881839294,12.373419586020436,30.990005587997793,32.031899545446606\n113417,31.233463466539522,26.83340073441401,18.564962568936746,17.910078019779753,12.348864251272964,12.561435982001333,13.674400986129251,13.81943872569274,12.253498815454671,13.063120024636548,21.391338123124626,22.015163160716284\n113419,387.48452948834796,328.60878385807274,231.97965617708005,193.46618242603614,100.81677443686749,98.37213382304314,98.71110377461541,101.41657097446141,97.36638560745713,126.49379594560581,265.5870487963349,267.70808508715845\n113423,27.896667399761164,22.57019153765575,15.82404529539467,14.395847952503233,8.531559177914998,8.737196111750158,9.810757829906132,9.947009715494081,8.646209262845163,9.014479358004984,16.969242818202815,17.632818523039614\n113430,10.201485334404959,8.436576911209874,6.931697119886335,5.014364065506924,2.8744154090259486,3.48035439016896,3.681093989769085,3.5241088054113776,3.295243831301452,3.2838837626512682,6.859765647799801,7.427049733143264\n113442,34.42306487392034,29.49047111519517,19.042129173580683,16.464018571956064,8.453529802865862,9.566342814041537,9.926266470860973,9.987899742838644,8.581528333612324,8.469107103194156,19.54605520729896,21.007870827048254\n113463,298.2214910918097,263.7875598242069,177.64711676056683,155.57279058143106,85.24775575146275,89.30838140048824,87.71986135920447,85.91666196921909,82.29362313598993,101.04868926286039,210.27804695146037,213.89917963208998\n113464,26.42677300593505,21.05677687699511,15.246044307964825,13.375587864678844,5.085614419709228,4.630310470203382,5.139321125259878,5.293669760279829,4.613354083865006,6.527524682401827,15.494813577490607,15.908318739428493\n113466,23.948832293473455,20.645017314659167,14.916438050141968,13.827008462800276,11.81088605169807,12.652567942598186,14.261963155428436,14.40257877496628,12.52409898580255,10.65729419826803,15.392951910087676,16.690755676850056\n113480,24.471978188246187,18.340813988755603,13.373136004462443,10.978696474450567,9.998153390579281,10.924273759260902,11.021796377982392,10.826753185245373,9.80104213677445,9.144988571402035,15.741250452839209,15.74303122831985\n113497,284.0792103471339,248.90811924294218,194.11424605400438,177.87855651746085,132.91254181817553,124.57971809276106,117.7271478806259,125.04234872900955,132.85464884660712,126.20031663844375,213.07482010684882,217.77714807199766\n113503,55.85997137137533,46.457686352406824,29.85378725938915,28.430637318526838,22.81002124938947,24.293707237188794,26.256737757044924,26.55908126446192,23.265131316950793,21.568085020458962,37.16573883578731,37.6104009995119\n113507,14.86837713354704,13.265085015144424,10.08020690008824,9.100267389518622,5.625026329421739,5.956775549956986,6.431721408839384,6.6475768326929385,5.883672231284989,6.0316910309348035,9.773874735948214,10.213103660762085\n113521,47.36637571216049,40.23542132380712,29.5943676129126,28.453793288146354,24.92806475360643,25.789758432338697,27.016748282250294,27.620209340646802,25.20745363479591,24.54005379610665,32.53395451141678,32.44156135694157\n113524,31.496717674340445,24.741808382430946,16.998775877573834,14.039198578155947,3.4874040107743767,3.3085094778668633,3.6381544714710135,3.788545074403554,3.4045921932558203,6.1056837021573624,17.002238937795354,18.100138839749874\n113532,1137.287126453196,1021.5638820968213,817.9199011527602,750.7543379296204,534.7715219506031,452.7524239944454,347.9940434098466,353.45767492328747,448.1653047902306,577.164158881871,881.1695348888074,910.853951036994\n113536,8.126573691749812,7.371676667555861,5.376667573718234,4.636198823555826,1.8211826258529729,1.9147763562559688,2.0013437375519114,2.0002887229148176,1.8605195730359807,2.3722849135229778,5.3497847140170975,5.761203260028903\n113556,13.455464178055303,10.308603645453854,7.7968789273143075,6.803188046318995,5.9766001834099445,6.3243580643743575,6.4390416208055035,6.5337456948152655,6.075928141557498,5.666619161899234,8.183005457315655,8.304872302170073\n113571,11.062700960184554,8.481156133699264,6.746645726214956,4.981588816093672,1.575231282374071,1.768689193977952,1.7945703545614964,1.8585135536294535,1.7698941251058296,2.7911033699191337,7.121331846067062,7.668208622301341\n113581,583.5836955849704,434.2572958680063,345.4857793019524,268.5853363927793,127.35913220457473,150.8890637976613,140.90324796684243,122.8428034587279,135.9337740513349,207.67457400425565,404.34905342044607,414.6750058327986\n113593,216.33783313234952,165.0387280507035,132.27014202161203,99.88513658262944,57.580441998063435,62.99447886657864,49.795951216992485,51.842372204600046,59.6709544047599,70.42257340804427,146.9068261730461,151.56997151591352\n113595,45.06041304345513,35.48656048217621,25.733143936981797,22.480942887392928,9.180048594535828,9.014682254284521,9.938740379729564,10.394365747248083,8.958348520017722,11.458801832878377,25.659573876301202,26.08548471691134\n113601,203.06775889655285,168.39188656652593,117.41609690548462,112.85572120020846,134.05250588174016,149.84305308589967,153.24975059795915,155.45485144742088,137.15496810836507,118.37452685965408,135.52369346947975,141.80765870939976\n113602,26.754423672251065,22.571381668064937,15.970184079659925,15.319207817024042,12.576916596952179,13.077526386142681,14.53775869151011,14.667027447814418,12.808542704016514,11.61307623431086,16.87564125554872,17.803574249728953\n113611,116.28301103170806,108.16572373002518,89.71126042580369,85.33273926011994,88.12850548886574,90.85119400036832,100.37947190468564,102.39045634831737,88.02116383818161,83.33804465068675,93.36695829026964,92.8785875271979\n113613,89.52801950032905,71.80264748448182,47.95161336122776,43.08522550359079,30.18699863533258,31.628629935253336,34.92849229577153,35.585980351172196,30.468414593111493,28.681773829432913,51.21211752851751,53.60662105412705\n113632,14.578291657947101,11.86865069606977,8.335478791376156,8.030781350008553,5.175380146142451,5.290085954599373,5.856891390529471,6.035832927899033,5.2397110129131885,5.185529951525534,8.831764387732203,9.08244048855616\n113636,205.75977683234052,174.17365801027097,135.18024594098895,118.88153903555529,75.23946072523879,73.7364253055453,69.76989582443848,71.6146922894847,76.02810599897991,86.33905048777764,146.36991967762543,147.20755416286374\n113646,29.680538746075023,24.401130412858173,16.26688805130917,13.977689811315676,7.311323377449721,8.129140939274597,8.833318380763581,9.059417812056928,7.974274817642245,8.03092546755142,17.076532833373864,18.180219922918116\n113652,262.413075694564,226.98590728825945,175.5785348540544,152.7723479973317,113.5634056946852,103.63002641444751,98.24099736099609,102.10407915487522,112.3752069294224,116.06204978080012,198.09354932159016,203.7409092187923\n113653,98.84556393237233,81.91979634635425,53.05558489660247,47.43884155292732,19.116115554240707,20.017737690833467,21.409529790826475,21.685565726297426,18.88733433064693,27.31403277882795,61.07383518114607,59.949809828047904\n113660,62.580614921643736,49.83217249408652,28.426321526273636,28.374917965007107,19.99789553739381,21.49347486361829,22.797052854864805,22.868358614524393,20.28593814113381,19.637582372153748,35.39910703934772,34.195618729706794\n113665,67.21896245623572,56.376386655034466,42.936239619337066,42.76317152769195,40.25660780510086,41.154532662623076,44.14203423573666,45.35622812756593,40.10014601764756,38.51173063312965,46.27963697240258,45.539576766854\n113675,31.092601138633814,24.558747899592408,18.86485624607833,15.544216489293431,13.930120401437717,15.650524521853585,15.909395364903212,15.600115086171485,14.088256024383751,12.599465141660586,20.88399596630616,21.756731526420875\n113678,82.84058380676214,73.44959519939673,53.46409878617929,47.60847858918479,31.412774800939896,31.902539412271082,36.819824232128035,36.48351716319128,31.279112377748294,31.921820810409542,59.45044515453923,63.74688258539258\n113687,221.9283327206657,173.80629907224753,144.81618142419686,117.60351419696055,61.57922105662583,61.0097992512764,46.860019586901586,49.494141468213115,64.09271106207267,80.67571716052417,148.48575866848677,156.70011038547847\n113711,107.60393471103437,88.14918304563514,58.15290737141926,52.40555444784968,29.785615961914466,31.894135207363686,35.957579294474655,36.710529419429136,30.91171526281182,31.956918950879754,67.53950349764798,70.03684263320818\n113718,810.5499612702697,716.2045138243736,564.2756224922651,498.20329216526096,285.2811761580858,268.9836935457037,242.59282033021609,240.27819765371316,272.7727893568994,340.5200936775907,611.9732277217557,623.075062750768\n113728,69.00849673665964,57.34807228843384,36.97481069820639,33.75842744524806,15.848020816962064,16.594074150637816,18.416108342335846,18.498239539450427,15.717894982025786,20.385401244172357,43.279058311681815,42.83779785937824\n113738,25.914251447737865,17.514168937673332,12.508220922940406,9.401167014453847,8.313905503555198,9.587018477273514,9.820523785705825,9.558136355461308,8.504088802335339,6.971573937212472,14.612190242011653,15.086995082403744\n113774,50.45313178635523,43.824894728647344,33.45789492479904,33.33872072533989,36.29312202976249,38.60398492166177,43.24813952787563,43.754808908749254,37.229050253150895,32.23635674096226,35.80235314693259,36.13500081262455\n113776,304.08231140053795,258.5488322133077,196.27914954709672,206.28803188373712,278.4819882918233,298.30363780948113,299.89807737074386,301.20323962994485,269.7115701527642,211.62844410518977,229.67928021515533,239.70930499667782\n113806,109.15712389285903,97.17377401074297,74.44960510890509,70.8734458314938,74.85633674398579,84.84047823243924,90.81879208080879,90.30257362860871,79.83040447381043,65.55239259691214,81.11038549839915,83.46207333313345\n113815,242.9662911924629,206.86747092798174,128.71420650101442,113.39502957097255,81.57816475228546,81.28807204306597,62.345441447978146,66.35371799630039,79.92557346721246,88.14516849803789,158.02320239962623,153.9690905484672\n113822,100.74768433145363,85.94489534794755,59.240931818410715,57.33125225453527,36.80850974425842,36.6661347611534,38.1716501208495,38.80609036219805,35.14434299430567,41.61659503726863,67.72852864157373,66.06183351334477\n113858,18.257153414533857,14.135032063671133,10.902958551916988,8.617323075549894,5.251876326415549,5.930857729577261,6.020830433953062,5.907398591304741,5.367964717165256,5.75318182553677,11.95033455446659,12.144151641044543\n113883,31.81087921352972,24.866135235144558,16.006225548251333,14.130937631242157,7.632975722232174,8.274327931869514,8.832679311596522,8.790946972141716,7.862008271062342,8.86112192341091,19.169702768489646,20.111167144277093\n113887,51.657742986804806,36.842148737873444,27.7077638063536,20.778753066902357,9.659218460246333,10.644974025335737,10.602761398466017,11.008770698881039,9.767511065166465,12.103939862691284,30.41597021579058,32.50970061435152\n113888,78.9849279372916,66.19373236405337,45.9730220364577,40.83122344512744,20.92436986459183,20.85465945070888,22.377473844959958,22.949992251625186,20.271431247395736,26.069810621442844,51.25035708873066,52.41933025165366\n113909,22.62128600859477,19.214182656722887,13.175029432497363,11.536859590747572,7.46965447235887,8.35043734298889,8.558309601358577,8.583256139325126,7.561416047827601,6.833515815637945,13.20324226604608,13.419520137174679\n113931,21.17395274032581,16.796595697177473,12.71322657012223,10.356645192465411,8.332539418776259,9.269878784626048,9.2320908031511,9.430058946911954,8.382307950112992,7.631456358583832,14.045299528540063,15.302246076003025\n113942,179.0378565388773,159.45673100544778,132.3110090081082,119.27833751118565,90.26461616476568,79.43876953940855,67.19968663493951,71.56414729499717,84.54632744425864,91.56245237446369,139.43585063571228,141.49911268651212\n113955,69.93632040696868,60.988574110822256,43.29770059792686,39.177556317416325,25.445684850933972,25.26332248494799,28.717059751685373,29.605449351590583,25.233550970842305,26.41116292307707,46.16772471448442,49.342825243790244\n113960,968.4810888755512,842.4110042176326,538.9151703352494,470.992490740031,206.5351567167449,196.89734679435932,193.85854733812042,192.80933783245644,197.28088029260556,300.3794329361219,644.9290812828668,659.9587788306296\n113965,751.1978044546264,496.81729404166407,392.2427875767056,276.2405965332528,145.79794456578682,178.56006181755902,153.32445568257762,147.43278144285682,147.87383896283902,212.4491856461051,468.60123109196496,486.3394642174554\n113967,1112.8114999206969,997.6995664840132,725.7819921162369,671.8006141167223,545.0744787226619,530.4537478400118,516.2681760991343,529.8561684740916,513.3869032472157,578.6292383544959,852.9785840308617,864.721770489914\n113969,25.403542615132622,21.044167320708464,14.90652342299058,13.59756772813959,6.672299176744767,6.584238837229629,7.795780208862114,8.091594558351293,6.766954635079556,6.881265935202838,15.730315507647823,16.399071135449642\n113970,53.585510513008636,38.033646629467285,26.883192368815866,22.37247748833338,21.40320052484892,24.538584540165118,24.87841935384938,24.546675231264906,21.82272028646274,17.69779477233154,30.798317236800802,31.011811247707442\n113973,74.78225547741745,60.64213824746247,38.390852315744816,34.735787166863844,8.992008032886837,9.096607051369402,9.847083463428689,9.968711065024914,8.693813321647907,16.71123741112028,44.68771492969755,43.13573357479855\n114003,104.11019421930803,81.93993316199725,50.59340019170543,48.273809644055966,39.93223402791286,44.08682602872261,52.85635540093979,50.34670515526611,40.17962696743408,33.30479261502498,68.84891361521366,70.48622255035158\n114007,12.855235301293689,11.029500795224946,8.306463976568399,7.2666166782481225,3.607884304779329,3.455880968650255,3.927168579624465,3.9751226482065034,3.5754242150252393,3.8226417574367675,8.177629318470743,8.952039537288403\n114033,754.8010417732477,664.4595790085464,549.5993783234071,486.09797313377334,336.28278564949807,308.2239486803748,277.4649969164208,287.5790898811057,325.47045055005447,389.84556609998504,597.3409604835595,589.0316854626922\n114068,94.39667131994179,79.44243963806836,58.31633849299897,51.28286116122937,28.385603968674044,29.96303844110996,33.095128244622884,33.277126781283044,27.35030303573378,30.822433705462334,60.61680769822799,63.89673053585197\n114073,43.764811635567035,35.98263442058668,23.24613496972001,21.377552813540593,13.819557076606005,14.34175784847272,15.665565456579246,15.857796170027052,13.39864552732735,14.395586012814375,27.702677008300046,27.923448694116466\n114082,13.529220941070673,11.186929430872922,7.247729057143703,6.617791100113365,3.745738621460607,3.830436269095858,4.590282287812179,4.5861803238834105,3.844373708088625,3.5146030423325194,8.015858877319936,8.918762696681819\n114083,513.5299589064557,400.76879036063804,295.4169097769613,242.0857730762644,180.5571059856905,202.13200209319456,178.05148497971712,173.14994448736172,186.60512170753177,189.14625371951345,363.0561739721568,367.9015913020292\n114085,38.68911276195989,30.24155847011922,20.14798631469295,18.55486559963467,10.482442466803347,10.35154841651827,12.08580093175672,12.096747390997766,9.825698601858477,10.823212246346692,23.796968453090383,23.96718975364769\n114087,6.641308415993689,5.617849539499918,3.91925202377371,3.5483207101656276,2.021440328906425,1.9085998326470606,2.299392084617456,2.320212224892521,1.9618593532570006,1.9005472448677738,4.275050695678186,4.636295458098321\n114092,60.718157543351474,49.41062474423265,32.201126987380235,28.752742947656362,7.553780807903581,7.177505749029939,7.702302482910379,7.7921188697216355,7.037000096231237,14.103121216352902,36.21163330702643,34.93133731315171\n114113,398.6262392182352,351.947692569575,283.19122854003353,294.2573751981921,178.39920613704143,165.45905848238448,137.19156321013037,142.13109729801877,172.17379040976036,186.16887219425024,293.1886488728275,307.3118029335569\n114120,1225.7589671539872,974.4058242120116,808.4403158957646,690.7241201902092,736.8991572777933,810.2603495798247,805.0940477206126,818.8277006887273,700.7733722979311,645.3211678294377,874.237908600465,915.0256780567598\n114130,46.20315697083983,37.75820787441468,26.91044892305699,27.255293985754086,28.037142836500426,29.11324548016419,31.935075185905585,32.16814649580176,28.449484560470495,25.338380477320882,30.92065837788963,30.763915914523672\n114131,69.30840133734512,59.09116345393051,42.79686364352684,36.879881749949654,21.348004064612674,22.58134604018587,19.873898307821374,19.98297687566844,22.689100193883405,25.692259312181303,47.998973592017514,49.68131441322116\n114134,41.35667096617533,35.758312435318835,24.358930511905104,21.81241214767137,12.321566035042952,13.200165518606378,14.104836953134248,14.226841035931974,12.510032781521629,13.874870362913581,27.162717742205682,27.886143363159963\n114138,95.21221696619583,80.38726856830205,60.78601662392788,53.22110991149001,24.408826018397136,23.466811252662588,27.115612560990286,27.29324589800734,23.336377951258726,29.77122455456095,62.964122419444315,66.27447901287577\n114149,19.835311367114613,16.424768722345057,11.702322115365686,11.213414343742535,8.06867782873285,8.033158485229267,8.784946465266293,9.093748283451484,7.967778995005547,8.413537044313143,12.27960315667314,12.547477801881321\n114158,582.7086576027501,493.62379541583095,344.3513071000905,287.3259540315985,190.28238211269178,212.8574058678886,193.11738769463327,194.54359968708093,193.10143941584937,204.12724879877268,402.50719297706905,406.4427068146083\n114159,51.25355532933515,45.59859481917792,40.217263480125,32.20618081284569,14.332299408987877,14.05622588007712,13.266588548576953,14.31665322204561,15.424460635844502,24.934469212447794,40.53103388585187,40.75097101504313\n114190,1804.8474979698328,1560.1651504652539,1239.7528884741646,1093.7693665907043,788.9795562779651,976.6578708236725,1183.5093165063208,1205.0454231699246,868.6483097100554,803.2753129707368,1345.8778188081533,1355.288128021335\n114205,572.4842813546372,501.03899395713233,391.87077233122676,338.0554503371372,196.2222715651652,186.12968127472922,177.75444562196896,182.47038981053186,191.36376339717216,230.10008309791698,435.8888710971652,438.2457292166089\n114216,2158.3231597043064,1597.9617917816413,1172.3669654692449,1170.9452734776362,1211.2425194769862,1291.081779589307,1407.5099650910286,1382.7055073943145,1224.885176023322,1096.0123632408672,1433.741325077442,1403.922424849205\n114218,25.72824820831819,18.669295860111376,14.481365589333354,12.338107050647025,10.943094392440367,12.218554600790336,12.936987886080688,12.27516283403987,11.151272737729833,10.706507025904356,15.835621513370661,16.82011851934671\n114221,27.37386814010102,18.196199703452425,13.040805459788876,11.058357591420533,11.019890983773394,12.021652362471483,12.395925243274828,12.286042992387085,11.016674729198261,9.805765401200388,14.930882815761805,14.940245173844673\n114246,1106.1645558411328,855.4359551718226,681.9977744898177,601.1315742453694,525.4548042676846,575.6309946693236,469.07262671532595,474.06561162619204,538.2468362989879,503.08129916807604,795.5589063971488,790.6393764127741\n114256,40.350709143756994,34.01650513595403,24.818841005288196,22.4328752741493,11.396364925881848,11.211462571446054,13.103860856664694,13.001432843169104,11.08904574696186,13.103782463953983,28.384597793834697,29.578395886424865\n114270,37.68178250975131,31.740653366074937,23.45574433137016,20.836634146800893,11.390372786163452,12.042336252181665,13.843552370831611,13.88573249618352,11.843001005549072,12.220244732795091,24.969467745309444,26.267760113587055\n114277,388.5801307608429,325.5773648956619,240.91423601766053,204.72321344891748,146.35947658710433,147.88954856389074,115.67148891512385,121.2252716255788,146.4809619689925,158.43842784362622,276.78171025181877,283.8762847925099\n114283,82.648147299247,59.48565703456544,42.1812549624907,31.705509992480945,23.031623576965117,25.946094454188238,26.147461803844635,26.695946196443245,22.949022918015306,21.5068681447141,49.50491758058905,53.69015580222513\n114286,13.677440040943434,11.91708318733374,8.159285552963796,7.640307938937835,4.194840669637132,4.210084715117365,4.64960473714887,4.77595477611671,4.27096917807586,4.48696250399103,8.922532669591064,9.463476563172232\n114288,935.6081718946804,845.8230113425396,628.0506739627991,646.0268582550979,1004.5867841766819,1241.3242391742206,1315.0561283541988,1346.735800371326,1056.7269472453647,701.6267918156883,711.9712027636734,720.64440810748\n114301,33.5791692091832,28.907882520745062,21.46635136995519,18.4670716973789,8.456075932248199,7.59771494604435,8.794791925150262,8.86543364116826,7.664376357900769,10.033989576374594,21.44953684811882,23.099652924974365\n114306,33.92744237573022,26.75720283163192,19.635018924609756,18.984363363969845,20.567557114739703,21.59724564244665,21.710982338455178,22.14036448630986,20.105886216810163,17.556885379818937,22.497504877350604,23.856597500357463\n114310,257.1035258239672,235.08341544091914,197.05529642378272,188.3958622720867,166.888442110603,169.98813552656387,181.9783286959807,176.21049309684855,165.74459934500783,172.1950728655421,214.97721154652686,218.36724668792306\n114333,105.36160555372221,72.25287135140897,49.519968441737035,38.50569152976803,21.226375113913974,24.6526508908215,24.991808365564246,24.259191852524612,21.08703534790653,22.720426459269476,60.76309885047905,60.29383975826693\n114341,44.07158355421536,30.41411744422231,22.258681391320547,21.800040031805914,28.520872442259826,32.05649474204862,32.912611012409585,30.16503596591388,26.970152485870617,23.733129126314125,27.509543832351785,27.67447739672637\n114353,707.72640253754,589.3833778610868,388.4466480093839,371.5020417239881,220.9062682258303,222.03507540857018,178.49491822590693,177.18818275050984,217.0884958864123,264.0566852926837,486.288060382045,495.6855806148128\n114367,16.45188744811301,14.153338646438861,11.37780654483778,10.20802386810579,5.685720987104237,5.483637098089182,6.087467958964256,6.197571341285739,5.529554295411427,6.047463214196313,11.098297183242602,11.626187131259226\n114389,10.479239979481061,8.919807096263517,5.775646759937464,5.296182395526709,4.37248782849238,4.617177680445825,5.012606536158855,5.158186271081201,4.6449759453797945,4.039541187449214,5.830465912819342,6.518749395657487\n114390,9.133184779818997,7.689328058571787,5.608667757413203,4.923262175333593,2.0350409528649456,1.9441463770178498,2.1711770661524743,2.217864993449454,1.9532161430150397,2.557066129537897,5.897502285051351,6.033232113167878\n114398,24.065952516341373,19.195272543286908,14.600436244205476,12.670605808275718,5.277170100083209,5.400120502710188,5.7050208548087795,5.851672088004301,5.340652017494576,6.7850601914494595,14.358889563965954,14.80029970382161\n114415,260.659362064551,234.5400363430815,160.02627704635563,140.88326508014228,93.3724895155565,95.91383285650619,105.83888429126017,101.23978809520055,91.45010079835231,114.80727651455365,192.6524898078475,198.16239654835204\n114420,50.379876105748444,42.56351621632723,30.57245640235624,27.699428125012087,19.075862731865357,20.5771900471583,22.834349839858127,23.13415735295752,20.359583935300734,18.610388978622186,31.874762975816672,33.954900923383185\n114425,25.337357950312743,20.093003487820273,16.85124477171822,17.34088209269911,20.35967899907722,20.75346200872439,20.882146278766207,21.16523091303226,19.383969855784624,18.532829910176364,18.869786444628907,17.914388118557056\n114434,79.5336680516706,61.31943667734721,49.030730379337555,45.49337065897858,39.257162019335645,37.872414753122186,34.25631290278447,36.06113708804105,35.89772358967506,36.05973087262467,52.31064795524735,50.61632102097311\n114444,29.190884437427705,22.025854384768735,15.709074822601579,12.466388673291206,10.781793093693715,11.841157846453731,12.085055368023003,12.531952794484962,11.04537083149472,9.202703465466874,15.944258863645501,16.810453295066875\n114449,19.589038682272594,14.287473694888423,11.907033316102426,8.705400038751495,4.860653020396427,5.2741455397847545,5.308640420672975,5.533672987798407,5.16634263146126,5.3566922676532736,10.828720149220278,12.043382713291393\n114466,34.94772130355639,27.601314004441683,17.77128887378337,16.661989789006363,10.846782909414932,11.210600097483672,12.639929981941942,12.667596147923806,10.873110256277984,10.69289796189219,21.312090942318136,22.102197040105516\n114467,109.82448109138814,92.3281255671673,66.91247583254653,68.27177805420365,75.34409900877408,80.3895544158331,94.1510495856037,90.75269715532932,75.76872610953359,61.103178386372775,76.99181163734319,78.56658462628684\n114468,28.03444843964923,24.037968757545343,17.135052111887077,13.79153729653062,4.486328029795885,5.028627832846434,5.365905144135535,5.324660969054262,4.594015900072949,6.730748358876629,17.161172913429258,17.894080745871502\n114469,35.05185378661349,29.05193129077237,21.319291799216227,19.49403693022347,13.277372494353427,13.973479944172015,15.404443577778238,15.574401590502903,13.689641914863866,13.27836925515995,22.88164461362458,23.621521441103976\n114473,1275.163041603051,1077.528045994471,799.0823534533365,711.2539031718061,487.06689508265436,455.2605516556454,434.99232629171956,450.84170500899035,486.0870078199058,521.1777603551809,886.4892914027461,907.4712610539398\n114475,57.425530572630535,49.09868026137284,36.321247688083915,30.713070992965914,13.13831694555232,11.758800321003498,13.798683316140272,13.889282961684323,11.9232162956696,16.829789897300845,36.970722150225626,39.948181088532905\n114476,53.24324525440798,42.69599173181956,28.336029840125104,25.16222801150785,15.425513735579967,16.809271780159733,18.039154621054667,18.397752910304593,16.201349324461788,14.948560750876602,30.32262256842918,32.22944200986811\n114493,28.77262916795962,25.714863614647726,19.635872118342913,16.66800937016897,9.577488897002045,9.567364800572152,10.840572804634537,11.076394305155224,9.708581616146578,9.97222695034711,18.981194903641924,20.798363021764047\n114494,529.186587104212,463.8533189064113,329.2543090350574,293.8643801695794,231.09906512818267,244.52247444479468,216.09141024665658,211.73942939565146,223.7348166101317,227.15074953761513,387.4664961524873,390.53490376631964\n114499,16.870030604917336,14.066555190779487,10.225044646798546,9.232965074911647,5.957728424926205,6.405437923136183,7.272517604068323,7.4018886993674515,6.425891664778801,5.7500818137974345,10.357033693938838,11.054641169424487\n114507,8.80375791064908,7.18728720143883,5.132976342248052,4.62775684592163,2.2115223724169883,2.0787008161549987,2.3912936276750756,2.451358653008877,2.0831152152048915,2.4345257488714336,5.371395675378257,5.608975346901733\n114517,29.05881091746852,24.359715232112837,18.589257771043886,16.79988909376554,8.474508156571055,8.25769874790136,9.18197350199896,9.337773283581996,8.246614721982613,9.56700343508027,18.585393940291436,19.48445770076384\n114523,53.15802549201117,41.01940313015297,27.059153198963134,26.017634775953972,18.919603507056394,19.68693545194746,22.05402648088731,22.308548394021614,18.568270491230056,17.871685289579048,31.13576764975642,30.691597068604253\n114533,13.762773794791078,11.097595999440367,8.918397893752932,7.346052296666851,4.667063810696669,5.244482077533266,5.37115542974466,5.245030162554441,4.91433577094168,5.023983491221637,9.255299107449197,9.471602984856283\n114544,66.56238995404684,52.6867976295479,33.91942516946198,32.10766338369133,29.67749476888744,33.82978954186475,42.06462177064517,40.38164076290209,31.50064326457489,23.699948251467987,42.46604136133229,43.683464257368314\n114554,1276.671862580171,1224.0587686506926,1105.6561485779578,1074.8240553505439,1032.6250521670443,1057.323493370532,1101.2160071023604,1109.656643840542,1043.270927895797,1020.6173203489802,1138.4165914495213,1154.4258185479619\n114564,1976.0576687876103,1704.9423084442092,1275.0608294941994,1102.4629488316664,733.532553958988,712.8383819184471,691.0998300496542,693.4055768327622,725.2107620174644,764.1220998910613,1436.4474908181132,1465.2685862667445\n114565,20.145594416047718,15.999227244739865,11.795530025884606,10.400351930454507,4.632257806769032,4.582850860194,5.153439626672284,5.252003797059465,4.6102083907650995,5.267639121452459,12.107559975532892,12.313050047326557\n114567,24.74357570680418,20.375884179802146,13.573063141429902,13.988802892318187,12.059070647413792,13.253690174806495,15.1955212485099,15.404415232539572,13.20706757044094,10.87207917167391,16.124515868188475,16.285715506595952\n114603,10.246307721438656,8.70918119100301,6.275819893496588,5.548651994734626,2.8099345781099343,2.6933666782862407,3.199667811739919,3.2120973311693852,2.7657443666246277,2.978912949097872,6.739586261059448,7.282986133067063\n114604,52.94306717559783,44.91267362427942,33.02642279613361,29.972189883237935,21.253614923269538,21.557009466300602,24.217033755418132,24.705167212360543,20.863525608010413,21.459255670684396,34.970748883685765,36.47803793824568\n114606,325.3107520373208,277.89857049758916,188.19073818706,168.25294460824804,93.73031057156454,108.76456186928816,109.76070235419243,112.03599022728628,97.67705476259216,116.16801008337471,218.29155388181385,223.2409241533469\n114612,11.270834010843275,8.555302098116304,4.191924154787245,4.227224847014204,4.7732555658150115,5.798101058708678,7.640411756184549,7.408112458151465,5.663967236098481,3.365917542209409,5.329430782151254,5.57486592870485\n114623,276.94550759406206,240.278642395586,184.07852952885963,161.53279860959984,74.06652930688401,62.40968010974187,53.20454597068198,53.461325719248315,61.28929985387279,103.05105762299797,199.66582141343133,203.79814999659982\n114629,182.5150934553667,162.6580574358668,125.22931681214438,153.30706734739636,147.94940415695433,158.39439714636566,170.5989713991892,172.30296823595327,149.04563175234307,124.76191722299322,140.07965979494645,141.63374135334624\n114630,13.548501186545664,10.955766474420438,6.79560831450096,6.4274318325624336,4.270227641019083,4.5052758688965895,5.334838750228295,5.332038122986631,4.455398941841895,4.04590207489942,8.217066899953096,8.68668049917221\n114635,386.15468889968525,337.48667835591993,274.369481632482,234.72433242232634,89.80040832772347,74.44514905658629,67.99553262630698,70.15441552207488,73.48186809037526,129.22835840427098,284.4241752973035,289.83134657445777\n114639,72.98791966292488,68.92535844106756,61.32680934573751,52.35501748312397,24.36735341643544,18.941012223807242,17.31822262172137,17.36307771976212,18.422510044639015,32.65951983576212,62.03902972608637,65.17641399567009\n114641,32.15246465261433,28.940566182311986,21.213172094208414,19.170025644925463,10.284575965016797,10.621236537711708,11.14154783208992,11.299289687015666,10.335321349418836,12.15604577897473,22.361970308770548,22.838274653082276\n114660,34.749289978871,30.53652551219042,22.069940871443926,17.994784298727,6.1426069335410824,5.743557869877619,7.841648026105146,7.7178772442810315,5.970655027715285,7.772308834483321,22.900504459253096,25.36340252320138\n114665,22.068155471171877,17.678894590659574,11.499495732015387,10.57282761430025,7.082013371880674,7.710216077676289,8.530198198655393,8.730666059611588,7.606323504581235,7.219800874676626,12.70268310997499,13.017021370658115\n114680,21.992041243329407,20.40788487152051,18.51098121660799,16.755529221416555,9.977467503820273,9.332610392058777,9.633223017497446,9.752037365537715,9.14329690365154,12.416804003668853,18.5902632905933,19.03892865557453\n114681,1329.902885603394,1252.7170878132213,1100.2398554875454,1083.9172360004625,1127.9546450339772,1180.6659783186751,1161.7821963323531,1224.3656684893504,1137.3384609019702,1053.618441243014,1170.3165529445548,1180.0381479460095\n114717,828.3120749792996,742.4827372543559,566.4968642572652,536.151831124372,490.341350114265,524.0587369363338,602.9405871529489,600.2508850498724,508.2598070820643,441.2207137813074,639.3174899122275,670.6520199787649\n114724,120.20425916946006,92.66980504960378,71.27600491390332,65.3932186207524,64.96219951202634,71.09805057144537,70.74663040372279,69.6059506868729,65.07751492581104,58.00826177466199,83.09328038379523,82.24349553595535\n114726,382.4120290463369,347.092326797902,275.2571026535923,246.82775300054342,183.80927587425705,192.42376910927342,209.3366410547954,213.92309789402853,190.01175384287518,188.84630617709138,281.43227141300076,296.38978634300724\n114730,13.741427633355988,11.739006296751114,7.494956281377589,6.279453595848575,2.9720792886291814,3.3619120721414455,3.5666554811775772,3.5456325688303547,3.1061036511332873,3.0414516613827454,7.4812914954793275,8.206341594693304\n114739,14.823338699020372,12.502926308968545,8.106566064208796,7.234847172998088,2.173696258322047,2.226063197922745,2.4541684712702976,2.532795843696611,2.310695091383191,2.9416023999552663,7.76461193080946,8.226047491919084\n114745,22.030865699258054,18.225412974295576,12.24945047046303,11.299363321837331,6.8672596775781285,7.148504973604342,8.135570200539846,8.174986627090759,7.120642805630407,7.044433542135468,13.60177409722321,14.766142946767856\n114747,41.529102195020066,36.37342018148036,28.853747328772517,24.60722039166246,12.446682049324618,12.195882465388655,14.055931909619671,14.417561875594133,12.159319643933175,13.916017728038247,27.405139115295814,29.489160910449886\n114761,57.36084683796289,38.57584169770818,30.485670481460083,21.09115083237458,7.944445742255919,11.501120429532385,11.671541394699315,9.858476697872469,7.968088921626409,12.60229485293048,34.141059352258324,35.75798080762648\n114780,30.590505864401695,25.894733328046442,17.626422570216278,14.205597541360929,4.80704763536593,5.192025065987157,5.385491422960535,5.416379036147341,4.832413484848335,6.844476086854022,17.913949908369936,18.486859696749605\n114787,10.74094336200305,8.373368916726603,5.658325707028992,4.90056078220506,2.129747418928201,2.296196721828334,2.4176213832807787,2.495785965983701,2.293628968288244,2.717337340248729,5.999840326732429,6.338148912928015\n114809,214.85831721350579,179.6139754215821,120.13837843408864,110.59867356396904,93.26373758764313,102.07807623195355,107.67768465560822,111.17666097294138,94.80044509273542,87.8644046219243,137.14931715637826,140.9505453261169\n114810,343.65301702906197,254.54249586621185,180.0370125409935,161.1174499082747,129.95291978829422,139.9572415102033,117.32433360557437,110.25872284898915,131.5651172307509,136.5947115114643,224.38630756566013,228.24620812515215\n114821,11.87221981462172,9.045183640388887,6.107490065487344,4.921655251714754,2.919192060585052,3.2819198683339574,3.444551905485832,3.596384686425642,3.20951287192218,2.8464408675172232,5.880601678639346,6.499042520649258\n114824,83.11830510970428,68.64776291705131,49.65113123981955,45.494192448963,31.289505526974697,32.693143852197515,35.95914596834917,36.224086728809496,31.800834735029685,31.748073144220736,54.13783563578926,55.97454167980299\n114832,833.8617084011132,670.4996355650655,470.63700475979294,373.0337365743657,173.06731576287157,187.2331224972804,158.91813223429105,164.70319297412843,175.96856146433583,235.59456146544807,533.7939655535389,549.6444491392197\n114842,11.621302598246347,9.623431870414933,6.952243965459487,6.023008022718361,2.3643966275652066,2.276345601023305,2.49793422346216,2.5596700918864266,2.278717560579739,3.0781790699928915,7.279773002785481,7.397060343372335\n114848,28.275529215665262,23.628859059136374,16.219387244381544,14.942280905910694,8.312327951976169,8.454216038114646,9.14896387427671,9.340074364159557,8.332067767491642,9.532169666614982,17.525043740301165,17.899488033680385\n114859,11.493772478507173,9.926601861720243,6.728475270532418,6.1369167492917915,3.7174404513231982,3.8085567512633354,4.275192706735953,4.304199557108964,3.754868992185233,3.8111061911809863,7.645372191358214,8.053154151659097\n114875,17.64889605533639,14.185446522829434,9.365586458530446,8.188444849010692,4.222746317446441,4.526597056993834,5.159249048646832,5.312618328678449,4.525340293455151,4.18131309537466,9.529648673270232,10.327531662712008\n114883,37.68454855547329,31.167842424144396,21.88249273441905,20.34462503660884,15.27061093968514,16.230664513789552,16.901594853568234,16.976787972485585,15.698461976430005,15.396893103945725,24.948548687932327,25.508303946002997\n114887,27.623331486841078,22.3648156739542,13.96120710100657,13.105271030005818,7.748191874664307,8.208165625503039,9.434186678783309,9.56329759757417,8.329544244429762,7.31361723587897,15.718537516207153,17.06051967177869\n114926,187.8176367184403,157.6000682231814,117.17013284172147,101.60855303569234,52.36057790610105,46.52744514267732,42.89682122439072,44.072856996808696,47.2641470434073,63.003293920656866,126.0939353636273,131.56950415975138\n114927,17.961348048520893,14.549427108305295,10.107494572826102,9.413176260190312,5.109869484575016,5.077236209609681,5.731910484645348,5.85542199474221,5.003592080319357,5.486959490099856,10.964902313163702,11.385968646615675\n114947,21.783799167781655,17.611696489183846,11.007452664206342,10.135608880399204,5.610096217949645,5.991813105023496,6.908730941355638,6.736163507820643,5.687120184346363,5.777292002735534,13.694801400704717,14.222934684857272\n114967,218.8707147141852,207.17622109802804,175.13640861586862,167.03638900630838,143.42198464947126,147.99163854699432,147.8803910981484,147.48064125973477,145.13158346290825,155.85842033334515,188.90758694379988,191.37868169438357\n114988,51.82395942854686,42.59571623440696,32.74184118557619,29.120817134639804,10.648820727448458,9.295313648884266,10.67014843731642,10.854781242958024,9.374502101381104,14.712035360280113,33.51885978548653,34.21365557614654\n114996,7.294166720121363,6.566708209287578,4.648529451038249,4.122224256517635,1.6102563254014461,1.4273649439857201,1.5574108178202153,1.6041923514272964,1.552534246553587,2.0567054600851393,4.5548969741319585,4.987557961988219\n114998,11.503522544554725,10.374889039984927,7.378096328020591,6.635765765042752,2.5948605333953254,2.214789679352398,2.4007464950489443,2.388040356693837,2.3693787560627513,3.9417080393033355,8.199793518267176,8.474192290477529\n115030,5.6576657391186185,4.35271789529454,3.407495893176627,2.64402758135922,1.0296800372192432,1.1257889114460644,1.1771504425935309,1.172236741033918,1.1002608650110715,1.3445492018599794,3.391804344611481,3.520649925798005\n115045,110.5674172472981,89.80592427458798,66.27283546864874,61.8415810157959,32.90597799495336,33.15421571856094,36.7912058244656,37.449758531559105,32.09639728994285,36.810003570319346,71.81452767135136,71.79072880875809\n115048,26.95042938970941,21.86373051254976,15.55476259331701,13.705226324080048,4.972337307893134,4.6268950667982836,5.5032411431743,5.776295515539437,4.846584902950897,6.090263221295,14.890563607085848,15.763126946092651\n115055,129.73192420058396,103.75636913098805,72.03685660721666,62.391639528302555,17.085672426320592,15.641034273243767,17.59188135277819,18.16998161338593,15.378605576770322,31.040795980034886,79.25999467692812,77.87764479147927\n115067,597.1080439145852,501.6267768195256,387.6404851822588,335.7983205144036,182.46076864782088,172.0321190615979,172.8101263063851,177.32104784969468,176.13609529386378,226.53325562373524,424.83217805437096,418.5297253466807\n115079,243.19809711370365,212.418438619661,149.76477847773688,130.04993210724206,82.13095085986177,87.036649009321,95.282791257073,94.65110559926656,82.16847415743725,97.73528639717782,172.78453903034935,178.7283764377644\n115083,82.6914789962854,68.4097300436797,46.907143706310436,39.76776244888876,19.112887735746025,20.103398747753424,22.714098122023966,22.73271511790351,19.23591604515988,23.626381197555034,51.43691161663554,54.80148557190711\n115107,106.03791461125938,76.5386503107747,60.027340364483464,45.49861155526447,31.574823950024786,33.75428439702034,33.43032208944428,34.12658356193922,31.49174705086759,33.312375218658666,62.679269996131474,67.78435270977135\n115120,11.995248771679702,9.96966195406127,7.158601202393103,6.616222470126358,3.5379744678195166,3.4769899395592896,4.033234356926462,4.148414480573525,3.5569825542876155,3.659783548530594,7.206151022127328,7.632886936790234\n115125,58.87538909773484,46.08749044286045,31.17809682245292,26.27529214472297,9.87508138515758,9.899892203107127,10.636704583002281,10.876239541072714,9.752002988171286,13.420823645879599,32.44877390581026,34.263405540567604\n115143,27.068472439672743,23.907210407799496,17.962296339631195,15.08405151402181,6.420600477748783,5.677330934233861,7.09932794629797,7.1086513993555105,5.794842635130749,7.8290473545624595,18.31383160175205,19.795323439665925\n115156,13.643305689487136,9.215734472255253,6.1834940450190805,5.166383679681178,5.524406193803732,5.953280447692045,6.0529769099132835,6.248680702317059,5.513101093333341,4.511275952388398,6.566128249110866,7.3152406954397\n115163,65.71297495325825,54.879166468399134,35.85670873136421,28.71134834360446,18.06839162603841,20.462566089729666,20.838012736018634,21.279754591282348,17.806252615962258,17.411077180777244,39.88392765590158,41.65493758691459\n115164,32.240112664816486,27.43878724794175,18.914839871940806,15.971817928139629,7.589119908385542,8.13164176504035,9.07063139809268,9.122871652182054,8.000091429872763,8.98437044364488,20.604196414587655,22.011075845771018\n115180,733.5664436501792,649.4254769059027,519.2828197853886,457.37047257378555,255.05744214313555,224.53607697311793,209.85220775926084,206.9334324757258,240.17286740339574,302.93637531725915,550.1146192471308,557.1218285147118\n115185,47.886696546473644,35.7476726504722,26.14575672531594,21.96678687028628,14.99933141210647,16.39749130163153,16.29020367920023,17.055599088255995,14.908150895769248,15.2335388910932,28.65018522069474,31.322625952286188\n115188,165.79313554082208,138.38782270331916,92.75872139041364,97.07834462153048,116.15424772379077,128.25613028528298,127.44023278603963,129.72929038046152,117.0490303702972,90.29309420625641,107.71516763587297,112.05456772683382\n115221,6.396661544437584,5.532400109524301,3.9586125338904834,3.3717639632694603,1.5511802662246876,1.5241368893519889,1.7180734641314848,1.757926454603117,1.5730760567174025,1.8228454189775807,4.120425126048774,4.506242172112677\n115240,20.74789083261123,15.642804216240918,10.412845467545468,8.306593669605222,5.230798009132397,5.991861519116236,6.151091740772686,6.291739869204123,5.529821985515035,5.2131868229333165,11.492798098067254,12.100307173679644\n115265,2284.60762213774,1998.611294146386,1598.6810387593227,1514.014108219515,932.1115557068464,890.5786176046083,833.9076180916974,851.2919133836588,872.010667625191,1171.364801443591,1801.5095245204573,1780.2703632084504\n115309,57.83812560366265,49.73942388553391,36.61949480100639,33.57636527468455,26.686556451325476,29.029086719828424,31.350311787702942,31.737890531181982,27.906842708850434,25.523416230612163,37.74435728872839,40.19760039970726\n115311,7.768043408782235,7.010786373482271,5.0933008785381615,4.349420912263478,1.7715029765128985,1.8503124768940262,1.9718633568849449,2.03143537937226,1.8856054065817136,2.1266560471633573,4.763383139930856,5.256425674464368\n115335,4.272169408896316,2.7671962340011333,1.83153096778186,1.5037744715656427,1.3399294804736317,1.726607702970079,1.8522556804228383,1.80584243777566,1.6761677376652582,1.273144880591499,2.0903074151199994,2.157219785762132\n115336,45.94055025338029,34.32567058215174,29.12252191865308,21.81338421459659,11.877733899542967,13.047934722851053,13.191439264879769,13.64787279245917,12.22285217202087,13.910450504988194,27.42020578840853,29.281613510080035\n115343,262.9793760146898,207.62828119642737,120.75352051529919,105.7210259974033,65.11041672284642,73.34284314925429,79.96788736709075,79.98001479381433,67.2380555277901,68.4676944521978,148.16629653115996,153.1770523876139\n115363,119.99292537073494,86.02341704567934,67.27154699142649,51.58406322834734,42.84035498188724,53.477028763302776,58.7931751106149,51.477899821619076,44.175531282753816,45.718479269697106,78.73986300338593,82.25078841279861\n115377,126.69547001863846,93.12318565696611,65.8516997524671,60.49613615592731,53.52673492949832,57.06574444159861,57.2958559540099,57.782076225501385,51.89797995321768,50.12453702789108,79.43408831861919,78.83511732003811\n115392,313.4064261776929,279.84106737792143,202.566460596753,192.63412026705893,171.98784965596457,183.51717794026004,179.21321008112042,181.9628122853325,176.29475934569538,183.47080298545322,254.69947085101893,253.4919840070356\n115394,34.00675886352376,28.435226426785313,21.591827170867163,19.406645278369457,10.978194996965604,11.371559757370456,12.836716379230209,12.97103478227628,11.237073315359453,11.554493035671305,22.485016573330125,23.326721770761583\n115409,1255.7467508036368,1155.0791460093535,983.2367284681327,963.6564789141612,939.5604058157336,956.1778062368456,970.1380730329835,974.6576838391402,941.593747264421,943.0106344506049,1054.4585365255996,1053.5658281786332\n115412,89.5909046859071,77.16824531700084,49.11522035212183,46.895120717178855,37.166814679012134,37.472180340684226,41.6883670360248,39.433446243115014,34.4885640132062,35.70219909701481,62.206497853090866,63.33315727778967\n115424,249.95096060121784,218.32592422250778,169.38859029911595,160.93108336205418,155.67078010472497,167.53204133546356,176.08812829154138,177.68678483723042,159.11100570060404,144.9558877263796,185.3224454033496,192.87216228880297\n115432,392.036582356286,314.09352668011115,254.47375188562413,211.48620474444368,132.1556427994102,139.74384527759113,104.5193378035962,109.36127149864093,137.8219099210936,163.21182521233533,277.32498854104125,290.30066872678333\n115434,45.75096325035599,38.525067890476464,27.599982863542444,24.815434205511327,17.230799260731313,19.121841533633077,21.976965945775216,22.020176716117575,18.989979501902376,16.754338998096827,29.45690405223093,31.561535234720044\n115453,33.27485837028464,28.554742219582415,19.040056072085616,17.972849851269856,14.866019493875106,15.85642139162208,16.726241082259694,17.016059954410746,15.0107477819112,13.701911073246103,20.8855751228359,21.59234261892899\n115468,1280.3319311186403,1084.2454207144922,820.1957413866073,733.850661692454,502.91442864132694,467.564920217283,450.3661933730469,460.8546282765589,495.69446314186763,550.5175043633379,906.3521407690167,927.3967739857317\n115482,13.763561731295308,10.281412732114324,8.247743710288033,6.232090344392163,4.767771720678057,5.430922405648889,5.7100257763616815,5.436225012262373,4.883975140176492,4.959084552658607,8.290559433881633,8.972025571294617\n115505,260.61046459429383,219.20154849307866,160.53425340962264,139.35387175188333,102.26918795313551,107.04971250120201,103.3781700557855,106.95030981644804,104.784259356355,120.60156013721897,181.57058757719966,190.72867161523595\n115512,281.41028633408854,216.23170188268267,181.874634898376,150.0064563596231,173.8576698506066,231.83236472633297,222.01718864266616,240.77737197973943,186.26001039710624,149.1281792800717,186.16528273157584,195.81563013241674\n115513,380.1501287476392,298.22523947608795,230.80525197684966,191.21608019320095,203.59781374360227,219.9686239466641,176.60090509743873,192.33874928207047,200.96423598942096,177.4019024468426,280.01612932188027,276.60035136331976\n115514,15.773246416955727,13.705932005997962,9.934860758400632,8.462035091191355,3.0902533245283323,2.8025915171129077,3.161585587389232,3.2863017133597237,2.8521027096905103,4.190332757666709,9.934136277717766,10.780476035670313\n115535,79.87041344231938,63.554476262033404,50.398006219190734,46.68022557627874,54.735474711548335,60.579807694981746,61.64397404560098,59.87675216902989,53.87030902663498,46.20994043256123,59.30262449056808,59.05788641406933\n115541,32.79556145362503,21.993908703804564,17.364021510147392,12.603372141177639,5.208567796167584,5.7769945195965295,5.9265589995245485,5.949650819273172,5.339756893322132,7.016376977991494,17.66400119998362,17.647569764667153\n115545,1163.3196082537888,1084.3461952115977,952.9218867881364,934.09607307543,960.6049002905797,1006.35268302945,1067.1757745393945,1057.547755900243,972.1101441396371,908.2751138136474,1020.470018274564,1033.4829518396248\n115548,9.230532813334806,8.028550934785995,6.247406214341633,5.336789265680244,2.5009779818531723,2.2579965984618586,2.638517874306074,2.7432362222583113,2.320499112775467,2.7009688377440524,5.717105568078284,6.1787144373253176\n115549,758.0659540552525,635.6420526974341,451.238137685736,405.14738609258677,285.4569469340043,289.5631630997205,315.6828542303198,316.081045652207,297.1092321806987,296.5204681132573,558.3676428035776,563.8770582354008\n115562,46.701226749405066,37.364602288011554,24.939493582506575,22.16129324496951,12.263520789460113,13.219623391010554,15.341745362435795,15.685860877706673,12.973856529727193,12.334761653518314,26.95676195575348,28.155173269933886\n115567,169.3202738537966,144.18845227788236,99.66699914655186,98.03577086693582,109.02578286569874,120.31955474583795,125.19208408351129,125.08245741382815,110.27919011459343,93.57374809189128,114.43567752386464,115.3414312147697\n115593,50.8257204657745,43.48312488177713,33.504223787716,27.79295460680548,12.667454641285282,12.297061438082592,14.735053088556104,15.003657801353999,12.507478706608229,15.086243070399563,31.9399446267293,34.07865795002058\n115603,55.34213020971388,48.508086851509596,38.66493943042399,34.2427248472607,20.843521765206862,18.926677538327585,17.594365892090988,17.72765253967169,18.89966437886884,24.11654387612191,41.58269268432298,42.43387349608063\n115629,12.580228386696568,11.262552461746937,7.647430323158457,7.053017511786359,2.2235455449131374,1.6553397644512924,1.7714655402541506,1.820736185369137,1.8759270428512957,3.6795853249402013,8.430093403378326,8.755575896902178\n115659,34.54995716263171,30.563078198990567,24.180781996595616,21.196978407479605,10.486397464658655,10.808985328606918,12.535652039149166,12.794476365706446,10.976710443991717,11.148190419368976,23.143517601837694,24.603653422651053\n115662,341.0558287695718,267.6035404183192,202.14710779461686,163.61614713383744,135.10213070966194,151.17384916409569,125.98736380755234,125.41767457831307,137.53830377698478,132.6555306116655,236.9526827493062,240.95818758564315\n115675,32.10565608536139,26.08234728363499,17.211850187885503,14.167014436578617,2.855009060071455,2.479354471724597,3.1201833231109477,3.1474403415089616,2.545511257118357,6.186113690468577,18.45812400454239,19.175614788427062\n115738,79.02859192214108,59.34392054643277,43.4579337873199,34.073908023867496,32.050468354754265,37.111998381115114,37.48004938856998,36.10911122344456,32.433224667583985,26.42676140447242,51.12246635994549,53.51427508037425\n115739,134.32959929463294,117.09028596963054,78.84449806326337,77.1396043579415,75.45338432099658,81.99488652468992,87.17073392077096,86.12690760333255,75.84092224229065,68.28364462836925,93.35274627926155,96.45827193083531\n115757,152.43447738086945,115.66432403846247,89.06822943882975,69.26724897509277,46.986851409609486,53.08832573484986,52.886777854532326,53.80648694116701,48.53717440274347,48.37005200534336,94.93580754011815,102.24255548325355\n115762,46.432528512278644,40.11847357246721,30.196500071284834,29.28287626782216,26.60254494463312,27.157046380864166,29.84788509286487,30.327614150722027,26.712813863915688,24.63117747436941,31.74343611009732,33.00537215632905\n115806,22.840574811059962,20.14229962025133,12.44998703887736,11.012496465245851,6.549892992720826,7.475091171705162,8.004475497450715,7.537832085806956,6.332236055756025,6.189282708797333,14.286989654205938,15.485200358309223\n115827,15.478418351705788,13.144197464998214,9.422468233260119,8.51334987485157,5.44020994757266,5.773268333801045,6.176488221633668,6.32904515935063,5.634540024162232,5.548707150860365,10.170708988859625,10.497097578688626\n115834,104.16508275383146,88.6892213167017,75.01153535920976,77.82047865716235,103.51438620399385,110.93848931488213,112.73016683724822,111.71604535662892,105.27983328762095,86.1649045259359,79.16032904861754,79.18720166255395\n115863,221.710998953312,197.27108610041714,141.59464877650777,124.87565389717773,76.95053736316795,75.90620153410003,89.33687529146707,90.56547543403333,76.31200267350704,84.56651521506916,157.78152666207424,167.84602551163243\n115888,39.524274595192146,32.68903938081133,21.03856171098482,19.57880825028024,12.855011205642443,13.776904395160289,15.140390884265232,15.331034185466192,13.00092722715918,13.144671595699819,25.267819297802525,25.832117890181028\n115895,57.20505364461749,43.78000481316854,32.57816272957868,28.21123023877037,21.53401430500718,23.85516087398734,24.01450868893136,23.709742928263665,21.933229591272546,20.779148120992776,35.999764753515386,37.090480608430866\n115931,1057.2062383074021,935.1763157227866,748.8131906115564,655.5017899997611,370.7812161703438,323.91037410728575,285.19473264288655,296.89529957787295,339.75742246660434,449.40498454919145,817.0741382309742,818.9162764099879\n115934,32.7988805026156,25.604861316214027,15.296865988587985,14.224670225249072,9.223402791025167,9.572330248782986,11.383784928849535,10.976676300843703,9.309021032869667,8.938618182958429,20.69576373302831,21.34014397270727\n115952,1333.0668697725312,1211.0078959930547,950.8049600147991,821.5438610821959,412.437081145595,334.7568899367187,296.0711197855227,305.4183590973956,367.9241921425576,545.0108619517139,993.56309744574,1043.9471313876788\n115958,248.2945398242619,174.8896158475872,131.08876655118084,92.70823442744242,53.69376182057339,58.93402406515375,51.045954191268805,53.14912859722614,53.94449643912013,63.14832091320911,140.15259739294788,151.8998843509676\n115965,21.110028356365042,17.30269055825578,11.987914494037483,10.968219631278796,7.262464407029704,7.5090963924794485,8.668187646682771,8.695780762248344,7.484710693996377,7.278951422351373,13.36151425887931,14.108625735391557\n115976,65.35251109734081,54.33850693628843,38.22751970387509,35.33905281441236,26.77141612617039,28.497842254275636,31.84400656429318,31.998973499028047,27.69662585937014,25.548684701451425,42.911235833062776,44.50040233394162\n115979,126.09253321937193,106.87789427471769,73.89509867442493,63.57986421514583,20.12413521983577,19.443164502591383,20.736957188756147,20.897648816914348,19.506880947946097,36.433319438378476,81.01831978106001,80.71628926983652\n115983,46.238409672322966,37.33896744674463,24.910560042403926,22.990283441538363,15.143065097447838,16.236299045106293,19.06623676277685,19.175250179203193,15.954908763678226,14.934454418893878,29.312770885880617,30.402284804829677\n115984,70.98651849779968,52.91556940717984,39.501042440941525,29.891891757414157,19.22251359537904,20.391298945779905,20.326762671872217,21.34414768495125,19.04767796906853,20.69423988866931,39.39261146218967,40.47837968269649\n115986,183.50382623572753,159.01227550901731,118.55296223086239,106.28622609967653,80.64370253266192,87.40230254375166,93.73368978732809,93.98718028485165,84.06449435051013,83.91083944973072,127.64722977939222,134.28007337056758\n115995,49.15579894690719,43.90275116153657,35.006343495123026,33.85582163936128,29.996192158539905,30.04828218386044,32.09800013087637,32.51666705370866,29.435411353964735,28.31712443254182,34.69033368062112,36.53467295892648\n116023,47.69015404155466,41.05007037523976,29.335539117417397,26.541477563836388,15.605995486015178,16.548933658078735,17.277317940320458,17.551350396327745,15.687799817516119,17.713649433667964,31.986663298921894,32.045313737328186\n116024,1450.0659573222893,1195.8949192090122,837.8693211870265,852.6729220807925,994.1479997543482,1085.5272055820683,1191.3204992365606,1157.2300039577738,1018.6364920850583,833.2175036013932,1070.5991484730807,1067.096101262885\n116029,32.95914354055854,23.838179361730923,18.371882616667413,14.881562384546083,13.86635003095005,15.686981395647617,16.80883184964242,15.72708151987555,14.025760112008424,13.313910983104527,20.528162216911838,21.868300030553634\n116039,74.18518442505587,63.24038590897502,34.209096550992484,32.521688287567756,22.420076606470246,24.922839959864692,28.956825786846267,27.194286801719254,21.622357238035516,20.603350075939648,48.7316180038218,50.33945187879623\n116045,23.360306578510734,19.762248109150562,13.667168533101059,12.382170549670747,6.840559665286164,7.323031495741128,7.882462160117163,8.076471775660076,7.243816333299871,7.845296768267976,14.147910613456174,14.453413845680327\n116050,14.624588143677084,12.07225446384173,7.726212267069714,7.46194505611957,5.141876156127999,5.51376345253995,6.249791492849262,6.227967707773706,5.464309848518532,4.77534412371306,9.332551489384555,9.783972693778383\n116053,36.33530164711429,32.268208480732746,20.802708969481262,18.630274255289745,11.746703752890866,12.466136412254967,13.486586924518406,13.384078876231627,11.993903388312111,12.235325168533084,24.348205608687113,26.169541675238126\n116060,44.290304465198,35.60960543073253,25.39173642254509,23.74892585311291,19.63327581904322,20.831535114547968,22.532782846563595,23.15600529478197,20.360003026623467,19.15015806995882,27.129652612611814,27.1066594290296\n116071,47.14990235341837,39.63755000836645,28.239538233545343,27.246490718190657,22.71138865186871,23.54507087463667,24.713412349330053,25.44088792768024,23.005704018205304,22.80538165580409,30.9503876155848,30.73117919155432\n116082,67.26815247367446,58.19050405829557,43.38852282741462,37.40109925620311,21.32302724235655,20.965070761005784,24.3840957478185,24.883163440277855,20.840166478826006,23.239151842808706,45.4427195130859,47.84103356175162\n116095,39.400027695217226,35.44678987174833,24.363786500524526,24.306316313871942,22.66579414949647,23.228158197539926,24.354928700796233,24.93647490459533,22.464833271015014,20.5688488668828,24.881657137333452,26.2296301425518\n116133,76.00918236956649,67.95303554219373,52.74218866163867,48.12173697036371,31.65715730668934,30.629064731607027,34.22302646736868,34.056688515141765,30.15520259754405,35.15875805579155,57.881480584307134,60.87101104492095\n116138,183.05391735159026,150.67961943129268,114.41643036860661,99.69972876173127,35.232685063681814,27.238563806793632,24.73728908125413,25.80197008162517,28.80788368697617,52.76079721121007,115.18610825512158,117.38025609705807\n116142,547.6195809715063,479.4056869202278,350.1492644846198,293.76387034906327,176.35511166836912,179.10928242114463,163.6078272729592,162.89718354254512,173.81586474974657,212.67089753925995,401.4816277129662,413.670082545019\n116145,1043.5480940427442,898.6574811102811,700.84845844602,646.6668384447576,610.9435399774428,648.2927441467311,604.9009061763551,655.4328549408337,612.2292200864504,591.5368954197257,754.7434276528037,775.3563280861841\n116155,65.68415208203375,56.51748558128448,44.01588174558689,39.98411953256517,25.800237247831177,27.24074599679413,28.344628547577447,28.113991463584842,24.909802123972558,28.255240392861964,46.972050473120724,48.09180972381003\n116159,45.93658895611456,39.72990148518267,28.21677015939049,25.31906532655365,11.718988621408105,12.197380952028544,12.99190353540359,13.119374980330381,11.682697743892295,15.495404227496328,31.24031460536782,31.68740370769812\n116183,27.2609791899623,21.05203832974601,12.914889630769027,12.75025081215275,11.96315153577525,12.937113131225214,15.041703726099044,15.064527360155774,12.481391028687886,10.422492830876925,16.00554772042646,16.039869903320916\n116213,23.766693345321343,18.572379179373584,13.146529400026937,10.190215140739708,3.8365941897784697,4.363926611635464,4.485139011967892,4.73512818031852,4.219924451799453,5.530957083622746,13.361157286048615,14.465973605542153\n116215,68.79179550611114,52.62322094270557,40.3911791719279,28.818344515398298,11.210503578377217,12.816044645126304,12.801505957552118,13.338565655960098,11.822619773537337,16.079944765780766,40.22772612173152,41.88622103432476\n116221,1691.8246198284655,1415.6133338938503,1022.4704963842846,938.5920106514195,390.4067864344807,359.1423293721637,370.86570696381017,364.272721934093,357.6845583528589,526.0063347769179,1144.8613982845366,1186.3842385919204\n116225,65.75937549981991,54.17509875210125,36.70143802886842,33.11113692212727,22.028273570937355,23.53209912036817,26.386440496577745,26.469956965704885,22.834287150982245,20.897256534406658,40.18419777041018,43.02849825162246\n116246,15.708613692851232,13.537551815829724,9.678710121205487,9.508206476500579,8.402066312911609,8.825945239422515,9.704556332830224,9.713396491810581,8.651426475734553,7.783217521155376,10.871715042054301,11.336240104810456\n116254,12.987415376573107,9.97766930556313,7.288613936675885,5.795921628751522,6.067778275141118,7.021970062479744,7.127882012378323,6.838171076598354,6.258512827820225,5.003715607144153,8.586223573988086,8.905465096676119\n116260,93.05560600303292,76.04146876836701,56.902844048673735,54.246907620304036,48.58776147278789,54.84854859488652,58.72588550623562,59.4820672610685,51.11650068839548,44.2511291213635,61.66675720201559,60.994821174226146\n116279,21.90085471453099,18.63097083815067,11.985617044125334,9.420526536555283,2.599103271357367,2.9745030572935995,3.273183433947854,3.114700703972482,2.666650032755098,3.9658547957654178,12.705255997285713,13.486114415284304\n116295,52.31192842327728,44.274534371102305,29.778735232159875,27.887853043088683,12.954624003447162,13.103175536347804,14.821949846409701,15.246060509977596,12.878089349277058,15.657941479931523,35.31433058221381,35.32879701837992\n116296,418.08829692006043,316.21990186065057,254.64537078470332,191.46480309695224,138.4885146056735,157.61472148534997,152.97591894439392,156.14377873311872,139.75218155144805,162.8253865540587,278.4847407283197,293.2756973630913\n116314,285.89859497158784,242.9977420445281,172.15088588542278,146.65847821148685,81.47668128322375,79.30171606494613,86.93048255120254,86.27096194259127,80.1882041806482,88.68476580242154,202.6308329075699,212.8837889233919\n116315,51.70163447320904,44.96793548898525,33.83744980133562,30.883432262167158,21.746886187507442,22.022377747493156,24.395447598426227,24.694124549302497,21.85032509550523,22.027349824930074,33.785847869278044,35.971442227096695\n116317,25.08889410292483,21.432991698497695,16.165519831509055,14.462840393909099,7.967182281177573,7.7144460489241915,8.685808423232032,8.961953145030877,7.914847415413097,8.097635503513894,15.338068291800125,16.81227171419164\n116322,429.9522643942835,381.0446717197643,297.6682287476533,270.4649755010276,199.1541283005223,193.1779306916589,185.40054864015067,187.584851453193,197.28861062221722,205.48854881668586,334.1750330617691,337.38091655435244\n116323,358.5086763886125,276.1274158356608,205.28647727695696,173.56795374529446,127.06198525868443,140.66481299803763,113.21667234952447,110.9918930757232,129.78626844792896,124.19035070230986,239.78285406501058,244.64212987160482\n116328,46.258622356690616,39.21550066098092,27.717605781985494,23.75412432079862,8.216238612763787,7.14646120930169,7.877889849423267,8.03844931809743,7.243919711085709,12.6523480760314,29.180967709805408,30.817339510879513\n116339,52.07612834627188,38.666980169846966,33.36186179060254,26.707196535424718,23.456463775774484,24.401337123423446,24.326186011064017,24.747294727939085,22.171864643248764,22.897567384503596,32.95930904675323,34.45235452594127\n116346,39.818606893502974,33.39447646710727,24.339106958107617,21.141397910015016,9.847040053014402,9.505208774464728,10.813946468780124,10.997446223486861,9.494596934678274,11.287819400106951,24.648839201344824,26.478083553134084\n116396,17.917574971076405,14.342547275337417,10.079060153803855,8.808278620023213,3.4317483289041966,3.3339106317105656,4.323481725792368,4.348308953403751,3.433210485034897,4.077747201545714,10.647758359398392,11.165713310275109\n116400,324.4797461734516,291.5999418273749,211.706106523888,204.90581563957338,163.83111974255152,173.28557843588266,209.63378184729967,207.90329493697783,173.11211490833946,168.54394957604495,258.265813633059,268.72977153865116\n116415,44.836168673105774,34.85780807545147,23.441486716299828,20.50510171994322,6.588870904333728,6.412127501773168,7.714243707231261,7.775794649314607,6.444611086275212,9.110405000956366,25.333682141619192,26.066418678872324\n116420,7.456204198411957,6.047563341059105,4.328627128257396,3.940703373876871,1.9584771757117991,2.0200258107493885,2.329614626906622,2.3916984609920235,2.06910315906229,1.9931819544135319,4.3124904746315655,4.532617096515101\n116425,3455.3845397001346,2817.306936735787,2013.4920426923973,1698.830696268584,525.5395954479735,447.0893202590456,475.3230692777519,484.62646150540235,457.7565286207345,854.0889439726485,2189.997865535071,2227.5042345054044\n116445,88.64618209308686,69.41883850576872,44.39552304938552,42.52902885295628,28.426076005111916,29.65917889674846,33.94461771761865,33.96698451042023,28.163641695206195,26.676576371034432,54.11503384956586,54.2655047895832\n116458,33.37257035206467,26.388050439710263,16.82871192531257,14.831668665588913,8.698494910309652,9.411294143226158,11.10727165544627,11.17990164940148,8.952410781083163,8.588349968279074,19.595082606656934,20.39267629305628\n116460,10.28904022437,8.723927065810708,6.474426078167357,6.185892262991651,4.452587508710915,4.651357957602599,5.06795462531626,5.169817379295723,4.735599420002931,4.460542337471597,6.515095309670861,6.870947078762373\n116469,52.43660810848089,46.54404948482825,32.39214463421177,30.481680154512233,13.940751718856834,12.801404905271486,13.305051391103502,13.61100106250646,12.867730257414028,18.633036555816346,35.29514222055215,35.54245394281838\n116471,13.97962527061331,11.685811156834484,8.178951623297184,7.179484160987191,3.3587448916919893,3.3836368273975834,3.7009344476369295,3.7912678584476724,3.409381529704137,4.08322249698115,8.646027165219238,9.045559069234118\n116478,54.7709956232497,44.18682955562319,25.56150326353511,24.528707017809293,14.914698260534365,16.30509286361363,17.72844634909474,17.61488374818138,15.290783837928279,15.636277028084967,32.092410748538455,32.14136668342743\n116485,46.759263691729856,41.58667058076407,29.35332191481336,25.17374112737469,8.116426196964165,7.6907273715654245,8.37592597575328,8.503650368840102,7.692326015176617,13.625585722549598,30.752122177210104,31.92773282278645\n116496,26.097729036392142,23.393115919191775,15.678376042986514,14.304594503914831,8.192149605198578,8.631083771083675,9.277448629741325,9.04723037989302,7.896796778926191,9.271998683224519,18.01357746661951,19.220154822761465\n116497,13.683860268083768,9.438561277016685,7.809466088897408,5.755480488300505,1.5768859367222223,1.6433312364059394,1.703866079105846,1.77027211722294,1.7294033980891839,2.6947139190327647,7.0189601101771535,7.5651084527057275\n116503,704.5155399243533,526.1491334138234,380.9768728260817,291.9561914866202,182.77669905152422,221.73619277250967,200.72306125780486,185.02262839247163,190.72796501366378,214.10874975858076,486.0539892607869,482.7382241989984\n116504,104.73856279992,76.07106691491875,53.445661121672565,43.47764129144953,31.2846213755393,37.40318886763365,38.40697756050442,36.21147400922963,31.522244205781263,29.956832007047353,64.0228543305229,64.09889541531834\n116515,82.5407671626644,70.04991535563317,51.412872494262885,46.70352430438751,31.34969771984282,32.8826421074867,37.74210983063459,37.856767750428816,32.24729261042077,31.24570372800786,55.86452777394788,58.88811102472453\n116539,271.52050378034806,240.5338574630758,180.6295579736758,166.7000001974038,95.47683100943266,95.98994852754544,95.64123246144129,97.10869932410732,92.2138196868802,117.16204378450874,199.3406412128689,200.0426784542201\n116540,24.245376801238933,18.774255239156727,14.652321793135247,11.94913685747807,8.20958395543548,8.627590044244474,8.776773200612064,9.12554586842683,8.609930070009925,8.142543235857632,14.31528940198021,16.033140090564427\n116543,14.683125013382568,11.909487791242562,8.449158735655152,7.10946987987101,3.645223120422799,3.6875986651050443,4.278993547331642,4.439257618201315,3.645983713844499,3.6811624011438973,7.86629192885337,8.5078634427982\n116544,59.32837516475106,52.60867509148073,38.42925175144341,35.5041772179359,26.44864067210248,28.166758613468605,30.058011953715415,30.543712777815156,27.090825237636338,26.908788609336018,42.66172389802975,44.80332551129202\n116561,168.09340382471004,144.82406584613588,110.22468878145119,97.30468521244954,55.34810903867211,50.91341797937718,46.70492076262848,47.27000229981095,51.2906758137833,67.14438664235213,120.21874106834483,121.42272093051689\n116583,20.51093477160326,16.476897529509237,11.681055255265907,10.334127762131256,3.610698487898219,3.234364033317081,3.903345429845431,3.992621340601419,3.2384456650155484,4.5969791850616994,12.550216921580668,12.883094993912149\n116605,62.67721977090564,51.55825538785036,33.39992508956946,29.029225096534077,12.726365394365128,13.54555951534595,15.166922871608573,14.95398393123016,12.842307463538113,16.15927399821361,40.4240828195971,41.63075603144034\n116615,57.85129543862063,45.39942066071851,29.81118992380047,27.38965800294807,21.44793474004591,23.9249800048752,28.825236018564258,28.542005895783863,22.610069155484098,19.397673387462707,34.931645155736426,35.72438402440298\n116629,39.603200996674886,29.530532033556625,21.83167677024786,16.64183873038733,10.678836727206766,11.246332127794686,11.3817677189317,11.902156295335713,10.803008211586347,10.542957594977157,20.29988282763806,21.41490676103172\n116649,12.90804721079314,11.15975611947837,8.364070695072774,7.413271314788488,3.644556768668864,3.394792880834352,3.9637981877930684,4.048245941950476,3.5405120564771657,3.8734597137606426,8.300724184695834,9.154151607418694\n116655,14.118559272821688,10.781868211004781,6.598317896441881,6.478074206011831,4.901434271051885,5.130917520972584,5.721918829336418,5.786427713230692,5.060263518188246,4.587547181080225,7.745639833694805,7.761154284372287\n116658,148.19320242934106,128.61495418847178,81.81884968789933,72.27042240704813,41.720531963110425,48.77364445140689,52.92368473643323,52.72480185439085,44.00505993117842,47.28168407583353,94.78701375030558,98.47336815551502\n116663,82.69551370684538,71.50091386847252,48.480919291828435,44.652988002755194,25.827381715571807,27.485581411947642,29.151761899827047,29.459562115296066,25.90528530992571,30.050385367176414,56.711335035094756,57.8147049002008\n116679,68.86466174490211,59.462676769151976,41.431519180774686,36.841764939082,20.87643922744497,22.621632825777365,24.64863465614061,25.086036676094295,21.99749210547058,22.9576629799512,45.51758558238407,48.00118779388963\n116683,759.6590286186112,702.7879879175489,565.8536437617502,487.5851395984713,298.5202679668912,289.4204862645452,261.664912661481,260.1617721290141,283.7360587485559,372.5056882554177,630.6172323956359,638.9295254214007\n116687,257.2359335752734,216.34210513577358,162.88369680284245,146.65087413675607,104.16672250826504,100.30115561256302,98.05867394767654,101.28960613960908,105.57341838213875,104.43485581239356,174.4212573788854,181.4131490943763\n116694,6.756034377329446,5.5477481643656485,4.571269792034413,3.742296155398994,2.711038767254779,3.019029985628494,3.050422557541648,3.1706672252492125,2.9239951142428375,2.675521878691892,4.41784928167022,4.932944038059401\n116724,114.57384390584828,98.29938122081218,83.09982445793096,65.3736362276339,30.201860363846095,30.49431516897217,28.64887551481667,30.73423403211249,32.62060539401327,50.498382951674934,86.86785290394748,86.59616936597553\n116748,82.03139821436476,64.18444208629651,40.20237693480367,34.47323073201117,18.880499853080323,20.45547802053194,22.02220936897239,22.113052011626888,19.298139535106916,22.362112811871018,47.70758074878464,50.412219754558045\n116750,13.555385326028668,11.758428541493327,7.969242596156423,7.0681458919601,2.6636867443438845,2.8326168186832286,3.1806387223234047,3.2299634297789637,2.908190288902471,3.3562927412437835,8.084489334910963,8.76780474253693\n116772,61.37880067034719,47.18052025460042,30.89853504708813,25.04616495681712,8.669750945828136,9.632424886984744,10.042468973662368,10.312646390241811,9.291654930551326,13.07224150853367,33.25730407398635,35.0016366108168\n116777,770.453076328819,669.6930757695604,463.4001193094957,419.927818924288,268.0745718104123,263.9061635511195,209.01632672685133,213.75951716778897,259.8380762693787,307.5103134467341,561.6452673409768,560.8858404330398\n116779,519.3497768153504,460.28290248224704,380.4337204024308,391.59764944371335,555.1871809090652,613.6728832411036,703.1025385412813,687.6837907585649,573.1907629757469,417.67150050327535,418.3349800856706,411.2242653792554\n116787,28.8101713607111,26.030127541562685,20.00904706141993,19.28073927225675,18.888688317923663,19.415498132160366,20.922440050308207,21.26530125088578,19.37020989844746,18.338029521399626,21.95504884638773,22.636463615631133\n116791,104.03999975105764,83.57068409616222,54.29400564567906,50.109395627780316,29.619461000767988,31.154825184508315,33.93243485725147,34.500167917526575,28.898951150438506,32.32363163106211,65.25115736384602,64.4272036352248\n116795,7.238450433480906,6.131586215172557,4.092607904828031,3.579019058665519,0.9815430248058689,0.9019452861165255,1.0065461531800861,1.044285295801437,0.9819344903183452,1.4191587893779047,3.811804553998594,4.052785533209065\n116818,441.76236792293355,388.0388475011799,306.62600717518836,302.7927600178722,365.03969606509946,402.51411858355914,444.4615467137353,444.2904966097893,384.8937947331126,305.29315866401834,336.4353412400246,338.4795570064857\n116819,2046.5074002878143,1794.1292105328623,1406.9206818940968,1293.22723360893,999.2583969013442,1056.5713191414318,1135.3923243970426,1125.2054280087393,1010.8968880123605,1072.1831427019163,1588.4586182381925,1601.4016291321254\n116822,153.6739746492162,132.71277629289784,95.65490709772713,85.55746015144844,49.762593482283286,46.271962290475564,38.38213741861432,39.79352427221906,46.95924457337852,64.51014004361211,112.98898203293959,114.20809689883498\n116829,42.48804541453855,35.35670776366521,24.50513553015274,22.79407603154075,16.343812307751225,17.53657147671375,19.759520639103673,19.77981983784334,17.22016418846828,15.40536321258348,27.67173005259224,29.255494523084856\n116833,35.44304119731297,30.360182000113507,21.91578379229543,21.201498412322398,17.979149224599045,19.08155017490329,21.128879763026628,21.220883621962596,18.660704874860443,16.571201473472122,24.128628187006804,25.25278418760573\n116839,316.9691381899644,283.4193631162353,225.24426648306255,210.60089567424933,151.19501625104508,146.2789477729176,139.36548512806593,140.15780338164305,146.6488585620007,186.26616055325962,255.32536602425643,259.7707733360051\n116852,35.865264245189564,28.443218603788573,19.90207149865407,16.738258843443482,4.100842006394739,3.425732831968045,3.73548586137833,3.8715337913328454,3.577581584934554,7.64516361991937,19.960437180058154,21.162474412732685\n116859,285.99915796402746,229.95369057630464,167.49574118561978,143.16463321253434,99.96580508053866,100.05847470161063,82.6443056942692,88.31173023867514,99.99783246476957,103.81180663170137,184.6045325393763,190.0745681634078\n116889,58.66850331781267,49.80987092617122,36.80719801194641,36.982472917054224,36.421862774963174,37.056300434200736,39.25717109981469,39.89103573404044,35.367340255738775,33.690998817309286,41.219300174657,40.375255598988325\n116897,413.31521467413086,370.36187439255457,283.76936746123386,256.9581862611592,145.13220456140363,123.63139601519762,102.89952490654889,106.52731933969606,127.00692154241302,178.81995331201108,322.6328262562526,325.56139151582386\n116934,756.9836362425349,689.1595010116263,530.1406670292939,486.1860454155997,328.72882397289084,288.29778923732016,226.3258689369841,241.02786777438826,299.27690131111643,376.73007095782174,595.8185400083889,601.3996076160345\n116954,1278.4072182337982,1163.6360126763495,947.3016361377568,806.5592468006407,475.07266627049836,438.09811532830815,401.00558406111213,408.5112147857896,451.2513286989528,522.2294608855481,973.3519522290636,1011.9184447883424\n116956,79.06850705792465,67.35544084180647,51.14307100484107,45.92445080901277,23.66044510069438,23.048664542220543,25.754461877832888,26.118870354985162,22.969180713487482,26.8232682159153,51.88393559129971,54.83734789811694\n116964,37.71705107267976,25.269907444059736,18.976882111788658,15.32091931158091,14.830159814744787,16.90126937365931,17.464805676234946,16.24319879113639,14.34456821888603,14.088168460650492,22.536482730697834,22.915480955086938\n116971,71.12069667961437,59.0297752203293,41.58142682132491,35.22042792708765,16.49254155654255,18.390027250299763,22.64177385473244,22.556988082406356,17.48838666870764,17.989691363548072,44.24462535108944,46.1455230161215\n116979,8.391530602241836,6.8510562109547815,5.371257369027069,4.8364800103673895,2.8680379953629536,2.940399363826826,3.608173515434185,3.772788287192838,3.02124482676971,3.110345241054826,5.358023368133966,5.44635267119572\n116981,190.11717229435797,138.64229379585134,104.79121129018317,87.45218512947856,92.16692243186307,102.48682480764475,102.49958332400527,104.30657072920275,91.49299770717276,78.59182808145188,112.23428152817144,124.23962308746704\n116984,18.765316172381567,13.602342147578296,10.387510471071124,8.092184768410755,6.086894409954593,6.709803385517409,6.70747630169441,7.0251525116838955,6.27626446817206,5.789625632415917,11.019793141171352,11.805668500145973\n116986,144.18432152970416,121.61714836511618,91.42222167262148,80.89295620315539,26.835839409935094,22.804448041934368,24.407498189527097,25.066281648486576,23.440187732673813,46.02426027771451,98.04858849400404,97.91299848090102\n116992,62.184410922648,49.84348145448377,37.19728746791786,33.747200160165505,17.202205364386888,16.83000420381773,19.052639416420863,19.261244395141503,16.44432745374016,19.828553028699304,38.9644039430945,38.869060438794406\n117001,58.735598211651144,49.77377979460461,36.320991212409076,30.75782285473187,9.974320214225475,9.625952089901869,10.47120966637302,10.702753429462884,9.451976199310632,16.685193499050023,38.41419458707893,39.779316883086224\n117002,38.75289073700305,34.10102781558261,27.162974419866156,23.464433781505026,9.832335854139762,8.666016926059314,10.136204560904408,10.438681007540774,8.904737851614122,11.479203688881725,25.927534453606825,27.824691849860496\n117015,8.1798755698794,7.1527600026800275,5.110531689194046,4.569734220641286,1.543348407062247,1.399127129636666,1.5402481870520885,1.6019541591200517,1.5069963323875029,2.1531959464836214,4.805028563474006,5.0952759767076605\n117032,23.97278408123298,20.03566354150333,11.76530024926434,11.026322865761106,7.418252541751941,8.29126066455049,9.051194314381261,9.033351557178465,7.850802389283933,7.287029665601621,14.114140017556737,14.809294314140153\n117038,18.792340061947943,15.988748933609807,9.872535676869033,9.461683776525277,5.418581208528326,5.666640876211957,5.989818448038435,6.169157053958963,5.549660653145408,5.552172601987423,10.256287519116533,10.762149113188972\n117049,25.5379540115947,18.54089994397033,13.607939781156585,12.260687203521895,17.441166517244355,19.333116213921087,19.776087457580413,19.39569178369267,17.4515018500197,12.891779394017426,16.301174445514054,16.303564904104498\n117082,28.263141508233545,22.358481198262325,14.596437424865826,12.657413527395097,3.9372093484215496,3.8701755366282775,4.219933121731949,4.419968978998924,3.7813592175710276,5.745612645895167,15.621083810128239,15.767279877699547\n117106,23.242413258361026,19.90053519400049,13.220749975292332,12.154502600351357,4.36155290766608,3.964390232428318,4.668497858983837,4.778577560095534,4.138852447154524,6.221284767768067,15.37138088732738,15.717988737059146\n117107,58.18836559193677,49.28996894797018,35.350982037232285,30.46283688183907,10.069265168584886,8.729458157431026,9.909876822244074,10.110592239379484,8.980467962044617,16.713480736750057,38.85065385412683,39.78507078140102\n117120,36.100351482004,25.72080180248121,18.733215465941907,17.028721982536513,18.489418718439534,20.4967105413151,20.698769096272294,20.28149404166903,18.661384837619533,15.604590930157306,21.848877334049448,20.932051454506833\n117121,24.89811165241691,21.37634168282362,13.825937551950048,11.290630245541648,4.053934657971012,4.492918302430452,4.810292474025716,4.760729730952932,4.090894090007996,5.079776988310878,14.218263140692473,15.158060141501931\n117133,72.03378329123377,57.29297667228126,41.038781392905,36.23577306668067,17.0537718022314,17.050150706105633,19.53156126411658,19.894155128889768,16.579348958190415,20.064408470788933,43.9510968107463,44.52573660622461\n117139,60.13992655088073,49.00645671569185,29.621286892563564,28.21509668333606,23.251350552695218,24.91243870322394,26.874845511174605,27.13945708617027,23.468598922966944,21.8564890783953,37.41386326833412,38.50144499461866\n117152,7.583456787367419,5.972735415050405,4.12327203623121,3.649968586134925,1.2403901774166757,1.1027817228963943,1.2813656993335252,1.3344457451002787,1.1294228369400294,1.4629478787471806,3.984092078109776,4.2068666866880875\n117160,419.33410321137296,360.02053763779935,277.0515285363388,242.00508072784615,146.9201945148443,135.62387055241032,117.4292623221754,120.68723191371282,139.98898963606277,162.94710720366402,306.6873846550406,312.5482602666044\n117162,27.63144067522883,22.693788646299378,15.726826609887379,15.804644666066118,14.994401615200847,15.56882644636209,17.447325952101153,17.493056243736767,15.181075333941362,13.714797990276358,18.395531759764925,18.642690331736418\n117165,157.46084735076238,126.49058382388652,82.33991065037985,78.53668561811047,59.65143114855236,62.80253702592013,68.65123848152612,69.28314925360593,60.82952374465085,57.58009343716517,98.15573002755556,100.02167402008476\n117169,737.8101545405332,643.0507890730338,491.59540899306757,428.0950924046428,236.09665868306362,232.7452527082504,229.05218213250916,222.41089874013187,232.64535963769012,280.0842794200244,541.692710286182,548.5934649990306\n117173,291.2275620046461,245.88082344795652,168.05559862243538,150.17977903327062,112.2279782871352,112.14396200787105,93.68947635406995,91.96206066113056,102.31633528175088,111.56489809405623,203.5334152185669,212.68099746250832\n117177,213.82074211395022,184.51260717164078,127.66197965728081,112.96934412155765,87.22832715172211,94.9657686881229,102.89447462775249,100.1225377751627,86.64411225410525,97.56511248610283,163.1303659509444,168.64556081048454\n117180,909.5550610341638,796.9085091959611,615.9214742032214,540.459093370437,291.0196635361427,273.9403629003625,259.6960529895226,259.5127957680041,277.53050020442754,362.8253401470571,675.6183004107102,681.7917510634752\n117218,2949.5965414561333,2513.4968613741357,1901.116050865404,1682.1694846079704,1213.9421632672047,1286.8081866951802,1415.0331452405726,1432.0163699059462,1237.0136336923038,1215.29592441813,2236.0626598988874,2236.3626777864815\n117229,231.03965661115996,179.45225829234127,144.98427253184224,118.8389831125138,71.53689150860403,74.21414760524556,63.45070817791772,68.0099624164647,71.70106014084953,79.14619840458822,152.343122764997,157.02286717342787\n117230,70.03494594757804,56.94134878388418,39.21219730175315,36.29617277959647,27.140372983545788,29.32219990721914,32.58109157023251,32.708730293521384,28.683701572515155,25.81465823861026,42.9328963227691,44.40883593919677\n117256,93.60065542968529,76.80231434353986,54.05714084033308,50.39985825359265,35.985645348846674,38.309774977122814,42.57334226267661,42.40968482692543,36.79180423353028,34.65287870036024,61.892092575371066,64.15819221938196\n117266,25.84654185237127,20.40670370167532,14.708809897987235,12.512404386587882,5.310816572469012,5.6468652650547195,6.132941539380191,6.260730088473283,5.562693732234166,6.747140575238058,15.57786186935128,15.982795630642583\n117273,1228.3657827341515,1160.7700380942008,1045.1608088273701,924.4363072861482,547.7898377911931,468.1374323953563,319.0049471799182,340.998260559809,439.9866310021549,662.6631473621671,1049.2730507286606,1084.4390181614729\n117274,2320.4903827301314,2368.1840075151376,1946.864723544394,1487.9726390155206,591.6651502782235,637.2161265404156,543.9439524126706,544.8818443794676,622.1284105431405,969.8325573658677,2112.2488096611314,2161.294501333134\n117280,228.64544845869102,202.70653655347567,131.13539850398237,133.28485585909175,158.5726728749798,171.7030697997347,189.9786044623965,188.33371192641678,161.34117725302394,118.73343063774587,155.5594412618185,168.53229189048028\n117287,230.76403114365874,195.29993012082053,131.72639748239143,126.2143368642554,127.71346808008384,139.81615741537377,156.1117167738258,158.01379161715434,132.7189092335999,110.42974742754953,151.57894178146793,157.42619094030337\n117310,192.8622256055078,168.78766695257295,117.93298601736677,106.69169049915108,66.53837179547804,73.72615751188914,79.89656141034509,77.47562470994937,68.52885540282576,83.67275283809678,145.06761596862137,147.18234322773256\n117332,23.227005362250825,17.823647749655407,13.762585733700343,12.66761900725958,14.031144080014583,15.024640194857733,15.191330786241222,15.24611190891701,13.827587308306367,12.381172626694296,15.718372553329706,15.184210116192\n117357,206.4784947301938,154.59545588554042,139.09093810924475,115.25216756937085,115.5016422721956,120.29013198881765,100.49909262835989,101.99859919052943,113.90764705015968,116.65862166016163,153.32192381437451,148.31207992240599\n117373,29.023623593907708,25.24233426746946,17.733181348165246,15.756112501981532,9.662738311790038,9.481393039859332,11.30355403293533,11.252266539555915,9.374815188155399,10.160314137285479,20.35856136972787,21.6167349409415\n117379,147.08707561317834,132.50255283334727,108.66155228018567,97.03883128446952,53.855166706429785,45.668448017916106,36.03024097408003,38.53253433928973,48.25353674615642,66.14534815225512,115.17155690649581,115.80398785910766\n117384,418.508459237166,369.0481604777388,284.95705063398395,251.65562928025497,172.8594952902862,169.15453742060956,164.27197083289718,167.08415863527603,172.07078541569427,182.60167552916212,315.4770296532113,319.0380585372918\n117413,423.3780128026149,373.98345881651176,288.5108987869935,270.74742645648496,249.52001231839253,258.0091244470412,281.3999810457095,257.31170619731915,246.04163176508933,258.52324101692784,348.2074897034098,347.72629116395075\n117418,80.444340298094,63.93403547015295,41.51610196112613,38.037608937948626,20.61380004354169,21.1703284924757,24.695589854549624,24.483606819747546,20.069577695491212,22.31888849692829,49.001983188855704,50.31003104399447\n117429,53.41606841070854,44.615835922876215,28.685205854645353,27.936639609941277,23.88096959562784,25.844560573218164,26.817546338288942,27.38189704945448,24.366844029432464,22.579108505606516,33.01432681851196,33.05336268343907\n117431,36.38201964009776,31.401657877308473,24.59631360695538,21.65761159571943,8.853234265571531,8.002711485947911,9.153762226750494,9.341471289072063,8.18511151717514,11.02198040135256,23.797594226845685,25.57228822226566\n117436,190.33309836384151,166.8865635908124,129.97529195261873,116.3208014519799,87.17138418356664,87.1440825603781,73.49624394072413,76.10896083588452,85.15013639986171,84.88668357156759,135.74332284520096,139.27734186383077\n117466,21.224184709926725,17.77488234641415,13.444952465109484,12.47016949542295,8.10588196388829,8.646896528704602,9.847158076981959,10.146203508170327,8.81398004311322,8.031540294218262,13.618271334781939,14.067468803393512\n117467,58.2149762316793,49.85882878916383,35.23511067571349,35.073901466104985,37.626526439773805,40.3469091740234,44.76084858061612,44.70203417298342,39.22000283860233,33.18269805792978,40.26169250929076,41.367270534549625\n117473,1467.1381362738068,1316.5162047371177,1093.27226652894,976.3396941083392,616.350584069969,573.5007915996824,477.88345919348217,498.7029624561474,599.1638708524728,695.635306021818,1148.661912024251,1164.4441082544276\n117477,171.71666448313258,153.41336848113724,129.56896292596448,111.30869022566918,54.96374380892297,49.07027298572987,46.45235530209783,46.6728368703027,47.87871640589311,70.64275591176293,137.08602704464147,139.2335961004951\n117490,77.1539863880215,60.41667594657032,43.12096511713721,37.580774312625245,20.406790280488938,22.09926445971701,24.49328637282176,24.849658357934807,21.08991896314702,23.61084139913163,45.53880235974313,46.77708668178442\n117504,281.0591339229283,210.32820012534552,169.41622362698646,133.1630261261929,86.68422365809919,100.124589486421,92.47979238124161,91.09938872361471,90.45793119922222,106.69832150885014,193.8322907815597,201.1287906272531\n117516,36.727735769259084,29.794597416993778,20.4797705639907,17.406968678751,13.627614866122642,15.019476461373886,15.420487120299692,15.489469508305248,13.942074795502434,12.648129933633065,22.799161111629964,24.790984557283736\n117521,21.54519699028942,18.254578237526168,13.844139885438484,12.204011079779747,6.010164933356435,5.836909928210841,6.785966431427463,6.8315291518079615,5.910227468042536,6.620348108911494,14.592773443175163,15.365328372585108\n117546,251.2191619600737,240.42642357843033,179.1869106662362,158.8424551652156,125.77742960114372,131.42841642917563,119.2676549441426,119.42974669013462,126.8258988809539,127.36875673563854,207.57733674065543,209.6355389002968\n117554,47.46650641206266,37.791987174287684,27.428654506149467,24.10595716876327,7.947462626655254,6.485616799051903,7.489330606969318,7.80880891647428,6.542437825334141,10.848656626778837,27.264972166030667,27.68883576179663\n117573,21.58726883129143,14.32296736059845,11.528905393023736,9.609924659969248,11.817415798180983,12.459145383787343,12.925156980860761,12.834927337950928,11.511169694302469,9.908404612028841,12.701070774474642,13.262625761163962\n117577,29.068926960286262,21.531040094904785,17.149778465029467,14.831829630420389,11.879013199651755,12.51433981084869,12.568756756672279,13.127073788582466,11.728216726014711,11.072653086641392,18.031693063450195,18.84608494769253\n117578,92.82049023755124,86.01282484445231,77.26059383117818,69.73096749967901,30.15991007511974,23.152672518245502,16.29325082703662,18.76021335606997,22.91919063779662,51.607832377719,79.95836579240995,79.55170759919334\n117583,10.195332599243818,7.257083938786282,5.919943429774257,4.235382877989968,1.1506617720938452,1.2668268823922324,1.3319212687849622,1.3491863949098248,1.3105441292363713,2.033422323727902,5.6506694049822705,5.892821763011195\n117586,54.85163561191399,42.21991603227774,28.12866559568821,24.107292919247257,22.597260859601217,24.499218400463462,25.0749258930461,25.725178514632077,22.56912394148674,18.995875429354864,29.970269833067494,31.737078738078722\n117589,23.01170047705109,20.070561141976928,16.115537099856326,13.98205997075156,6.31631548809535,5.46740924294598,6.318149637697305,6.501394953321791,5.594001145076678,7.30162475685227,15.152920910079342,16.245770945828642\n117592,101.53086095950921,86.35349813599233,62.6641997939381,58.457500945526405,51.72695566264519,54.58065991346807,61.64955087806417,60.61867608074415,53.464087291547216,50.19784661077987,74.55966024555529,78.51087245813946\n117597,41.15755944922401,34.181118298018546,24.64817974184333,23.343610333956107,18.674929322156085,19.176259894320616,20.982915437804106,21.56027203641999,18.65219301350127,18.243702313258677,26.3749801719405,26.95090302791469\n117605,29.54603614908527,26.055509233988968,17.246070387250292,14.735847483671305,5.560784707591162,5.628200945026666,6.114906719325954,6.047983211556954,5.217123069918088,8.02859168755548,19.17251102617575,19.93879210413395\n117623,9.408179795441212,7.942303008596367,5.461043176267047,5.033359132772725,1.6392582162180818,1.4228913063263184,1.6160477309747938,1.6919993133074618,1.518705224753798,2.3937162738819655,5.620771209077788,5.657405577638773\n117640,311.145985948478,283.0368146564414,236.62824599658012,220.88600499470263,173.8124255575182,171.00508581333986,183.6519510341098,182.52227800265817,167.83499198515733,190.82206441643942,257.7661934671986,261.85390995921574\n117657,94.63196304774495,82.31188724626863,51.051155308161654,48.56247781481904,26.765743908707993,26.231449199728072,28.936393336291104,27.770605896400138,24.790827299922846,32.45520954441131,66.61714666920753,68.03458876567024\n117661,748.6262785825638,669.3847617712117,538.518537499494,478.32327840409977,300.92554951644223,262.2401286752726,191.30924694701042,204.02355867974484,265.21594458821613,356.987450160373,588.1465918651819,601.738798907049\n117678,174.6553420837337,151.13240482042863,100.48056720973625,85.57431059017246,47.53432704121574,50.646353428464835,42.073354247779186,42.21454690807755,46.58182262274689,51.033279260135096,116.14867735473901,122.12046015461712\n117679,20.00224803873656,16.176175749940377,12.885408754074277,11.52664768988485,12.804897302062324,14.171211070683297,14.524711776699466,14.518135681738281,13.436405331766652,11.119011597275136,13.456819127505348,13.832120138889499\n117723,18.903256884633553,14.666017644150665,9.063511538922665,7.9432568954137,4.9203789035413195,5.177128027715002,5.447822412026789,5.6515462323607935,5.16541778328409,4.953821289563539,9.34633241720754,10.241559846312077\n117742,698.573013179826,599.9225026495963,393.4021960600095,345.5010021834344,143.13089772059922,143.75329780899148,136.08473783816177,139.76899871701752,142.87982889125266,204.70990142856334,466.64206204381236,468.35712943148974\n117749,22.248699414499868,16.91256897336122,13.976550813154063,10.036008668018432,6.378891318076534,7.856216829054547,8.12627384350357,8.054217579275688,7.2512876983186745,6.274402487830919,14.255701210935436,14.971335461224882\n117750,39.65211068767933,25.1608759062458,17.369741400728348,12.314966551336044,9.137223817057366,10.440173068475374,10.635911830130269,10.953067921226532,9.599871443352532,7.695852562715042,17.52524369572157,20.05912758617441\n117751,19.83710004488387,16.84709586263835,12.648271709706187,10.557927717436161,3.8723188239790503,3.1290558564803406,3.805419220920641,3.8977212875646434,3.2234527340360697,4.778714636796984,11.946137195306285,12.808911882832417\n117775,18.96761177548977,15.961065573035999,12.654354433451605,11.329562236091217,4.826965560070922,4.3415787245570545,4.994991155272405,5.104730172318212,4.473914549054365,5.731867297839735,12.691877341640483,13.1509068439005\n117777,46.89168212817279,37.51366391127261,26.2179723770128,23.18465314685416,11.277210500396222,11.17861430560818,12.569648500714875,12.821198518789995,10.928930244210424,13.071687700278506,28.364593240516566,28.92970643472918\n117783,17.736319643714697,12.10422332647076,9.906635807236624,8.36072590229675,9.454678858582701,9.915642329819962,10.216764656868003,10.243264264845036,9.661507801977253,8.500720783829708,9.938968373010637,10.423580462389713\n117793,255.41226399820226,224.46943241925194,173.21383763139173,165.34307102791487,146.21911610239033,157.45495541382272,174.96536250009234,174.54295723342398,152.21332699411326,139.39990224551914,193.6287392564426,197.79357528560504\n117796,25.236699147267007,20.67636359247352,13.667677164169996,12.436318270108089,6.6971872603455,7.138569280693476,7.729486754488354,7.788758834089913,6.8763625920277045,7.404990949183389,16.2510323292871,16.404243290308223\n117803,19.024867976404995,14.22523916217493,11.220448760585274,11.761091669481223,17.42220127637643,18.91097300498843,19.20557793980981,18.86151835952451,16.84743901158759,13.90719133381978,12.668566604695277,11.93675974922935\n117815,278.5843765780808,205.1801101685324,160.96052880658823,127.43013808381677,128.70587441374155,139.51109385753625,109.81477543882937,105.87315774947119,125.40811375474027,122.91060859198359,187.55775390951223,189.7271941611273\n117837,54.46765239329603,43.55157499090543,27.10380245828275,24.597828483040974,12.783403154576238,13.51349559974314,14.773918541132234,14.909987045254631,12.624257036058326,14.16622139018647,31.297117335055127,31.949685963450086\n117858,278.12737562861446,249.90569676348466,202.45360901974553,169.40052824955725,97.02184067774598,94.12877538733558,86.06969973865274,86.26676707462394,94.90292202483243,122.40712607959078,218.76967653794685,225.77159045021614\n117870,130.05295151273447,95.99007420404831,73.77153318771423,64.05430875893408,75.5471428895871,86.3604791959749,88.13917312913053,84.91974145167029,75.06370130420972,64.1320667991244,84.68640524372908,87.76693750387919\n117881,37.29165029908818,29.654556425594674,20.30005756225717,18.408439608351053,10.77939963471868,11.637725655957684,13.249551975674729,13.375079859175091,11.44928826042619,10.884392971406694,22.158702075387485,22.905562509417592\n117890,15.550814914903652,8.902442224072713,5.870232685022583,4.971344310985082,6.819662845364398,9.04057013566712,10.099909132723951,8.617788710938665,7.251668571847272,5.61541052542333,7.853551417530519,7.662293808680881\n117922,29.91257579598901,25.50763755494562,16.68253056862875,15.074961060795324,9.086611458589342,10.170043320498447,10.788470206352402,10.87686788612411,9.798125791010023,9.590154729131227,18.885931671909603,19.662796910888503\n117923,31.247199251360264,26.05830525812058,19.8984022583526,17.522424251445685,8.255920581813843,8.379469084565763,9.745486282376088,9.88687298611807,8.395000318580603,9.601661953699105,20.21135826772936,20.993206669336786\n117927,41.36045547866519,33.97700732137285,23.480717731559082,20.392837298643208,7.372169315690213,7.150610601309424,8.499754702856846,8.707146505592743,7.1271821697602835,9.151224251454774,24.63584239297734,26.404155318805557\n117940,14.804763269863184,11.739706309593664,7.733464561095531,6.89617402543049,3.7631914813466456,3.967299018742936,4.781142448569774,4.7956628365582485,4.03143159505618,3.8724294009618836,8.620610074953579,8.9646287291528\n117950,19.295601673091937,14.827011016009495,9.751052357058935,7.4539379027064925,3.2824497490795697,3.7664305112260417,3.986447228220137,4.055356701300878,3.6000058981334817,3.739341886588621,10.276583648726945,11.178136110876617\n117967,56.45485455439845,44.938297810522684,28.488399866684635,29.061464450985532,32.708789173880334,35.47555030621828,40.508580784355466,40.05927337476257,33.330255210053146,26.695546387493856,35.1217990504727,35.17794100376281\n117982,25.429340235996992,17.16971626445244,11.788213983159041,8.7479798903852,5.057726599637877,5.749548757835217,5.737583303817641,6.02034700040492,5.320534813830346,4.932856759927442,13.12127943836545,14.376148288340708\n117992,14.380419386986203,12.522143916344858,8.029912116961716,7.395052247406203,3.7902621257990545,3.941155774320518,4.400674385527622,4.450061645149215,3.7940466728489266,4.1981425885186345,8.862434371964506,9.202457159134786\n117993,100.02863872308458,87.14285516954747,68.50959006346257,58.58199536822738,27.933325293710357,24.862403743155948,25.13255341667882,26.05734784080195,26.550809045971405,32.12570376398489,66.95790511771038,70.88927957450713\n117998,26.756727455917154,18.619467542748012,14.830746620742442,10.667914234599724,5.975288767108515,6.388180431017025,6.408721415029658,6.743537756719551,6.093594837723199,6.852344123733731,14.222745281293715,15.60760183484657\n118008,126.99543183142586,106.66940530247209,69.1614083073244,73.86181201200877,105.4275911401791,141.27773387461505,127.82332246650883,150.43267028825562,116.4482649950073,68.75901736619392,98.52341490030807,97.64030628491794\n118010,427.7616524931129,365.8408471645587,282.5779480679351,258.45487244740383,247.4986363542444,262.1653738477652,261.29575887388347,266.44408481533657,245.02080252825172,233.76774298503216,306.08359612222836,315.40278011030546\n118023,963.4877971634268,853.8966102476212,653.9894877583655,620.8832558062651,550.1855928746775,570.1646024872457,562.3868735485395,563.8827860042993,557.389051462967,569.6764144642375,722.0668008390633,747.3946881349693\n118028,47.96605831566775,39.18637023978634,22.74838830606363,20.22850240325374,8.647260426077253,9.422350090746457,10.050997830386608,10.151873379706343,8.916414945404027,10.202445724120713,26.102174991226544,26.814339190521764\n118033,14.350091709086279,11.726134560832154,8.760892366555264,7.93336967487715,3.750984773978811,3.502549809363722,3.9374772756620717,3.9844361033625058,3.508444175875472,4.3189015781438735,9.19071039711207,9.407524411976377\n118034,36.37866212745675,30.33968249234443,22.712334072645753,19.352332587784097,7.8954124535318995,7.347636369897502,8.666974282969376,8.85954960162878,7.383666206911418,9.899370635721748,23.63428824880139,24.74771622666345\n118039,457.5062917511819,314.71636003594676,237.05620124671654,183.46024095498814,101.2785637922411,106.9509642691597,92.94545683496213,87.01072358698195,102.57592880353845,130.83911530984963,277.1803251130537,290.59659348984127\n118040,99.84416290991088,89.86825405155093,69.99285861147072,66.79429589114103,58.497780847419726,61.00972814061594,67.3388446419613,68.53815652962243,59.61068668581969,56.00500324230533,74.67520212584753,78.09575918386714\n118046,19.510866463989593,15.080752265572471,12.422913268687392,11.7665617565473,14.440738453284704,15.51893549173417,16.00647239140319,15.87287822369174,14.536387930893946,12.926578085095985,13.312480558418386,13.273344028805287\n118067,42.057205878160616,35.42804029792298,27.409342929700586,24.37430291537654,10.28072238151023,9.592007216661008,10.53942975433251,10.758890633114126,9.722742988438934,12.383687468546922,26.230027277059982,27.747940220208832\n118085,33.743215396060734,25.803234125696445,19.60972697939243,14.20390967506753,6.622583302583251,7.573183162125695,7.719335318103906,8.191294902263266,7.244979025615214,7.7849154789245985,18.376907223494243,19.69693265914263\n118104,13.098967288631396,10.981199847052238,5.676449774835351,5.227718509207273,2.9019445848043346,3.384907768581099,3.807442790016665,3.9778897991485893,3.509448712970248,3.263834304253432,6.17016454019857,6.4410795838984285\n118125,63.4111369258194,55.94022225323869,40.11580898058949,34.147271075809385,17.073846192550867,18.707323235294968,21.11549758118753,21.191400964938016,18.035997149505125,20.976469509714637,44.251937900699374,47.4468787192988\n118127,45.76675337018906,37.18259414298144,26.33353488315865,24.441837187488296,12.07849013538169,10.979240713876276,12.196217598121967,12.477457360884669,10.896569075026692,14.321932283567495,27.81782070440466,28.531556205543165\n118129,23.253347857476932,18.0317635240669,14.701172533135448,11.000672312742305,5.2702157096004605,5.857344200918638,5.800441204811876,5.981155756366578,5.449364110928845,6.149395346279369,15.142564537134692,16.119002179551178\n118146,73.60422207762737,62.03242920027047,46.740798202594235,39.47502858457451,17.028408673543666,17.71099696466857,19.14583739684645,19.38643097494378,17.426981937045987,23.89567471011518,49.318836774789794,51.691179975192846\n118156,68.12719047136707,46.472691670139994,36.43748527395763,25.26820166768765,8.332064396281675,10.283048594728486,10.557777792321607,10.88508475479102,9.107612456412776,12.840046005924954,35.43454481810748,37.920694433377\n118158,321.3922831183876,245.57235443524118,205.831867701461,162.9471954516458,127.38718610202694,138.2198770640386,115.72618549145065,113.00208839625303,132.632758898031,130.0806021804193,218.5733548437462,228.75223473104737\n118163,32.15473117227939,26.721714798143857,19.850469517787456,17.847038298139285,11.021559204083854,11.388338552657371,12.66161028329515,12.890543350782718,11.288793562725646,11.65048647417551,20.709527755127333,21.52225427390773\n118175,10.42326514482052,6.97578267048893,5.312503005006402,4.201535705043635,2.8026756485337994,3.1588694264094968,3.221301070252452,3.063736534890722,2.841678543401039,3.0174370655304013,5.839439656843823,6.115374831118233\n118220,225.2002624217428,200.8396247302951,155.2676926435348,134.89185476872078,92.38380718282154,92.27642674718959,86.5475094744664,88.3039127167545,91.82712051008053,100.82979517353037,167.13793640838014,172.9511156114439\n118247,284.481313759251,241.07466045369694,155.24445506175888,139.68695422166414,112.02460301960669,121.901167498578,131.96297319758594,132.55236684035765,113.20073368415778,106.85750464960572,181.6870813372247,191.9497752349703\n118256,34.58881717340615,30.538959760369597,23.564258265443303,23.745913283805283,26.609488816042305,27.548861511647758,28.85468825059746,29.63396450067018,26.717681427647726,24.342934095657604,23.724347570523822,23.736560004052258\n118261,37.26217469032556,30.943888367401275,21.339238894779637,19.281897161235985,17.49736010723821,19.017019329564455,19.425646186874392,19.808768381282146,17.6751645024407,15.21102302119837,23.472688737553643,25.43925989629277\n118274,47.3298195586967,36.977110397781885,26.136159541560108,22.71220503516596,6.837820053365662,5.976720404396758,6.928688837523802,7.201990770279343,6.042691921110333,10.187590269214992,26.27551074303971,26.748077830611177\n118282,40.84472550540524,34.671403119199034,24.135448769051447,22.62976840675211,17.74617478170533,18.560713533119163,19.92963849704862,20.180143755357992,18.075029471099715,17.664077086591814,27.140705866551976,27.746319748698635\n118293,128.384430253357,104.66839310429822,86.29876973417768,73.08310873756653,46.25048254848748,45.58397853075662,36.89583796501974,39.04058661649045,46.96153008664573,52.29491848261987,91.0889595556261,95.42406725753172\n118307,1251.740455679954,1062.5602330898823,769.673548902982,658.2988475762536,431.5813687834277,419.1853511380196,317.31296489679534,329.23151212028006,422.0239304047676,457.30843821664206,871.9168914077515,890.511327266153\n118329,338.0747643371189,293.0986796711754,227.74972568478316,199.07856752133748,126.5512672383527,117.74648322525435,98.67841578889882,102.95402942801817,119.14909900110683,136.93528570562862,245.60711567141448,252.35797380806213\n118337,368.17519030484675,313.3165305486614,227.4550095749194,188.13503217495185,112.30054755751391,93.01904316352791,98.32782248429825,97.5798250464212,107.61534043766078,133.77174940935188,282.64597784490405,285.2025317188405\n118340,346.58328657476125,248.46514330749548,190.47640085333353,163.03128632195802,173.68312028327733,179.62781639129764,124.6616120038128,128.337804660485,165.15221667067848,146.6572909547975,221.85006567589946,220.06690146524238\n118345,29.49625473566944,25.891659830295673,19.73179372577226,17.568734159954133,11.6649965481066,12.118096658844701,13.657203323898171,13.801655701734534,12.140816681535213,11.913445160485628,20.39826682390278,22.025433306299007\n118346,51.44473066295499,42.092819732908545,26.676125657226375,22.292823477470463,5.8199195099854935,5.799771128243091,6.583582885607857,6.585079438746607,5.625261625731309,10.276691639149371,30.013694521269468,31.56206279993269\n118353,13.273381936316785,10.370079272526754,7.379284695721978,6.443232160561438,1.8968245135371764,1.6001056826674778,1.8666907622550506,1.959585086303551,1.6315363278079518,2.6562621117109857,7.423392736069709,7.600776451775391\n118354,71.45563567126966,56.15872503623462,34.79073961320083,30.14831334111311,16.836735290272333,18.33851691060968,21.02523986093636,20.95812559886303,17.713785200989882,16.11308532314955,39.628135928773936,42.67181973426991\n118356,1007.8517914072801,827.2414722704868,578.3609918062393,507.4682397719053,348.1474462333899,354.7444790582434,308.12878570161394,315.74386748354874,341.4035456401158,352.5921356599416,637.4144925363822,664.4568443377888\n118379,34.88769985738528,28.081794858194208,18.085592073342827,17.72528550799353,16.858837598016574,17.428582142224464,18.485638089526457,18.60436886971481,16.712027855939112,14.669481139516554,20.389334565856384,21.690157541557742\n118380,33.51298868836951,24.993210629640288,16.92496590873762,15.281450990329345,14.082511395725032,14.640509272690647,14.9478269866909,15.486239877215022,13.928601424635595,13.16178842486014,18.569029709055208,18.773257277798553\n118381,37.19292508382121,30.93245768458159,19.098766536810263,17.962560900655365,10.686086493394432,11.424422115379755,12.425424643426863,12.302644243380826,10.746940847756283,11.255818721763895,23.085632428268944,23.772590773430043\n118421,33.1079961278031,27.996883179342024,20.132678143604103,19.752869797066502,17.644760518886187,18.81371502893766,21.197919076229628,21.191234992434165,18.531532917237808,15.921709295807135,23.422834050953583,24.05482179729591\n118434,38.28257149434762,33.3284111322256,21.360941119470134,18.311535733120067,9.989266675718158,10.657803307233264,11.454309529643805,11.375888517636279,9.861896717482862,10.68836249677268,23.075997614898142,24.90138047288848\n118449,196.37978131023087,184.9705576963186,157.65006255319042,179.9719291076273,104.40088618533295,95.49747933528967,91.46262329450447,93.05363534860052,93.80975384754609,124.1532356483773,166.3956493327786,171.928597279362\n118454,14.765893865749307,10.588268649476149,8.631605145534188,7.386978895923242,6.658864176217162,7.8436483441480345,7.937759375512872,8.236471232785865,7.422046763227097,6.597885264739281,9.366725864229377,9.705590767902677\n118459,19.72136695259285,15.309256387494486,11.733302734712412,9.514304533304111,7.047775326875528,7.96522298719202,8.120418268002995,7.948104911965509,7.291958972876515,6.746246493679393,12.666320300664943,13.042535396153918\n118464,981.3355320002031,848.9348140431201,662.9078120548861,570.4147039877043,238.22683561698403,200.38561904276867,191.4042426177569,197.74965436412842,209.72935893742567,332.40712433962636,713.4773038768346,729.823413751183\n118472,8.000930834178334,6.639967887532983,4.890364514145859,4.225112716650863,1.3802112018539023,1.184974127193192,1.3325591567246189,1.409992116303378,1.3017970755819606,1.8733763766824196,4.460184737778636,4.876294219886923\n118482,33.56604647634626,27.782877199663393,20.30340076586261,18.112234746960333,10.892332964917118,11.973349288485613,13.746832965043478,13.859479285980042,11.901435018250474,10.82248311339193,20.819164233836574,21.883912090834205\n118484,8.953907067362133,7.608325942804656,5.644472294663864,4.903519081260601,1.6883590691707633,1.4416070660431317,1.618264122097472,1.6646037233377513,1.4738984631605123,2.441967370927917,5.887606589334275,6.064574529799528\n118485,51.29531389446443,43.822349313537956,28.26333668871514,24.4282323493203,9.064568846920668,10.040316133228407,11.356692940302823,11.398012467027177,9.38142811621971,13.019026646186095,32.20565664627781,33.716396953853454\n118514,14.934332392094616,13.516456056561056,10.272896942068089,9.018019174315793,6.131372057133523,6.431868360528109,7.322676140521765,7.420274221333281,6.505040206004586,5.867200118919598,10.126666218239997,11.098551794421773\n118517,325.6848505164115,278.3821377812691,205.80562462873505,184.98779810776642,131.0958603062884,122.95610202801514,111.80959701216369,116.9116342948429,134.37575069385264,126.69662521063462,233.04117694460604,238.04704287625364\n118525,62.007337404940884,49.40924442914369,40.415335631223705,31.356664512841512,23.38595655781955,27.215758020781127,27.672087984705886,27.19915208395657,24.408767544146723,22.93310906305669,43.531608380945684,45.53243041643507\n118526,82.28314200088016,67.18899122718449,45.230265872620805,37.31621183995549,16.66271912519487,18.074756921367737,20.77719589972954,20.597001102567997,17.169095642510413,21.4598188175378,51.42342370179622,54.34297200226109\n118533,64.91055707735771,42.259309960466695,29.334191217227882,22.512026836283045,22.78082722850166,25.333473928614648,25.61192178797688,26.438532159957305,23.46482250998942,17.73302248335411,29.375882317710865,33.46380241624936\n118554,688.8673659623846,578.4413880267297,391.26839175371805,316.29698663742164,150.9736658941456,153.4652911881493,158.91884018477577,156.00963878197075,146.4138503847387,200.00907650022097,479.86824165094316,486.1549404492789\n118576,29.577732550170378,23.36697285703724,16.22820509423313,14.231819961261825,4.199251166564643,3.3086774816743576,4.053735868214352,4.035622088062619,3.358463314974563,6.714744884573251,18.344788053675288,18.573174236658964\n118579,55.36434850547001,44.953869990489785,30.566124980138085,27.87759596990722,16.336046706576926,17.731004830383373,19.086657278740685,19.28625317274696,16.77191535599128,17.475732653390114,34.89546374822206,34.748454684085935\n118591,381.8339719499624,325.2238232139965,226.8333102611481,198.26823274620895,137.4971694488519,144.9998478813007,131.25906395727915,129.33198390566014,140.20305943057502,144.3151341768575,260.15235095989686,268.3408847419341\n118594,73.88331938528039,61.584507098014285,37.30091395245328,32.26050925205456,16.78343290320212,19.673418999829828,21.453045097116952,21.214732920471352,18.118822708703963,18.56275618893944,43.388986645938196,44.93947932022932\n118596,15.08834458135453,13.237746702587057,8.992299987894786,8.020182067839176,3.607673216342662,3.4964992435946067,3.8026566614279202,3.964972899081983,3.6122775804498644,4.220416730817961,8.69720725607661,9.241982037648416\n118604,14.126187834309512,10.293499724238426,8.465040781566103,6.152880992441625,4.081896390655132,4.546652893613551,4.58193312277548,4.831313371491477,4.472570808039256,4.147435659372569,7.490750393812007,8.52463885661892\n118610,64.1958996912015,54.27051288637734,40.279336899250204,38.25486680889067,32.81098355800833,34.227382244135285,37.37389926690645,38.071666375626734,33.22120512548261,30.946098944940097,42.79084276883179,43.75711027065267\n118619,83.74055467855662,69.06511447637561,47.02469575983968,39.632754501238175,22.651299894507883,24.144740541130464,26.080959215474703,26.3870230797379,23.04556934518794,26.14175661676331,53.471815697306525,56.08461825471282\n118636,42.56106846362344,36.90965656183452,26.782106147846427,23.99884081006697,17.012946323519106,17.483821891435298,19.308501271737082,19.58133866366932,17.40105519033984,17.31799215520779,27.795437590116048,29.697300099835655\n118642,207.43625787806243,192.7392217428433,144.97301535132718,139.54714629121466,131.6625951114229,136.19963033965635,153.17689720371013,154.55537341548498,134.06425612556592,123.95107048252335,160.81232602893144,168.79716824465115\n118648,1401.8423069549713,1207.3908159621974,917.4132787652306,801.7444337546181,532.1811096749532,535.4501522308298,529.0938675358567,525.7054810306093,553.7485195638127,560.9575727500543,997.1540815016552,1022.6377048821122\n118653,98.43448956976953,86.67745845753568,65.69917804688592,64.44155292850986,76.45655241473565,86.78320874310845,88.17477124642065,92.62470064392645,81.6400308111007,63.90800229482556,72.14966747793234,73.5900465714557\n118657,7.882210520217384,6.958668986838709,5.580815380784523,4.698030519181931,2.2126644396943456,1.9253971370469003,1.7125484828253978,1.7938158676702018,1.869474076579102,3.0965100284713962,5.7609108338532,6.012120109858143\n118661,14.382862170808304,12.254502143962684,8.066799581220858,7.002791467431307,2.9550535106782463,3.335462695590783,3.8671559355632183,3.886328059436249,3.3734091828885533,3.5707714170700062,8.188159110101587,8.829113121797986\n118666,17.1695935620116,15.284604096511757,10.994259252050913,9.605180656104741,5.9226288096570805,6.448994268434352,6.759054170464029,6.889978596688554,6.196884227242093,5.864504675635077,10.60882909977414,11.196246145720762\n118676,57.69446827123426,47.88991657820569,37.274740661481,29.979107052460158,24.721070922128614,28.588698579323474,29.126570761936314,29.330863629068933,23.77387474274152,24.78027237255622,42.27098160311414,45.483868280834834\n118698,334.63837866973523,283.24481559891854,213.3190327375796,182.17966709539675,128.52131189153175,129.0347627092442,101.83831988841723,107.63513807587641,127.85883478722303,136.69190240063958,238.56576948438436,244.05020728670115\n118709,64.02084990847337,51.97131126105242,38.82713052113012,35.047865610326525,13.740882347770466,13.023423551834476,15.062084297535035,15.328265183460791,12.996998430168702,17.707496261311064,40.48785885597479,40.9848528639318\n118718,34.46999586650553,29.116181157999275,19.5370637587828,17.210081537886957,4.0608824471717675,3.202494444865906,3.4309035266135073,3.588562536845487,3.4083872529801127,8.199004120935612,20.37644703170475,20.429562344364463\n118727,19.38413519762454,16.168365481302054,11.24456560583532,10.882166150128638,7.451272110711128,8.133147386863314,9.237135951389318,9.335245154721385,8.111198140112457,6.966519969342416,12.545858412055056,13.01807352669984\n118733,33.171886285395956,29.296594496998324,23.74883209736489,24.209643431497756,28.271643717968672,29.007211623926676,31.629452787612756,31.802856579865338,28.14820872349602,25.607539658606882,25.83955161520412,25.425469331362468\n118736,190.17728001961822,162.11424123189494,108.41991968124282,96.54927523940962,72.56646479330027,77.7507767876824,82.14246340581288,83.98573144311213,73.37649594476437,71.00226772733625,117.10241852893311,124.98888832503302\n118739,947.0376101011775,880.4396300114726,724.2847107315029,667.9230122052913,561.1304927808274,559.8576344766831,585.3979034059332,604.0470955816721,546.4023368216027,580.2267984723463,790.3337081144538,808.1096514533848\n118740,579.1626588102922,489.6489552047022,352.20339916075375,336.65111601773685,300.72176331033666,324.2538254618421,349.61735339189585,346.6112532915457,304.29219953050324,279.625244342416,415.32026365551616,413.384633623232\n118770,60.042687552755474,49.137822283212614,35.74614596649905,31.619769668222574,18.26522495334508,19.102413976607256,21.960238209918217,22.27388642454082,18.98877930917685,19.494656004647762,37.74415295668411,39.050136311131496\n118818,363.467489895511,331.3052147412086,306.55824622285803,266.5239892914803,245.09902263989989,260.0014676611269,261.3445866670718,256.98682646259533,246.65826327237224,261.095294381592,328.08853116954566,329.819829497526\n118820,104.91426580810597,91.30774307059316,62.034041602585,60.60734581185512,65.49054872265181,75.2909142295135,79.04066661784536,79.58455636917232,67.10878274177074,53.202712572966604,71.47759333132795,75.47251393811115\n118821,46.43451937853923,39.90351860740704,28.942590404944397,25.599300228819484,17.066528081617147,18.073330535728505,19.417905572517036,19.835138739466032,17.583444386350436,18.28440625360849,31.756387100277404,33.21344141500826\n118829,19.463223926216145,15.938471370991408,11.432111654840888,10.37887202474951,6.58217686016075,6.963781132923367,7.865822039804913,7.925545643633342,6.848557635330708,6.483004682885122,12.309419517616094,12.683483491619421\n118839,48.98543565944624,37.74664851127189,28.364559679867632,21.41481176247105,8.54453376253936,9.01333697955472,9.112645998838179,9.47860297807387,8.746850524102214,11.695899349042032,27.355078665577985,28.982053156589085\n118845,18.04846461613775,15.474542882431015,11.248474687929006,10.06720609044845,5.6293912784995035,5.841381159916598,6.26275187346034,6.406024522487369,5.73880151642283,6.313672421649985,12.170359633233076,12.584702699822259\n118846,31.024439097741336,26.30561691068961,16.459298440744217,14.7481758547444,9.420140144506632,10.258516118764588,11.567342201550218,11.635507095162989,9.879555713409797,8.966249652079293,19.166479973843728,20.611987238734965\n118850,29.388632008386125,22.91845941266737,15.698336306261783,13.813157411372663,5.291831972364775,4.873192778531039,5.539932292604109,5.627853455917464,4.807731386316779,6.748711279819236,17.03403219398474,17.395464061289985\n118859,25.81489731397764,20.37324074628542,16.079366397653175,12.508575054000373,9.094930709733871,10.243256784413107,10.388794713936973,10.1438384245291,9.42615574703284,9.106683060930818,17.79796441186862,18.597703032617023\n118865,13.625065120657316,11.570178236632538,7.156501700356024,7.1997003376832165,6.222324065240183,6.793158361609822,7.298188846679723,7.244622392455542,6.4454615014131695,5.514362459678803,8.448516427155223,8.750150603771065\n118872,34.950753723465866,28.41571197190214,19.442768003348323,16.929278316715706,10.27441291690633,10.688152503219746,11.491597943016568,11.598509742256736,10.476435794585647,10.428680748415788,20.426027664080905,22.020118032516237\n118919,488.87876533152615,390.59249472248337,312.6898977072956,279.22969677462515,309.3084495381069,320.14665453783266,275.93784467265976,274.21997691648005,300.1615925731478,262.2390113707605,350.85730146119346,348.4902015740225\n118927,28.561194443173726,21.387810225890682,16.208853483309607,13.555251433404624,12.76084104477197,14.170983044282924,14.265982308590939,14.073523614873634,13.083985403440328,11.369580110032121,17.95004368181884,18.44000456473253\n118931,45.59463119068452,35.724569684982484,24.958485563909676,22.114618099095114,8.26619386216069,7.700829915683256,8.646458046806867,8.850638940163568,7.539579508090445,10.845909335939,26.304150155933627,26.837811427925992\n118940,47.108668149090285,42.0133795341428,33.84896944799786,31.360663234476355,22.451625889782434,20.669989767750863,15.536027266520213,16.349849111976997,19.97883045389431,24.508936031708107,35.78429693949403,36.263125249611434\n118942,48.74292351942309,35.566704751381565,29.030034903899633,22.205982322326726,12.191271438563735,13.026167889514358,13.07163627877896,13.684560137942416,12.508453898789107,14.384251166619672,27.42652054158446,30.31560805293962\n118975,15.770152009988776,13.047368041980457,8.934179399469746,8.322282458933971,5.8583340685655525,6.231067699042306,7.236722875010039,7.239212281569499,6.294951927168421,5.644442018902189,9.674096414658143,10.602812297927064\n118976,71.05289519811895,50.97045514999472,37.43218443940174,28.531090179984652,20.176280737887357,22.814695067968866,22.964570424078243,23.973827282948086,21.00138284576896,18.260453122800914,40.02052023772171,44.10533354595203\n118977,37.06535417317988,26.85229320861414,15.964223420301419,13.990721364389525,5.273550181079893,5.9569848365508875,6.579191526200065,6.602766133115548,5.445439889988673,5.964368630574057,18.562763086855515,18.79852193332474\n118981,100.3599334226561,77.20019189629231,49.63262922363932,46.57807390471771,35.55050054658746,37.897223340918984,43.90543854162853,42.9962995338206,35.236158496666604,32.5550719978804,60.994524461748675,61.37290066376988\n119034,14.543985831899013,12.363133804362556,9.203129098185336,9.388526073150413,10.309137154821258,10.76303498736004,11.98180165851617,12.044385158134146,10.740527289315505,9.45804000559776,10.548279135142852,10.551214094959668\n119039,15.184252904377885,11.581597940194118,8.696602973267677,6.6155817258224765,4.935261708472245,5.537008539285464,5.6146123866311655,5.39942109903583,4.955612256911834,4.829649921864056,9.891354521214637,10.099073531601126\n119048,11.882272753257414,9.45520817628226,6.745869682067443,6.084226851769272,2.286020453556509,1.9646990609986117,2.209409180459516,2.292295657605916,2.0069760800968486,2.789111689871528,6.878852170915006,7.083732792171128\n119055,110.63105572039645,68.31862616911377,67.39370169852323,56.18090945815212,56.02192180456571,60.419794603001286,60.20917606423343,62.9538610026628,54.86191049666566,50.065563345016194,70.40015790178602,75.76940948293783\n119059,21.81678709065649,15.36104626142658,10.720804841408691,8.830601529994743,8.69565659497687,9.64537278788441,9.821518152514411,9.739927947312397,9.089269487910904,7.776589198226728,11.861271566436058,12.133096220558791\n119060,250.72587201527932,191.02331138882195,148.8063889825757,114.02889800991137,74.92289752481817,78.72409873291662,69.01204764816062,68.5161312222922,77.39479759112045,86.99096894931222,174.04525349864386,175.3363694509212\n119067,16.046077168442267,13.588545450913168,9.467105613035551,7.997191854887108,3.2928793111952928,3.477729547908887,3.9510112630751193,4.03487008404931,3.5136228556365716,4.041520599016997,9.91421144225554,10.60702140148464\n119068,780.8074846965353,681.5976912880121,549.9063581746135,555.0409885066335,641.5305635677838,700.8998430367557,749.5096891081392,748.3244348836201,661.6539640656388,557.6202551118382,607.5641006795935,615.9505581161451\n119078,3493.9300418506577,3078.676909570095,2384.5039592558915,2339.1113079297697,2538.8276472920265,2751.6771823756553,2988.38337764168,2873.486660420844,2638.4189778056507,2228.2017194984833,2833.331363725753,2851.130458585502\n119105,79.86531101319872,71.07737111970744,47.45107124027286,44.950232394985186,24.88781382592033,24.60942799883385,25.70011230199663,26.388608330862056,23.907192604327914,28.73784657414606,49.27821703227765,51.72973708584377\n119121,14.954465091682499,11.85150918512146,9.460420758978104,7.775352993572775,6.057089331451098,6.542628130665958,6.578413747595992,6.867170997342253,6.117672010735164,5.819979215334424,9.981949422336324,10.606146953454479\n119135,68.17822253677538,53.61307235961948,34.18208911595443,32.58048891610865,25.174131683116308,27.270578869293463,31.247792788646166,30.94598360606689,26.3995562295865,23.113485771605035,42.4052117995244,42.82067169579456\n119137,946.9516687911799,812.3107693492027,545.51305024731,482.3032823121617,240.00340920226196,238.55747943189232,214.88088337367282,227.64896660063738,232.85730172619026,314.90833949483124,630.5194102705884,628.2235732845277\n119152,103.51719845450803,86.55679475902757,59.08967536315215,54.93936969921099,32.92314378905256,34.87772110436899,36.61178445365311,37.03380952744639,32.710189044701664,37.39512385596305,68.47857455083569,67.03069941754342\n119172,40.386836668428806,30.334065044460857,23.320289484906674,23.03475860327339,30.385690536402436,33.18050039034813,33.398347248054236,33.19905202395728,30.067608682661106,25.397935661710832,27.239468591558428,26.21470622022356\n119177,756.5045325548057,630.8382867390408,467.27071108571135,404.0155262965835,178.6213527022212,158.3310909700494,121.82755425984672,129.07410872416955,161.9229615559763,254.4685745684211,495.6029471350306,514.0994934101852\n119187,60.85024413452588,52.934459381705274,38.935343095353566,35.68010811690286,26.91497704746182,28.064973032687107,31.42383485689123,31.64196690216445,27.283245909057552,27.03943681064268,44.050361700686864,46.24517204019309\n119195,195.09822537162157,174.0868821801147,136.74224139442478,129.77096472978715,116.46865574034445,123.75128149672702,138.1985490724369,139.66823064170214,121.12085006058327,111.2338869647162,145.7026073729146,151.01992198984323\n119202,61.930588344952625,51.845469430735506,38.50327346741851,35.69311557722943,23.694449125908772,23.0169482076953,25.289393881428325,25.761325676785095,22.805526439653892,25.073223570595175,40.82499180145205,42.087990407121346\n119204,21.601300424513834,17.174176218329727,12.155951846532352,10.861494793896846,4.270157777232182,3.854411250839178,4.331896282650152,4.483003085925774,3.8724706793094956,5.35067716461697,12.830041948326196,13.212892316078367\n119210,34.66387863180717,28.23392201558028,20.463966843039117,18.916512761782023,12.989387182773362,13.373426282676572,14.700145680370827,14.982335246598273,13.087329788521773,13.263936798423757,21.662786527290038,22.031096846149985\n119212,51.43646674286023,38.43429474431001,29.508084952751908,23.128503426243203,12.234792932742591,14.105587757672303,14.284728196080687,13.902356836286371,12.653667196505078,14.225441422980502,33.15748590743489,33.29140614342083\n119214,151.0545986331213,129.1198424658285,96.80506867560402,86.37707294071696,58.553881986899455,60.655936405627564,66.34339617989853,66.0965394973676,59.73356828369187,60.32052282551395,103.31513320478015,110.5277621235633\n119226,23.401583442321304,20.02919663152366,13.985233821990926,12.027771574261909,3.9503541399823554,4.060036784509577,4.470595034090788,4.536371859271968,4.092231279489557,6.008868735688797,14.340545456731919,14.828369583125511\n119237,67.0970567105545,49.617303449906125,40.92504180381725,34.90089493443479,33.60328619025229,35.622026220421034,35.380906740137334,36.90579959416324,34.105982335892286,30.64592401159463,40.13511870170253,42.19412047005568\n119273,329.0860980516419,296.03271791057483,239.97693889377175,212.4244936946678,179.63036457792288,214.8311166411448,292.88807803114804,319.97525452869706,215.9406268743845,168.0259279728011,244.0776252570299,257.17062865691673\n119275,100.55247688960195,72.79474693939397,61.044980871620204,44.930074646488166,19.769143313501754,21.393368856928323,21.23116010630501,21.991041461702327,20.18852534149694,28.313912289619953,59.65051685521453,63.027210092874164\n119298,198.82080435215894,174.66451770615535,127.6893789413873,116.96641293525546,82.29139413525306,88.53586162477829,97.21931135591568,96.56108158445996,84.96658578654666,85.577307412773,149.4482905111678,156.3944443543095\n119305,16.865523834927604,14.169724765076277,9.737075783522956,8.45204324921988,3.9610754956465133,4.225776158966075,4.848911604217167,5.055985054577638,4.291440650111098,4.410224190316714,9.922724158380081,10.690308647491863\n119310,322.93093313439135,279.433232845483,206.7817421465317,195.60147128292195,188.46698865491413,205.09317525487228,220.8637023677701,222.97427770694244,193.35591213243464,170.35824718958614,235.738900155079,240.56785765396532\n119333,37.037932401624644,25.159471886826868,18.93504117323214,14.576391035962706,11.978368285819782,12.85989326047965,12.912223667480763,13.478360485357706,11.989260583251495,10.538065567441437,18.06826511318996,19.716691031402576\n119353,810.6940723418179,579.6574513897564,420.99265962070433,292.05050687511255,214.04428824682137,260.94402410526646,225.47016038333388,215.80255982552234,215.90139457981803,212.33667735509644,523.9242288523681,535.3494312179515\n119359,12.358101211464637,10.584301660587968,6.990272477832231,6.3729180544930095,3.8213797259502886,4.109476170196213,4.358853827193624,4.339675868405098,3.806174739852989,3.758703607791847,7.311217902529853,7.630347423453788\n119360,93.55171212900915,67.03629777791623,55.2297148799366,42.48390541019372,24.64673632125917,26.656298742760814,26.665773666484384,27.40282634367737,25.39329318015744,29.097858394529595,54.25370302216661,57.667518143063326\n119361,33.774554692391206,26.831117683043164,18.277240591043267,17.08042830518389,10.311789768145422,11.041973286809773,12.708249497715991,12.787638824087882,10.773528674204615,10.258360997363734,20.22644693848908,20.413809427829936\n119395,94.35848758486428,67.6538348155859,50.94711987832792,39.31386567105585,25.228249268201086,28.649149547984724,28.223411938595735,29.03544032793837,25.669663100986085,27.087902958713364,57.92867066889614,60.786754158472554\n119408,23.769688968988095,20.430456609762935,14.616444058001377,13.049374273628617,8.879990592470646,9.406646218175464,11.031015042372571,11.09141958047908,9.324870070799884,8.092914682663995,15.168693351711893,16.774837436197124\n119418,60.88877661561855,47.38530431833094,30.681572426793455,29.458883493604578,24.917942669033785,27.350129878373764,31.119006657228844,31.47130220890116,26.322178779657158,22.258910549929197,36.19131977613807,36.30284726719099\n119435,577.5125472113049,529.4515288560839,426.61500866342396,369.11320159458825,239.04421180869915,219.5552329781862,196.04774989987524,198.41697208811073,227.80970246340365,261.95484652739503,452.52932448328613,471.2481704235503\n119446,29.77447822352515,24.966382055122807,16.543934362003558,15.218282407962086,11.411190899400255,12.439557021472146,13.485325902041302,13.609341326979854,11.808471479835177,10.739930778552491,19.08114937242427,19.720570292319167\n119449,25.927853298589035,21.1323430691856,14.347813071250853,12.693514304896912,8.87591414820761,9.873833111047578,11.77850260256331,11.961619863983657,9.847291697330084,8.053760116239163,14.49085521065603,15.849620205204472\n119452,109.7036651421054,91.3473517407255,59.88680851273234,53.130372464361955,36.31692477077119,39.76052381691539,46.93108323526532,44.64638555027328,36.40118533882825,36.784667178204195,78.75164014110405,79.62070011498194\n119473,66.23035758089038,51.86283065707879,40.09960692715708,37.81083058190892,41.78530215727714,45.44939683816138,44.93948755734091,46.160304294893095,41.431494870228654,35.09030730006159,43.928157033286595,45.91127694081103\n119474,368.58739364503816,309.8307874651618,202.26192776604032,183.79063509150086,130.547107820588,135.62770748433184,146.91257585747644,147.13139757811751,124.74356118930953,134.69857777968875,251.87085368664734,254.2900389785152\n119476,273.9498339602833,264.1453543693052,249.45837843830287,241.22223617097043,250.32713593172846,256.7261608543313,261.977990057948,262.3394460360388,245.1225859891326,240.2257359567012,254.89883084504646,252.93507079910984\n119477,18.623357685108054,13.252981592644868,9.940212906911476,7.618337616553247,4.595672501451218,5.555478425414709,5.632222051502273,5.467952296498443,4.828702085490358,4.855081816774424,11.292707799736485,10.988635586379896\n119491,147.13283833444623,124.60828998941307,94.83482988992313,84.88024404694815,48.43397418665846,45.1174740866009,46.54784649354686,47.15851405430491,46.90397855228017,52.61074030948634,96.34159053517531,101.37922208977871\n119498,27.00351589718193,21.39300512151443,16.627724718648505,12.566234119138072,5.56273845544205,6.013597610619788,6.191432784839806,6.392461169633804,5.810955741722243,7.257989178536253,16.098044994305276,17.364683916172705\n119505,230.07380337977625,200.26878132538738,144.55748964476223,123.07036883687847,72.43724499120687,75.42350159567512,80.35994055322755,81.32960308786751,72.6984801519541,83.13878199616683,155.85852129998173,166.33029977002877\n119512,23.49902159222883,19.341303570430938,12.302178686663233,11.635756429490943,9.687855058407514,10.444217463895411,11.271379445565561,11.524223664119276,10.14630451830322,9.054526630153456,14.139906490874118,14.616613056588942\n119531,24.44414339389064,18.842629659201574,12.122235580881659,10.88165299410984,5.1692923355101605,5.631660807375635,6.342328816400802,6.569422576176567,5.630656353895249,5.673042588647081,12.422418711443735,12.958249981853102\n119532,218.9663749626614,193.40550969324423,134.1483165071861,115.1621397695806,64.23846449682027,63.59787689280754,54.755936677336564,54.66597039758463,59.33337803408595,75.20850753486805,150.93710438445638,155.83539439217392\n119543,11.122866072101063,8.900580158253186,6.7551991692765885,5.226849340063393,1.8422158010994758,2.0493919504995235,2.142575675121394,2.2171264214807316,2.0244614661678946,2.792061646163441,6.682571827476731,7.109245891719696\n119549,96.87431007126922,81.69822735721723,62.588805125134414,55.387403144419075,37.62820766826178,37.45817510038789,42.11878806007143,42.41076880253862,35.42310088387682,37.19137628245896,66.90878886502452,67.82626163144877\n119561,453.81346976525805,376.2637899452169,267.2046685685667,240.470947669718,184.57554505860116,193.96854698436303,203.56966586164066,207.839875117691,184.29819897704485,194.68786062677793,301.6759980203769,306.43351881057526\n119569,45.73509830559374,35.69594025589596,23.452469200291738,20.57274099447012,10.223130802241027,11.03657103379236,13.01067240806151,13.09611748529283,10.535367620004653,11.097142416869056,26.336278162263234,26.9709319744454\n119584,55.45431446526389,46.17529282822895,35.5523560315554,31.131096037351927,13.429291541692525,12.49693825580423,14.102437014096493,14.373448416856343,12.591586925416477,16.5645207925538,35.63882042967478,37.00541698673558\n119586,95.18944008516523,82.67225488548108,57.42763046157646,53.01338629962242,36.123035227054736,38.125193253807666,41.39187197332154,41.03411911812347,36.105584371356926,38.15031987653261,69.21982888290228,71.19463145535578\n119593,30.34281227275509,25.813452304311486,17.391555116332356,15.902416955554234,10.962184252285226,11.120184692314272,12.515113039345119,12.59991926583169,10.892433614073221,10.744593201729248,18.798925429495576,20.453905780940396\n119611,23.680749905419606,14.832147239414487,9.88607983022618,8.80709698177203,9.8852516977782,11.134459491532493,11.529069994903674,11.16932344524263,10.209826240305516,8.485719662223328,12.029217104502878,11.850285712754035\n119614,18.460587901113534,14.931318496307602,9.472476493363166,8.8089856503365,5.071813729005105,5.272956332951603,6.459846459839628,6.454875400290735,5.146001256675943,4.89069677684743,11.33054061637981,11.791399556851568\n119616,9.794205349230259,7.970429971482046,5.936216650694064,4.939938572733898,2.9787855783595005,3.302936184128766,3.3689647413099113,3.434784131322876,3.1241217106241743,3.2437294950628224,6.124614440085293,6.5631173659477975\n119619,65.0221160091221,51.30126430448996,49.367599014692374,39.4258110068827,21.974188249715727,21.933724667534285,19.868619567995864,19.631427440023412,21.83537446011362,28.74628339400658,49.73269958007786,51.11568930227107\n119623,46.53801703761921,37.94902244125072,27.48039616117769,25.29957119849262,13.201554336782745,13.481239312178307,15.61730233115247,15.911876309426399,13.5533632479912,14.392405081948905,28.84169754878344,29.547085124499723\n119627,12.356159663042856,9.321766968810039,6.891011116310867,5.682789500412623,4.6309505600732965,5.132699660132351,5.225263599405544,5.125307593893927,4.687051114590088,4.334459855133948,7.673861866199715,7.71900498533654\n119629,531.2021116042827,457.50312365700023,406.71114793953,379.31680198029335,247.7449243150719,242.77067678441392,240.04615706925233,240.9102059221753,249.6776602587328,320.21464266887597,420.84832411601695,416.75290435488273\n119630,47.40845738919582,31.973120286880008,23.75497200238816,19.54116664346268,15.360266572184425,17.42008832866567,17.507894561150497,17.382280770312807,15.600299960908613,13.623878025653505,26.817795454330877,27.37728099829345\n119643,70.99181934650628,57.23452480731465,42.40214456816603,34.2260864882392,26.211678441044725,28.90360503100713,29.277710505856767,29.664069132889455,26.75355705432845,24.989555946647634,44.41953403717708,47.69411075219881\n119650,6.599784795871553,5.409608367159637,4.3719111760290215,3.4591385872292744,2.845599018198654,3.0941984922715684,3.2208645806505425,3.26866158087081,3.109356138912949,2.8068389496352473,3.899753080108169,4.253946899959337\n119651,88.83303977427886,68.4840147019567,50.41747812094107,38.99423953109652,24.552553257381298,27.906198455878148,28.047678468358846,28.82458128242003,25.265170438962535,25.853205642187454,53.64025383858817,55.3514438361109\n119655,181.45605668853509,148.1708788724353,110.0412800320292,100.53227285856161,101.32792793576515,109.50077777850238,108.93450537949415,111.26934295971697,99.8413594520611,89.21725563923464,120.34100015814761,124.1385963294929\n119665,27.36274160312519,22.376035600478748,14.772699713827844,13.505054961473746,4.105446369766874,3.192185897445659,4.063104580957112,4.096669736351103,3.4741294893183157,5.55085291985457,17.05718035825314,17.818117284483804\n119672,47.1103123203459,39.914281489696926,27.9793350085514,23.617172707871184,7.0475947705194315,6.209208188409996,7.266265066433161,7.475702218507204,6.278485615399703,11.77773787916523,29.329525888575382,31.150540998482715\n119673,13.649724524634927,11.588669226557053,8.31555142116421,6.740930949257945,2.2439029882757597,2.3606228881465965,2.490085974056609,2.46856048680777,2.202091525179408,3.3547699398768476,8.503872201505576,8.548372744119822\n119676,593.4478461705887,506.82582559785754,365.64082696734323,316.58284271012985,226.73367449402377,225.0711229627989,237.90395361739232,234.64602881022375,227.4820847187606,238.88452038378708,443.1948656975625,455.20929532974446\n119677,254.07851071993187,205.16648625289375,147.73061574499815,127.77307718610581,70.01040358340101,68.85846658351888,58.36751575967323,61.22908820592731,68.24496786697277,79.99201756188049,155.4468294809143,164.76499035832464\n119678,53.376255611559884,43.30646976395256,27.057432613705632,22.187160795960402,13.63796378439666,15.4627203829093,17.680643758803505,17.403783624828883,13.878477592265721,13.119310752507019,32.93877728269017,34.76030749376959\n119682,44.17663171459948,36.67890872588274,26.49305285120136,23.14647908422073,10.491286261548419,10.504709303987747,11.48521718287031,11.915408262850438,10.34162219097425,13.179338549227287,27.50486084564804,27.905731003842355\n119697,405.4159374709177,348.78147321223076,240.2960526020707,201.2124633010268,102.80988714306939,102.52463875100186,96.79976564025394,98.07465994008753,100.82115873820581,135.14278299266508,274.95475936573973,274.6057144569072\n119709,20.36887190534062,18.296472526333087,14.322474176026741,14.395476556294666,15.30988357309298,15.491450085613375,16.405297148368245,16.721127984593096,15.078923365525343,14.38912243621833,15.767427096283459,15.648540349147853\n119712,364.6547792670361,328.2760240995263,258.4946872950032,217.2255975337274,107.27162469323578,83.42340697960448,71.87388710919704,73.14578536903885,87.75862922862686,141.1018881466127,276.42944649859203,286.5691600395151\n119729,588.4622604297475,511.32200143276,368.75597789387626,318.01130934381985,218.36116257506447,223.5632764302067,195.95433142772194,196.25859067125265,216.64157680367595,233.55332444454916,421.14376004492846,441.6377759788052\n119731,409.1540957229256,285.7359787223372,224.2950418681906,164.09125122024213,76.41561548849303,82.25300932866338,68.44325585920578,70.17447424966271,77.75677628824931,100.80252573979882,229.349541249406,240.94134867697142\n119733,69.02837986882362,56.943528477213455,39.14544828174832,35.97555243193631,27.551541897853156,31.3858496558689,36.66928479632488,35.687231398050876,28.82750578817122,25.795040168337486,45.82753843304174,46.37352842394518\n119748,1013.3050133845361,844.5511948610138,597.0296714382048,510.91427733156377,343.19775705449945,329.94605267787176,305.28023307461206,315.60117777523044,317.2935743763397,377.46069912325845,702.7737170977902,719.0857509657416\n119760,26.26270536082106,21.510401285905335,12.79613192167802,13.254076939819534,12.308010037186365,12.78686730147917,14.131687748380417,14.063987875735807,12.130126049867291,10.876818267801784,16.815651395855326,17.16418228910833\n119763,39.374643304010284,27.118178333090903,17.537789385398046,14.861075494814948,14.354694365205749,16.734022209344122,17.069117923349584,16.540708206383133,14.805990622601067,11.64005055808161,21.12518536835523,21.25991690156516\n119766,400.75211923655485,328.8235945631733,227.17468459379995,191.42983973604504,122.6405271780808,125.02981840448386,112.80496442504591,114.5613974261126,120.88928841095547,127.4399020150246,264.7877476042972,277.43929020863794\n119787,15.174843124398349,8.905551506182887,5.278927773659745,4.447817233578361,7.4334385040297475,9.512586774244557,9.957551572937533,9.237499174308152,8.142353230275798,5.288179802171382,6.641972388959875,6.829789211046054\n119789,39.23919025741763,33.139000808937624,21.012749158772486,19.511707768303296,12.113631911535581,13.004719017352148,14.099829226150002,14.093885491239012,12.092565096957784,13.031845123278671,25.423763560089764,25.665602142543356\n119795,24.855346981510827,21.900051490010803,16.559447847046915,14.504986032301668,8.035256461049428,7.63318837991199,9.16289169569843,9.341027749496147,7.745042609811724,8.664765087977083,16.722687082353147,17.74099977484948\n119811,23.12578403711017,19.309998525499516,14.252317756638494,12.623856871028996,6.016841375796148,5.909607042384213,6.70381127501656,6.7519967989999605,5.879174496043599,6.940097705672458,15.878953679484617,16.536460716121336\n119813,16.997227864424456,14.322305732283931,10.416243446953676,9.594622422345633,3.931620100526865,3.4449380644722023,4.095078850512249,4.200639418463139,3.6618420702339836,4.423564298239698,11.07906715517062,11.652720265646913\n119817,42.14840307348114,33.32313400487,22.39288560445507,21.6003437117543,15.621469037150085,16.912246814651844,19.018932986508435,18.958490323469032,16.383873963074866,14.69151055790045,26.718064488472702,27.108750530528567\n119820,49.46883820762461,36.08337366176655,26.411353776453698,23.210592388061766,18.06042496476938,19.660256293296197,19.886717921406834,19.792146091936992,18.06275125713326,17.831757794461275,30.66491190694899,29.572270284871927\n119825,65.85333880149797,56.207483467488544,41.12359289808044,36.670195807334686,26.255934742763163,27.91592578783515,32.00471448269772,32.20506785840375,27.065239936538383,26.31710008348581,44.07550242243322,46.02158771737055\n119835,49.0279247741239,34.217811134295935,27.1043875160701,19.249836719917763,9.249904955886421,10.422049867215069,10.491124872808156,10.95684914314106,9.893079930352846,11.63245765716528,26.075809000225135,28.71700939859958\n119840,287.3312061174019,246.12385487552802,182.513921372378,169.2418989056183,131.65198537671392,134.26498335475947,140.81978153167415,142.9403366332198,129.04911715469612,141.46373581145477,205.80237855574273,205.54924774759604\n119858,113.12209824718855,94.51617294207354,69.23616103305136,62.598259396873964,40.65016802638407,41.03242512508794,48.371848893153455,48.18913720162922,40.25302332437049,42.54120132658649,78.61216303746636,80.99234533929267\n119876,162.4735523687604,117.29121631945463,85.2003057076717,76.77329697583453,21.46139189193362,21.26187355153401,24.465186863202295,24.719808263149705,18.863178489882046,34.92775206085785,93.36659821194257,87.18628288269407\n119892,322.18155192492446,249.09531964749848,202.4451686020714,162.40548857698514,133.92177770189343,135.19964786861675,110.14708581801409,106.73531720652491,131.897654027315,146.20899026985444,233.5249991575834,238.54574806986878\n119893,34.83369817525163,27.210568669072565,21.326528899277346,17.78088064230672,13.807762246937273,15.580720586164638,15.705740049678889,15.41687728107367,14.05917902012073,13.247456896158859,23.535468434511092,23.754770674364803\n119919,24.18657655909141,19.234848625599007,13.261244259212605,12.285359215699527,9.006238082727425,9.57970963887891,10.799013663733973,10.970727467520993,9.422449942529953,8.628063944460719,14.157893339321467,14.551305037352305\n119928,1597.376674229538,1423.2352487435433,1180.9249979757053,1032.8147698565008,494.52823389452476,449.0421282339221,358.77625615612203,368.4569850063933,441.69526458723897,663.9549717108156,1241.254202840173,1260.0573965871038\n119940,24.50476023878035,20.746039738985125,15.535079451996056,13.055821330402285,5.004272250888437,4.2034208144387115,4.798875409286919,4.8898747941279685,4.223930871273762,6.188757083932384,14.943201132574815,16.198472773486728\n119963,348.94018295413775,293.92082912997256,211.3326516841157,183.75565607587458,102.11021119943445,91.85672260999779,77.11109770234428,81.13335302245655,95.49774657057525,123.7525520003187,235.1065685194244,242.5504344147396\n119968,81.38700954959317,68.93567024973952,50.6184616797012,49.86213789922761,47.18055686579917,48.923612186560895,53.365391743155534,53.63161886108187,47.42168960064145,43.351891898397746,56.595550706582344,57.19261802915667\n119969,121.19595732144029,91.04592672194093,74.32809521069919,59.654356940227366,32.60510805245092,35.00670458267166,32.91337809012014,32.13960143882073,33.596774149511425,43.620313594645694,83.75235502564766,86.5163301964049\n119975,33.187809589113,25.90616061532518,15.426453581867829,16.917778831377976,16.696343557208998,18.256148194162307,22.61707274990763,22.266143281592644,17.738446075334235,12.810494853424983,21.44468905448432,21.26154527900239\n119994,50.515370230239824,42.67101419085784,31.088823700029405,29.57324214552129,27.362472480488957,28.927013427558034,31.752748179257274,32.09049697297395,28.273710210516718,24.651277263340823,31.670314137766614,33.254892363498826\n119997,34.240049971544614,30.40987731164442,23.379024135676,20.7463877306529,10.938055359469576,10.182354318051127,11.157807376918655,11.451635964511585,10.446593385034443,14.000356393021566,24.771932523279766,25.50944828719624\n120000,76.42733327283615,63.395817353984945,44.20194784879404,39.685098017390146,28.193333716905414,30.312212506820345,34.30299991384707,34.106158352706906,29.406738573336135,26.933940160826985,50.52331630241471,53.68703533335914\n120005,182.2585995765764,164.3778054707745,128.62597402883173,109.8083518011756,47.85925361455526,39.74913589396649,45.29745120842557,46.94503976704824,41.01958151144322,64.74888523353962,130.9853147216386,139.78330935390497\n120009,63.34820813978286,53.57457036007033,36.522998893294286,34.313419907760704,18.121701205456777,18.774436897272498,19.702304170183258,19.985230752805016,18.206180134209603,21.91980533097494,40.554419493868686,39.88728539541803\n120011,44.455215671755425,34.78420329728407,22.028100641707844,19.131370720934044,12.509735450713185,13.497662650750186,14.805682962330607,14.711829532029443,12.853896937094307,12.767640235484794,27.213264710546245,28.208277267248558\n120012,69.41124043437597,59.721688508453724,45.16572417164698,41.96099223468702,31.277387642768563,32.70926377558771,35.915904341934564,36.267312762551754,32.18486745947545,31.147069163059623,47.97901239997413,49.79854760767997\n120019,1316.2808285588083,1145.3095620674928,867.326815577606,763.0025513088996,450.61714767039814,414.6031237870344,375.54317139710764,378.18375563145804,407.31312636372735,498.06163871250783,966.5843757944635,984.9082860351026\n120051,448.78256176193185,382.3906647245493,278.2164414980436,239.98346910284997,156.10909431622534,146.6633768578653,119.00347172226216,123.12997112160762,149.1363882277355,170.54662312433513,318.1792888025347,330.2093064381704\n120053,12.414618036164772,9.986115489396363,7.844966247265544,6.8189787809028495,6.774547529975536,7.471513279235358,7.650343501199551,7.567115269973712,7.057124821038488,6.013363858218441,8.12346321774876,8.408928786220546\n120054,19.479194387682266,17.41511820881487,13.36974316443034,11.55964308490879,5.3023224041361585,4.337352708805151,4.88954683987324,4.990072660577696,4.387081343401295,5.9385626368023,12.67990141368076,13.891639932730756\n120062,33.40394584746452,29.5456783106311,21.14276124266895,18.92915256984877,12.095066061924172,12.071552835836794,13.54886856793077,13.703320253527707,12.072244474659305,11.697091543939216,20.69551677575545,22.989926273015524\n120077,390.5460157274324,335.66543141607207,242.05578557032752,208.70154309500774,112.77258963325419,103.4849194203445,85.14112629504069,87.49302060505264,102.2288619720282,141.1148604878549,287.0279103190926,288.9824072220174\n120078,25.99248704845622,20.11408699645696,16.127667177356845,15.380001663022451,17.799576134718325,18.719779209523114,18.843673962901683,18.73579920498008,17.09308288591125,15.906580530393335,17.811317100326484,17.452764172697613\n120094,77.67279797194347,64.61713272507993,46.363099612130675,45.501849621188555,48.82779282872305,52.116778410517526,61.183233335994835,59.99227722326967,49.922280960108885,42.141769196026296,54.168779586520365,55.27226219615347\n120112,82.35788294179268,72.69501661720685,53.16969096705421,48.32837700450428,25.42903308021927,25.68474308374971,26.993632412708372,27.554525539254126,24.74101012775997,32.64578153865581,57.83408673481087,59.05784104243453\n120116,91.10403163358822,78.69016682242284,61.28423514473203,55.66566795115283,35.572315741906614,34.58756348355812,34.928953859523894,35.23161066821699,33.75985962235386,44.017784400472316,69.12973225933892,69.84839653066621\n120127,18.725044320112797,15.110731615613593,9.524426482225481,8.0261547890899,5.274861073564693,5.896056926697608,6.3690108167640345,6.51931382492466,5.553324034335751,5.081151426967689,10.622933910545571,11.37687610090163\n120134,60.94657079640541,45.45920849581137,32.033265129552426,27.892674206269103,27.70796126318388,30.621862014052223,30.992442307537655,29.860565907556104,27.044631020122225,23.506379344787494,38.74113284222212,39.250019882707086\n120145,17.14714701542614,13.32278180662247,11.325749674259553,10.022807013598605,8.79640722709314,9.377249363433926,9.481120793039691,9.532188172431193,8.975679907937339,8.858747820924394,12.692534962240737,12.532725989278832\n120162,32.357660316674995,26.66969937149895,18.890987231102034,17.980118820736568,14.588586435128146,15.401328392064983,16.77330772756928,16.873348205540168,15.119867254125987,13.669394237730982,20.94037309627604,21.478895112963162\n120169,823.9899257225841,740.3629949129314,564.4610834343731,546.2794905592175,540.2786740498481,593.9926023056167,614.0016139693788,622.197443063032,540.9804990158383,492.8957916842638,620.7598659451506,634.3776091390786\n120170,62.96884590893375,47.24215220979355,34.80725812862406,24.47001757428018,8.619740213094577,9.978371424663539,9.925558249470887,10.496140277353271,9.391887544790269,12.488226297319807,33.41985272826109,35.66471558445695\n120175,5.092667598378388,4.291228266625532,3.2302971046105253,2.933391601226626,1.4147896499200088,1.282077399858811,1.4722175924462593,1.5113142954421601,1.342569575979907,1.4204866645361311,3.0491658600219758,3.303684877231785\n120188,114.55099965718895,90.57228894440172,60.26911298635088,54.469860305437074,52.44436649431092,60.161228691124585,59.91896125389838,62.87864139042625,53.62441311545603,44.508771530682864,69.1437592533408,71.22410569349121\n120195,129.45275514589784,87.76628155279624,67.56734447402792,50.75721513017282,21.544671977546,22.772691280108727,22.367987195067446,23.71676977945047,21.52478048535087,29.22118957401,65.30811468877299,71.78478274243463\n120204,21.66028385169398,16.94060614477506,13.271499639855652,12.504956211313113,15.387166637341515,16.689269661643262,16.88275849060182,16.58066938017642,15.306781509444447,12.902364791870623,14.5228502916508,14.461369862973502\n120207,37.97927986073017,28.172865737143113,22.18184493440824,18.323316795231616,16.986646353860454,18.51773822250035,18.4690917200762,19.334040679001824,17.475999786942367,15.380573773847955,21.279739794096603,23.632750122284417\n120232,1090.0032360303258,1054.5163871965112,1026.6431616033537,1006.858779413142,1021.3226885608673,1043.9175244167802,1078.335285715068,1081.553625255246,1024.7311851689778,997.9154544624836,1031.4151038667392,1023.4577084270103\n120236,20.968522378124483,16.74829553454542,10.136228227328061,9.230861498993715,4.141959268593674,4.404320894088175,4.94627603205342,4.966789074623178,4.187785567733252,4.708130652587655,12.787020189472118,12.889027967482356\n120241,202.40509160292018,167.03749088721193,126.91569361094652,115.67538133291083,85.21270717086045,86.50804968901687,92.72786733511671,94.78604671573629,83.83026591700424,89.84041236079926,136.35996532218377,136.30963899531898\n120259,54.72689035106552,46.381032542443165,29.76248397434461,27.38849597568269,22.90333465599251,24.75851013957383,27.31648539656677,27.371671311260055,23.327782357297625,21.078685369980917,35.05697175644761,36.46553906184813\n120260,1403.0816499150442,1136.2848651970992,980.8556018267057,790.4698302397082,429.1783096079952,472.2377131984492,430.25861280176156,428.0148026069051,453.043642718734,586.7621753829663,1089.4035343833605,1108.2391478021568\n120280,45.45817210123264,37.94060845159234,25.394910847488568,23.738754592935518,16.500704901890657,17.355689604580025,18.574777300819314,18.99759226190047,16.74422167303591,17.51009954177853,29.008000437505583,29.402135559316473\n120282,85.8434138081569,72.92574535663132,53.764735988550115,46.747599189845246,24.48987432454331,24.142560564384034,28.054586570071507,27.875539979593107,23.99276338844871,29.094832133368822,59.30452980968314,62.43275298819962\n120291,43.915973527437664,35.99038594037178,24.766974189991096,21.001935569117887,5.629818428040681,5.1885766050953,5.9310692656043775,6.065135637512215,5.232102478016171,10.157430710822371,27.058232932017557,27.57059333781596\n120293,9.262494300809877,7.53678204087931,5.411154301670657,4.750697767344678,1.5215949667817694,1.3583520888166303,1.486919268238892,1.5369577826025176,1.3697209716879468,2.3112875278479397,5.6830525200191895,5.6942506123082595\n120306,42.248542671724906,30.515038294101537,21.627085156230986,17.79073265849513,13.26801773080645,15.078792661265407,15.175848671357908,14.798476418866734,13.50321444834671,11.772546579124844,24.9720947949135,25.21359839817278\n120347,138.50430832799088,112.83102203625434,72.958351310073,71.38819672062543,70.5030546758906,74.63164158487137,84.59625330055688,83.60921981688304,70.1015348203369,59.92593945469052,92.29517460829257,93.92209033909103\n120356,56.6908307193297,46.03129093157961,29.96688974391701,27.222632163769397,17.75626671860575,19.383129784123984,22.80121054799498,22.923872765722482,18.708920125817823,17.542691183552073,34.70412107143893,36.29673384810009\n120358,54.05311727489753,44.912359149651756,28.451361984887388,25.702261435847483,18.726741496881072,20.792364110862515,25.977186050336126,25.42873963490556,19.864589234971575,16.61347487858918,36.22053574550651,38.51231880928719\n120376,39.490151561945154,34.078780124549645,27.752870513007586,24.06194366331366,12.379947100131135,11.574324557088834,13.160723377414662,13.383053127040608,11.518928053629834,14.337540641900166,26.799388217397897,28.04979779978811\n120385,357.80287359119666,257.2356842634868,183.1640778810794,143.9890530882385,130.08298972714803,141.63542425923148,116.87865624099946,112.35168185064654,127.39286635827314,136.97248570395413,241.176607945562,242.804657806151\n120400,32.49593719411342,26.951069099252056,19.2701252949645,17.12174720579435,9.494244320863567,9.78578036025155,11.051818369001278,11.38414148267744,9.804151423570113,9.490079394286226,18.55063257624099,20.03015877166762\n120405,25.7742198902956,19.921318119890042,12.830347146832025,11.11197833853195,4.610776730640747,5.208638449000735,5.815215596590907,5.97404756031485,4.971023883078796,5.47106032341262,14.24142851216188,14.14810323662449\n120416,49.59384850511009,41.9851898436064,28.480814089836983,25.63802340085559,11.625708011286443,12.325412830033285,12.998908182610796,13.171107904132597,11.800525295173951,15.40960466406795,31.95948865430858,32.02270559860953\n120423,35.8394519300274,31.87571339581179,22.021010751294725,19.57684297390539,6.897268581372198,6.209386599910834,6.737186299385494,6.727830234720088,6.350246161500836,10.86916784504616,23.966439787159253,24.5691488264477\n120432,14.798788435330742,11.520300265992894,8.547574403992762,6.706411238313302,3.0591032175379187,3.3946273513933427,3.5294415648363477,3.727817529186832,3.3594786806710255,3.68423130862266,8.23873489979901,8.845459082804107\n120439,698.8032127246418,589.0397852473454,424.3798070839724,373.7491647209283,266.8851452415112,235.92552189415926,226.14996216497573,230.06913329982936,256.3572599537001,278.50221049109604,501.6840150840013,513.8068857320502\n120457,1809.3108280745978,1588.3170898380445,1205.2604849900868,1025.8686789881403,543.0128841545015,466.88457977807815,477.53333656472114,496.6283721363182,507.0233153473206,655.6154711615495,1271.1714992341576,1306.2909897163124\n120463,10.005904114745,8.178695403077569,5.703863809881312,5.002473076828789,2.1452405669311747,1.9329484882346801,2.3229296005703586,2.34162323092591,1.885598949694914,2.480258893817973,6.3396938077861655,6.601687918202526\n120468,35.79547027800116,30.148424514454852,19.598949287558593,17.179465014815626,8.893640691531477,9.919071626766776,11.054189618364815,11.03681688476535,9.458359615297661,9.890370462024338,21.500480254452977,22.522081745209896\n120491,24.42572692061208,20.415160708666257,12.674982405994442,11.0020920275193,3.5372948926269183,3.697307254605874,4.155180896544965,4.135379147214668,3.6208971767361007,5.301995011585767,14.416172584523618,14.86881465583142\n120497,385.9562321981605,345.95858190144986,279.9942932763822,253.44207368735175,182.5478618261347,155.82015912409003,126.45860959734607,136.29577276729034,167.30625865134232,198.58389791778805,305.5711884806375,307.1915364047204\n120505,281.06738410568,222.16636008042704,179.78622737111485,156.15936377298365,112.35790441477577,123.94031904200006,115.47971813935982,116.30671787729273,118.52979895946359,118.33712626991698,202.5797198358522,202.66763482198198\n120519,160.31193175250192,102.76479783154917,85.67238745635002,56.755080764638315,51.37511588545709,60.32078358774863,53.46615584063501,50.92429053964541,52.43145383647284,50.51519642939137,98.55581170886424,99.33961366361471\n120531,18.390263381727692,16.03701093479112,11.971446898809518,11.200455846979024,10.715627523681327,11.255976833406349,11.913407360776567,12.258268887525439,11.390182141772614,10.193286876001059,11.257505956781902,12.17425776157362\n120533,11.244594785421787,9.41403523203601,6.224029572512401,6.047530251227982,4.438622462403001,4.636163447354452,5.558308971844192,5.523150607158326,4.647532671873002,4.091915874808491,7.616410777544794,8.14203778430223\n120536,90.17558757002193,76.45198911118254,58.50580995987917,51.97319391451009,27.996866544867977,26.915584679791987,30.71506383971373,30.89353551096605,26.835252909525146,33.42608636395564,61.84453720592122,64.54755446252256\n120543,62.476544355911706,51.02730346364706,33.59903696683336,29.121581946974064,6.992331719462624,5.449347576822219,6.0958607827118865,6.111624986222452,5.655184227668615,14.36274572544884,38.52618097065479,38.557776432621154\n120554,10.271948869571464,8.434111123099775,5.6411861930134135,5.253277670089516,3.5351358345457604,3.717460982697072,4.2716102268739755,4.297140562947471,3.6899209100166237,3.4392707683981225,6.299660553130651,6.692720345724787\n120559,27.776150871683175,21.97050594157054,13.090267673585116,12.289757317680712,9.435985420964746,10.37081993787557,12.349044756544435,12.248307622440912,10.025894436729317,8.316043466620158,16.2881101857548,17.001231476708536\n120572,8.365590464247662,6.8130882166066105,4.685842257196399,4.173238388077859,3.3589064631408947,3.5965165378679314,3.7648360896983046,3.889391389353552,3.6675744079590027,3.243018404829938,4.28392854729934,4.852784338068973\n120589,35.18358800362369,29.25301786078758,20.272023628730732,17.511307329853512,6.965292509306362,6.7604892887081744,7.32046032802835,7.655322695126671,6.671888040888437,9.305821644620705,20.93501629779725,21.795592613963336\n120594,74.65699192113784,64.83123870594005,43.80495086475281,39.14194748420341,25.29213790360149,28.60233794532562,29.44787665952565,29.87272426336646,25.6352563737981,27.01380439527178,51.62880935782316,53.30494515470727\n120604,87.07166163621721,75.95267013039359,57.23612612910454,49.79369223801631,20.76307373692241,20.460553120612925,22.21392411217361,22.47847688042742,20.266376327775383,30.167380649636648,61.43174481723973,63.72996187549077\n120605,101.87480060929124,82.54008132541696,55.81305229516604,51.482795994406985,32.10274268099657,32.394029473895486,35.997981957926875,36.53338573669101,31.49163266218633,32.21395831161524,62.33358316108493,64.86045777992344\n120622,54.738151908074514,45.912459353309885,34.098253388504084,30.38555336747754,15.020786349124455,14.95606951401913,16.922584483132614,17.143069549218392,14.732742045163318,16.785143254035184,34.70949472447783,36.597123583320986\n120625,65.52903504753274,51.83962981972798,35.56065512132813,33.85061848508049,29.3368829487165,31.79932161942036,35.76115832635467,36.082545210880944,30.74123396899381,27.04760863851681,39.6067389626657,39.72196800173767\n120626,1573.4849471525465,1408.450891911316,1138.2330533542402,1073.9153804586663,934.0909196764057,1001.769134665207,1091.5298226665275,1084.4180602196698,960.299022516341,925.652602224397,1253.714236841876,1277.8593765626279\n120641,15.123202949471672,11.287041161390198,8.377607955226988,6.985709120510232,7.123272363387988,7.610712789294941,7.749860775800327,7.9728249161803495,7.331789714208499,5.830452177361947,8.140312947260622,9.4398878863289\n120650,28.951764542789622,24.257737783385522,16.73783040721513,13.805582878080838,7.244499129769748,7.894312480393816,8.613496421572044,8.85941691255643,7.863985295510196,7.776903398310574,16.65115410034471,18.61612390155811\n120653,103.24047784764454,88.03066539082022,64.76933598625074,54.977382646947305,32.9605273721886,35.44829825626117,40.99812276428237,40.38616365379915,34.31492195842659,34.96247154188219,70.42809577260675,75.4248980878452\n120677,87.62458689839188,72.3590671703889,48.5171644892918,41.40904990846613,10.94160404741871,9.203760908370386,10.435449725823775,10.544683219120223,9.435813893010826,19.66087530512516,52.16277020309041,54.838923948874225\n120682,13.033406416144702,10.871177465303136,6.151951685212702,6.031166549411081,5.636252732451992,6.353589418768584,7.201137955537991,7.286744101490376,6.162952937739855,4.902576486611974,7.136682141392392,7.513520071220071\n120688,5.32200686381305,4.135610862443461,3.131492226633054,2.169686422280037,1.0753162450207605,1.3265403842851735,1.407223200321774,1.3800876326387788,1.2788348193871986,1.1457732443984596,3.063991116062081,3.368359517069718\n120693,86.45000852420941,75.33157870551861,53.77829775006755,49.63509551177637,15.428782266831934,13.19761211405328,13.8754155992916,13.676677968251074,13.059251976206106,27.578926312594362,62.0119911025208,57.861667741631294\n120694,29.453620085853824,24.403340718585614,15.384149541882243,14.056584851771431,8.53830179355249,8.81960001213224,9.536095693354282,9.845725234765737,8.728812945603913,8.48606208952769,16.013651437454318,17.106296402476715\n120695,114.03055011037212,96.89291115494234,57.08467077326787,49.06314519450993,16.277533492886192,17.507952850093055,19.280186128653927,19.321479106176206,16.199967597513137,23.736896369239872,65.878553147763,68.89107147311383\n120700,13.020319180314923,10.772036102451604,8.334723492948807,7.424359448876903,3.977117771533873,3.746203610320997,4.247988990498793,4.402248768984024,3.80367849316193,4.174160861087291,8.056192979250238,8.352532581754769\n120711,190.23035016385273,169.12507740963247,130.56589431684793,129.85847185333824,142.59418845643827,155.8226248989084,160.4196116724653,164.01112667473427,143.77216390982585,124.55488877746029,139.79545091413985,143.65890886235283\n120729,2199.2605714103893,1788.9902926875002,1411.3172370667078,1319.890227518624,1047.2155659855316,1162.9964037286682,1227.1561575885462,1231.0680656190113,1096.4722267239522,1031.5960031963234,1610.9877774646943,1637.0968500138501\n120743,26.413921434424626,22.412734443010446,14.598862170527829,12.546979437833238,6.693852403886241,7.178045098568081,8.134515019208225,8.167123372406847,7.05209650399027,7.28109040348751,16.75442209946168,18.030401730965202\n120765,32.20299079839887,27.84616238550335,18.861518206671686,16.991583946687534,8.27780142073872,9.23805065961894,10.040765628726076,10.099163829458972,8.787203628585738,10.265967273869679,21.210012839612563,21.991188293456847\n120766,1776.278959100332,1564.1673773401292,1237.824195561544,1112.6581505222327,910.2194975929056,904.4262449640611,843.7766026632581,857.9129067309109,924.033065791251,892.9017534974872,1335.313182253877,1336.0030160582726\n120768,26.03610831735959,23.381850059494212,18.861804209534178,16.53936450772445,6.294065032546547,4.407065772424994,4.477401377079109,4.571930425517128,4.4769286400783805,8.094869463030017,17.70492282097272,19.12693372921384\n120792,13.796107697448745,10.728991153593775,7.9367129723365615,6.946701807571356,4.653036778807141,5.0379209158176925,5.090911832301518,5.231771016401848,4.84708101907894,4.630379069398034,8.035591635707789,8.470588219640002\n120797,15.32538603660064,12.773845711819867,9.265622344148712,8.079332915500023,3.8518921197213327,3.7563387161591177,4.516654135429932,4.554135677122299,3.762520697295282,4.188172609767575,9.888536872134177,10.364260328477695\n120804,20.255172979991823,15.742682359641346,10.061185321013252,8.791314200541237,4.239585350122547,4.427985658448648,5.25516535200375,5.174625659046106,4.319589161710077,4.494409697238019,11.641308736491654,12.233147739653928\n120809,12.557527744889377,9.134194916062967,6.96973281970449,5.539076986161599,3.484341769694858,3.7824857245271035,3.8398557662933923,4.008118653442669,3.649289051903707,3.467125960809891,7.09152734532183,7.725017276878878\n120831,76.41023835125692,62.52947563733199,42.03774817100167,38.00559005111356,23.31836775437241,24.66702473423411,27.212187292625124,27.156946430440204,23.21722955300523,24.911723605359906,49.20137850595362,51.65422133631242\n120836,22.416616411555907,18.848444288470745,13.25366444126317,11.895066876819884,5.459337225196429,5.1960181788058675,5.559479177107115,5.801980152145253,5.156944460552904,6.515631946272831,13.098527563835589,13.901761179557498\n120846,48.799266422845925,36.56049971926615,27.222001427480716,22.712575816777168,19.759684160041942,22.4474751895479,22.661033072420405,21.659364038621224,19.473441481316655,17.392717220769292,32.654134892338526,32.16507257534522\n120855,43.962196013655316,38.9000990599589,27.562608349755227,24.106274780391004,18.282600451151144,19.150405546850305,21.922008326136208,21.88141428499798,18.90961925967485,17.23125068390418,28.907885545585657,31.9535985212721\n120859,29.78949073259841,25.060143798548047,20.7576482413436,20.734567788590237,24.70411224806611,25.888787786700824,26.224543244277307,26.08752703656523,24.409594720814027,22.285645353478568,23.18940388147511,22.671174578736753\n120894,19.37558824924198,15.117724981222707,9.504785199844823,8.59076957177928,3.429574057571772,3.3905504699610503,4.042660402348707,4.056309524473853,3.3228112414788953,4.070895118405155,10.938242022351888,11.314100492018934\n120910,10.31529268694699,9.09385999919304,5.861720291952627,5.1458739963659115,1.4947154843719603,1.2541202355823156,1.4619448075010943,1.4563082130737255,1.339950419603678,2.39090906119579,6.243467410318894,6.72121591452587\n120923,649.5705587781969,571.9625530148755,401.2007546305644,343.88333717470266,159.68688976193133,135.4425428097005,104.99271453784364,107.59554516409689,141.42340206469166,231.32430614792275,457.39580239730253,470.3641289727164\n120925,24.46224050402652,19.329029700940286,11.040217858968628,10.001079815938294,4.484378415217004,4.748196571243717,5.5149917207511505,5.378107168476751,4.41079211576386,5.148317357809568,14.841472080612316,14.753268906834766\n120928,885.0465913791071,723.9138440904969,479.8306831287021,410.0696961362523,202.50839686954586,202.57228064291704,221.0668635624384,219.75581458434016,201.80325924576485,231.69926854441863,576.4272031490588,587.7883085359134\n120929,125.99096763734181,111.28077469643947,85.22873335276232,77.08525848979194,59.63959506750015,55.626139672819036,45.95377445113082,48.26363817027899,57.35554809244827,62.765705919675696,95.25220042581677,96.95235581782123\n120934,454.0350261025596,383.59920622734927,288.743311301003,227.76517569260028,102.94566769364803,110.2944665791823,94.89871131834023,92.61537325242915,109.85171780364324,154.47789858636,321.7696969963903,328.8852335384109\n120947,227.13000032411898,157.64102890444195,139.44431404734726,107.86455500042933,86.79166109535349,91.46964495067805,83.64647399965811,82.27234620208519,85.98191393449075,89.388700028339,149.74705024298234,150.23784620743564\n"
  },
  {
    "path": "data/seasonal_day_of_week_loadshape.csv",
    "content": "id,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21\n108585,2.0254250770765605,1.9560841562291102,1.931533183949555,1.917412930001181,1.870991230798718,1.0925988513773959,0.7894457489069683,2.887598553777502,2.720936101913026,2.67639685639061,2.5315321883218136,2.556103352667514,2.629409781488954,2.6832137530615294,5.329917527091145,5.271482018361444,5.025626244644134,4.7830092445780705,4.956289057962018,5.226189216815496,5.3593978286050055\n108587,460.13648332554146,454.4655870305831,429.0669926615647,434.741830853948,444.3445694574756,296.4047781320603,289.8482773574564,584.6887510009248,549.4712555789746,534.8738517530039,506.68922759391813,516.0401164100176,379.02916039070783,383.7045622550769,961.946887877912,923.5900582574556,853.5789984572455,814.7971665229434,847.0159478913032,565.5106771523241,574.1182964256424\n108596,349.3051858012125,351.64913976861516,339.39699724003657,340.86203974409466,354.02594736806464,161.56172109203604,161.2741492718686,355.1826957309135,336.8143729662707,338.6322477219383,314.64952648904097,306.7173766508179,179.8041818846871,194.25666781701668,480.5554884899084,478.376154453772,419.4752887073408,392.316949711079,413.74183899216285,284.85783120900595,291.8695182739237\n108597,35.3882919451245,35.06602159759464,34.34030129969768,34.30386245716926,33.77494669870944,22.612417409871963,22.685421349261304,43.239444950697376,38.62953368610295,39.9652931095511,36.40976761263934,36.70367227433868,35.835736390289775,19.51734584661798,68.27136768505206,59.8837345771263,57.795166242492165,54.14356983783697,58.59067954790773,54.61736632257342,31.9756765920558\n108603,153.22502892021646,150.9172102020028,147.99019615705402,145.6607580501166,141.12272076314292,69.81371946452013,71.64139957678783,192.01396744380108,181.68944693272073,182.65549694298403,165.3506843853976,167.39749950514724,101.52055039779265,107.29513907734814,294.1360350791227,275.0852124447934,265.7156501961676,252.27116263833219,263.4730290772284,161.95270858711413,171.24130798890226\n108651,262.59749259010226,255.3721833106096,251.26475705882314,249.0360321598906,237.55615028873416,115.1225046150237,118.77530688403978,378.6780022446521,363.40635587761875,381.06977533670977,329.3113349858181,346.9582797516481,259.2474971152651,246.3178264803726,785.0595146464085,763.3700091318086,741.5529425155149,683.8345232176399,751.3494385060553,556.8627551772163,567.1549251066119\n108652,10.127268413812955,9.163253910742755,9.005505309972945,9.03726223983043,8.71273917740077,5.933791172441086,4.694526454647484,22.989794655473244,20.18545206296486,21.35986956354959,19.08371881965832,21.17222431361845,22.034743342245697,18.068256682794676,46.62233340620473,43.50795413014555,43.120952934304675,39.729859207900034,45.23669244655664,37.85921941860627,38.34878108160798\n108655,73.90601884765255,73.82893535608953,72.64745345301104,72.20862848015443,71.2998656398442,43.92138945311454,43.48718195063771,66.6073015013499,66.39150558297729,68.3677609424857,64.56715558086115,63.77373712082829,53.552660667553695,55.402236066510206,90.87772729069177,88.67178301798003,85.3227635361917,81.41675747349744,84.94244189929763,82.23799406962259,85.46694148801151\n108657,23.729805601865554,21.770890905423677,21.16360466748172,21.17597439194065,20.497726817171444,13.078005954308322,7.413017977296956,35.10762846812391,30.554417277215546,31.89759055892492,28.233640680153275,29.623668581585672,27.503275260088213,16.14717485977245,66.05246326649262,58.93472671305305,57.49624014115838,52.94356960177182,59.321619315762234,48.21454379524823,37.64421816988864\n108686,239.89889047311505,237.21874274770454,231.41576233230447,229.6254756001265,232.60675155243246,121.96528140948067,116.52517621694498,359.72474512879256,334.3121241968234,330.7560139816206,311.5119491261105,318.0544090614986,207.41970540972562,205.4713554706787,600.1460089101189,555.1489484963805,520.3376993286003,494.4571541279068,517.0266486278745,304.6717355095835,313.2230118970466\n108693,279.70182057385455,259.123893757738,249.48030618917494,252.3920067943209,248.21451614492852,157.06566274700657,165.00912183437515,419.8648600845591,382.78479854465337,385.5969463282306,352.38883130223417,363.74609100906866,295.0750671774475,303.08952883971415,737.4972383428177,708.0877202885473,683.5154900475594,628.2630913497109,667.428664980431,437.8273793385781,454.93593133468835\n108704,4.968066840900418,4.741865767899927,4.661456796437187,4.625009509561324,4.659185435461002,3.184327100054403,2.1631113923831697,6.877681101949988,5.583370892655335,5.192599014275988,4.804886072398999,4.754422146250498,3.8475698733400554,2.492326902608766,12.364607975937517,11.025517140594458,9.028863647033214,8.180127621498796,9.316509575948743,8.018071724472744,4.745512631691659\n108710,7.18770947913055,7.101917177131722,6.997079734044533,6.985055461633946,6.832480309367554,4.777327992391224,4.8512893841980675,12.69373209963934,10.621263484384665,10.673556361425133,9.87687495334293,10.37494428045343,8.883143001920743,6.814833025063943,22.609942257101896,19.95272683070667,18.456392263710466,17.669525314523586,19.480486005391267,15.066331249054928,13.189799583619966\n108755,95.9070206596893,93.5947047874828,91.4697571825472,91.40425903562004,90.3049289710507,73.43487501610224,70.95470767221731,79.04857544523128,77.56068220351236,78.28424348000114,74.05669510961566,71.51175587578886,63.478058575168056,54.757727348938374,105.61569312615522,100.55751677672528,95.50777091505333,90.89794411527423,94.06576894655161,88.4517194736873,73.13696402774613\n108762,7.404593306679754,7.186269907747886,7.0023052845499505,7.002938404943717,6.865036773796683,5.484489868871116,4.765511025830028,15.352117507014826,14.121088650341767,14.071858340069756,13.199872491606984,14.977272654991289,16.644040899487827,15.396762088089547,29.829205385513127,29.383819250826555,28.02751568108138,26.631120365043895,30.260135362562178,28.54889691914914,30.517492007490475\n108774,465.7098600343708,463.2488383861801,442.612322020241,439.83519939836054,435.84212841802406,183.62565627453392,185.26840956007334,722.4874191697098,642.0689395719122,650.1427319246909,603.4928274984477,608.2618298164897,387.4957944179327,418.18207772537295,1343.4506567560884,1205.223530492279,1112.1264333709823,1041.1538618354139,1091.2394278152594,634.0166426932047,689.3369380318779\n108775,78.42987567605506,76.36928183731726,73.08175148644104,72.67524403617797,74.78780652315227,46.87093652126606,46.93831419629243,127.80395952584301,117.32713635277344,112.11385957836825,106.05147871435044,110.29855228654671,79.4710846629694,81.925619900538,224.6748969324124,214.77899908438675,202.19345281473514,192.31924793561166,201.66154834502956,114.58985533126463,117.91321643755357\n108789,46.22983398343492,45.07472294143302,43.82416217905635,43.47915497907923,45.703570061891924,37.60582981608846,38.0265922842639,83.11442677707167,76.74559737479447,73.69194325792593,69.18433533939844,75.37923637391691,70.3748382980782,67.49458499802388,145.9550403477452,140.0138254172274,132.80401979152964,127.37964602840336,136.48197876624877,107.56588022696393,107.69748738972136\n108791,90.29281553987268,90.12012725070656,87.19735419577013,87.49937043405282,90.4793172209559,60.8016598795828,60.582923573201114,132.1401614551975,116.07203763406945,116.30218877445976,108.17315810141379,108.03392516354292,84.75796535150918,90.4429793916383,215.12801462987667,213.94094052850411,198.45671548030128,182.5414792312754,195.07777333575314,151.93358095466502,149.01880219580195\n108794,9.115701553916669,9.163692114422108,8.838568006120333,8.855066556942882,8.733613572441413,6.636715911746535,6.520139737191049,13.825170785387977,12.055147712962944,12.020409740914964,10.482035768536937,11.524038751996732,10.416296379295945,9.845268822187418,25.308303787830333,24.577988684898656,21.811655977397095,21.082720280906262,23.4176451211672,19.963219398634013,18.473011902264542\n108802,31.931302557444074,30.445191372909427,29.306751515948235,28.983867826657697,28.741713608410382,16.894901714078326,11.37294244994304,29.67572235312895,26.644523591172472,28.15605119607112,24.081845190296416,22.965104404987116,18.24378713822337,10.944256217850883,55.86301414381038,51.631221512188354,44.17197606779382,40.24636976644282,44.5877332447307,41.06936978011022,19.986527442299362\n108813,33.84136374048711,31.296357270759817,30.8172631704449,30.645141548392754,29.791038186772603,24.42565145235244,8.91570668262598,43.09145074303827,41.28736659965772,40.5885352717334,38.11577170245224,40.71408142208731,41.211947846417736,32.180645339754435,81.68692755909606,78.87041539956536,76.3012356534026,72.56353127739789,81.17178672110738,76.32818523909889,78.03914028573391\n108819,99.06241796433174,96.03797358377629,92.33433824170983,92.87344563181682,95.06530381019576,61.776466507009246,63.21404361910628,137.29789978670902,118.39705360270646,113.90637621072337,104.40179813552457,103.08329654171162,83.68529407648295,101.34773036795573,234.41191014539294,232.67067267976515,206.5294283620427,191.49557987044878,205.1137847735613,167.92518836938189,167.09144827023863\n108826,7.1237752008313455,6.991600521665143,6.736357944542758,6.721189555726928,6.581446841603245,4.944354786611608,5.12157305346583,14.505645010137354,11.77568468664922,12.006888311743424,10.596480333908525,10.721302207935492,7.500799312671909,7.944765861355761,29.13591969713897,24.712809041944066,22.48139152029398,21.296709671442994,23.91935815579584,14.082219306805303,17.414119216767197\n108838,14.490332172069987,13.18475163101578,12.983144900345094,12.949771939242876,12.473714415572045,7.957313829119331,5.295824203155221,13.278600692278474,12.544614617380317,12.854025388096396,11.303417811977706,10.954081417475027,8.373772222685094,5.6810910511045805,26.16809078475935,23.047325564445593,21.865192425131227,20.36846861290477,22.07798237943484,16.88221688312145,14.642386046448806\n108841,28.847629107728185,28.028570137581934,27.0225771869093,26.996197405659007,26.534235153817114,17.66520747422495,10.993025515213931,43.49553424023012,41.15162745490805,42.110161365030955,37.80971626465449,41.44709130958297,44.93686476203393,38.50004402581993,89.12945788664841,86.42547551952275,82.21042638704539,77.86936488686209,87.75342677838138,82.24794932219274,85.57813562519107\n108845,11.649158596686156,10.767358322793811,10.926801333453488,10.942094164341167,10.56197755450947,7.388295785701625,6.415944172048665,19.567616681655128,18.16221936109798,17.95246521277475,17.028776866023957,19.42958602424143,19.98143426441536,19.44998798083274,36.087273667293054,35.32280958943444,35.2475127828767,32.88335322085741,35.58271192834489,33.449744604644565,34.735852934674355\n108860,100.90727591314058,100.2499131254699,97.54441857439701,97.22137085619619,95.42423167567351,80.31396594601937,73.23818957358105,110.92363724662455,106.9094404743625,106.90018286087643,101.1164732719116,101.25181970046299,96.9068032230184,96.31182848013779,180.92867891565103,177.6487502583528,171.6811541371372,162.1116568885472,168.413556641624,162.7407669650892,165.0010777104403\n108880,100.29238188609939,96.20465352818196,94.74311312849812,96.17957041466589,97.79513967720415,85.26735497232704,84.33116041406637,148.01244341339148,141.52797169727657,135.39417979883922,131.8270954073933,144.12757078165944,144.78567030247552,143.42659206401385,185.6353356156113,182.9142482429409,181.95924055182,178.1020424848817,181.3534307261594,176.13749993234427,172.9030774334527\n108881,146.31625094299034,141.23175201228588,137.05161190817552,135.94992656775688,140.10872817079695,84.80444743468121,82.56770035180678,194.90144284019348,170.81067288303987,167.23417001535023,153.39150126748498,148.78944789410093,121.90155189241828,141.40674472089776,333.11108683361766,326.75994956256176,287.7170230777477,259.9839615893027,277.1544023507134,236.94524372468788,236.55667423204926\n108894,37.31993549973709,35.05333299341603,34.05188818281038,33.91880802842553,33.008289757943324,24.09200193646839,22.404370970089804,35.63141996240437,34.53010584126586,36.337799594689535,33.113331738837665,33.08686795081492,31.23240065081194,31.109134234477278,60.03233399501208,58.28029346708703,54.967341123430124,51.8254105591229,55.79337439708207,53.124511658917186,55.783839290708954\n108900,881.8690359756482,850.6094062451391,848.7188705038362,838.1592058389108,814.3594599560579,607.8843810760717,359.5353025570729,686.6205948021901,681.0731536502944,692.782105767622,643.5080310465705,626.5697753287251,573.7561169407703,331.79158816682667,982.7121909792467,930.6820767483722,902.6702050537646,855.8341727422034,893.0604385236845,907.9094149071723,561.9337754034669\n108909,11.533188650423854,10.768369690241608,10.344765865212795,10.249377891386487,10.30524287254025,7.760598516134671,3.9045940361432008,7.163631939811852,6.634222550744068,6.655110815996521,6.084402306934177,5.581325641364803,4.359913234557824,2.633986326446715,12.130268303477035,12.158138354770912,9.585713954601047,8.382930495522626,9.498733498116286,8.59284289741312,5.4529481834565825\n108910,9.249176683621641,8.832486517144185,8.476434661630941,8.489660263201786,8.65011577258063,3.255603871348656,3.3935227976531612,10.931711566355748,9.325160951818539,9.269515702816346,7.994951325727435,7.329239020249833,3.6310698451323273,4.9419685790286465,25.19408841197601,25.348185627976157,18.79803871404774,16.107602837835163,19.67930394359642,14.181371029531498,14.718584245972396\n108918,16.861099700195318,16.33549500001365,15.741117081955911,15.780443409012404,15.983584431988909,13.53634156195887,8.616499163569642,18.48520478605837,17.41081975109419,17.144947150573884,15.225816110677771,14.780184776811527,14.159423534301146,15.273152443864303,34.29828283992514,35.833939546459085,29.101008523144213,26.133686357429777,30.935267905705707,29.404142673052927,29.620026313874043\n108935,61.87844147275615,60.27713896762858,58.02922008908683,57.83659916953667,57.340445361637755,49.01359343300146,44.96950749266616,88.1932826441529,74.76405871831734,77.26097852467981,69.86635543720084,72.18618448581789,78.93409481892495,41.072924699227144,139.67654300601487,130.3732892137217,125.27352812379365,120.4215594260665,141.00465234455,118.62545560080594,78.02559188851697\n108936,6.206213166587317,5.9839588787821745,5.726441257543728,5.681824905415258,5.60113777649213,4.323473858600295,3.098593432453398,12.909333898112841,10.43562475065861,10.93168958024203,9.39210488753675,10.915433351743744,12.04646229504288,9.258498777239662,26.100262128741562,26.058485048001085,22.663312174789958,21.869809669556513,23.682747520081666,22.65325648327463,19.193941176439562\n108971,13.068351554417987,12.037099580460346,12.010983370982538,12.058007157669074,11.64757931288179,7.276836482625438,4.906205110473968,17.257563929010285,15.322831522205083,14.813262049888758,13.703365369505558,14.418082532765276,11.449297799124546,5.731455615341662,30.95860824100443,26.82289325515058,26.225849614049483,24.757665004690093,25.962571197743486,20.723024917884867,12.585095321126818\n108977,31.170671546445238,30.669609740080496,29.740228237927127,29.677365002178796,29.276249191205473,20.300556900130594,12.931028139043203,30.65020166125155,28.833513695913908,30.03469461901896,27.705181674902107,26.547288303135094,23.570024698137466,12.093177658195204,50.38577455736795,45.2640172796772,41.87260688449894,39.89107022872245,43.38923826289265,42.46779695062343,23.20321385447258\n108986,277.8845096853259,271.42289532253835,266.71881327003285,263.55006742038694,257.03119999099675,144.43195470010923,151.25794610956223,337.7504270682953,310.2439332975773,321.2190121893637,283.6139041256556,285.2747139513267,215.4569365297223,220.6526759344362,584.8317308995885,529.6442539665029,502.18835964409607,470.1030601397001,498.43630003459424,356.52506405760147,378.45947892171193\n108995,7.242659835455605,6.548117297866576,6.482731657713286,6.506850813018849,6.328326002629933,4.678176738510331,4.214723964045753,12.255850596846297,10.613685214188113,10.690128832820937,9.964063327532942,11.36491373762326,11.41030971102122,10.034923217619047,20.43273623285752,19.165917456400706,19.08159149137082,17.721453276370738,19.273352123295457,17.257722538441808,17.25340342654731\n109042,719.3048149093817,661.0432311385234,642.6871453204976,648.1626835851964,644.2502463382349,478.18160163369106,495.8933196829002,1182.5129451876655,1083.0261359386798,1068.8462454771595,984.3573718219315,1051.3965831984756,925.9557841985207,928.0952277654407,2001.4514621049957,1931.4273053577433,1891.0138834914935,1753.5567332812986,1866.0194615770743,1362.1775276736373,1389.3702877183412\n109045,3.269373154202253,3.1774619822224484,3.0295136462239487,2.9977163410861,2.9891612969957047,1.6492021078389714,0.971627519955126,7.12465103963623,5.754307001830386,6.07933846124815,4.970276369755895,5.705678391002581,5.5407686705612855,4.82224656904656,13.762048447651136,13.83211752520812,11.847364662997002,11.07474821930507,12.959268742037347,12.607587137563995,10.84191429656118\n109071,18.031732447901177,16.821897806069426,16.460140116259563,16.457058127686466,16.042859526628828,9.830514221410088,9.478766818541494,22.94436434115873,20.916232370489585,21.78627014186756,19.35393606574041,20.19070145400058,18.04300757990048,16.841450165237465,40.375069143798605,37.62989820514684,36.701011141349845,33.77903126088338,37.71749318899494,31.29010604574646,32.28392247424792\n109092,92.29324103783857,93.12635618347932,89.61569239749154,88.80948412970079,89.89508388943763,56.343173404634506,54.941007325782785,128.3447559851779,116.68696013348625,113.87882056781072,104.82258732459417,111.96913248450697,74.44738078617065,78.61066098229273,196.75323794237266,188.4622387904285,171.16670698396777,165.8861175296047,177.65545233570305,100.22510796341382,103.77109923758253\n109095,269.35063236006545,267.52550795018414,248.58659761677896,250.8812937757009,255.48306489022966,109.82846532531204,109.60708247013045,434.23490884189187,386.30089130394686,399.5474047928888,369.5633719960637,380.9945219824547,193.47746775504902,196.41769504486348,781.6534884810087,773.4379504799437,697.7962345049382,662.8748775509359,707.4893404721153,373.1361396813895,376.32818216885784\n109110,1103.7347914250333,1077.9040867511121,1070.280962822657,1072.2860288137904,1058.2923352111002,925.9002323295247,888.5193475642246,988.5563414627745,981.8308458398419,989.1900086721382,966.4484737702945,949.2372740892993,875.7446147664398,812.0933502333959,1154.9815251788557,1119.4669351971318,1102.7297152164383,1078.2774905366223,1098.2988984456363,1043.5320843024292,927.6057997382504\n109115,8.235285709523714,7.722800960413758,7.65756523457815,7.640580274872386,7.461464703854359,5.1831554008775855,3.6430816503064625,9.171053021661912,8.149448474606027,8.099407661569863,7.478403021318112,7.266273176828342,6.390500223855522,3.4634898133322976,16.36594490503964,13.521100720194518,12.762090714965966,12.095725263443606,12.435883659728585,12.075955964268246,6.441162385883969\n109116,92.19345882012631,89.002708726368,87.68339758489016,87.43449795327439,85.45769530675952,73.04501269266693,75.2320913492132,139.65092727728418,122.25220209811538,119.89245482765672,112.46611479716178,116.50044061728615,129.99488905306495,72.81502335071588,251.84958646671535,229.30741796396387,211.30496427428145,199.51768594218353,208.44348593035173,225.61068679525246,125.75432923085913\n109121,4.504966832670315,4.327960410034939,4.3458086279011825,4.363092494230846,4.257215248382539,2.6784310279903445,2.310976311164504,5.682159372259796,5.403153535229685,5.523591625746628,5.066802162394001,5.145568850919246,4.912707718984321,4.845905157360456,9.762404018464597,9.322459542523095,9.081947314251668,8.485854463531114,9.270067212437112,8.782534316714521,9.651391916870226\n109141,7.687811966389961,7.436051499007794,7.052034457903967,7.014090354256705,6.855485176594502,4.034329930007889,2.9217133623114764,12.63029085681991,10.61298964464253,11.016615023182762,9.619320879471585,9.937882637005957,9.944329435766608,9.533267411431611,22.624953069026294,21.97325727535073,20.164181435721748,19.05562687697538,20.688825759861995,19.50410209451424,19.21034103960449\n109146,7.393353730341849,6.838064239986257,6.93652477752913,7.0175765256757305,6.7115173053348425,5.498230125862627,4.76018252136563,13.391611351513346,11.9661249010193,11.623539375044754,10.902720988895034,12.388025201308368,12.632446455202913,9.637168087863186,24.920620438333145,23.451183229027258,23.060368324559562,21.520142536284457,23.06796481374754,21.58150205247788,18.08851965512902\n109147,7.950350308653637,7.915980388106925,7.7010606650546585,7.674251633034425,7.627676175837917,3.7320890835468643,2.588310378059007,12.53408166441459,10.512753414168348,11.03567275425663,9.363670136815104,9.473742990147771,8.911208269896976,4.874812766033663,22.7317559924301,21.756893624364313,19.327235965751804,17.856021649725875,20.006675756369617,19.535488289521965,10.470077011385579\n109156,12.115979664318324,11.510890520268449,11.315461706780418,11.391042487919174,11.068319351472782,6.8668076383747865,3.278753494458329,20.953033264350054,17.886418559909004,18.855289740465317,16.61588556449404,17.68659048804796,20.112516617750327,7.752000164327163,40.18803960146218,35.41538327410954,34.15206618342725,31.516576893520064,36.008227056599274,33.00385767304279,20.83023528490797\n109157,580.0049417434234,563.7457872218029,548.3469261389217,552.5880024417069,525.6294346973458,255.0097888363794,260.70270659914445,1367.3220635218001,1256.068588848005,1346.1011019451257,1220.9716929409071,1344.0278627398118,1204.5622120897451,1141.0168006384451,2945.933062194244,2871.8733353383386,2773.8585811305034,2559.023675950258,2834.3423763192613,2308.774741669194,2400.972754150197\n109164,12.237927519986734,11.924900198162405,11.711714855258624,11.673065896697713,11.462763005862264,6.1157254867713835,6.38712785025807,19.424129039163418,18.060166203221847,18.816024680837725,17.268234425642177,18.095805870588453,19.299830733880444,19.03268371048482,34.66019468897331,33.718736011662294,32.71244327261888,30.20520267677454,33.36825502387895,31.863820753933275,33.36690689327701\n109165,45.075069095372186,43.93265292056062,44.267280524999954,44.34476296762309,43.39668961636174,21.824260363652957,18.261229713794137,46.084567585739016,43.3868548688849,43.709467699556505,42.2501672276335,40.86075483114004,17.921389356185653,12.222709810963398,73.72138386318261,65.68399006526597,60.31690156963384,57.06881056774578,59.90222488589385,37.453719049533014,29.97387661997723\n109191,54.06062938339671,49.696210908953375,48.98674262639157,48.7139419341635,47.40009364140532,35.67970626138203,28.42998342624755,69.24360937460463,59.01780213880713,59.07037745427828,53.59798587662562,54.38453647963066,45.778741831026146,33.158208552591226,114.94206183167243,102.82755390196137,97.58337637709673,90.21940982360508,97.88072186701363,77.65798743763165,60.53172370307991\n109197,107.68313348499026,104.1849534849669,102.20470077376562,100.96506106946798,101.91894445281747,84.76807342311086,73.65633673145906,126.09846085369134,113.93573195270903,110.757118423327,105.28173551779643,109.45045612908379,111.96711341478212,64.81087047610941,217.3142924345324,197.64761226629236,187.63286594160405,180.6116970346575,186.05813811830498,190.10226219892442,117.11692745063894\n109204,5.0364115293446226,4.685461808122337,4.53814715566106,4.5760232470817614,4.4241386220220695,3.116469206808814,2.5632994193068233,7.724320768987219,6.823365960867217,6.875404364393205,6.181995845768631,6.367481845051877,6.902620807423202,4.824381357732316,14.857698173971418,13.611965203328648,13.141572369779116,12.314486050600854,13.285078837499027,12.401186067447533,10.114516298614925\n109223,17.02199497902312,16.088627539844587,15.52009377461644,15.58160596301294,15.666026472644674,10.432510843284433,9.28742295766079,15.095640047747038,13.749179087502565,13.450100701894621,12.575873057559614,12.242406188079636,8.811493147587726,8.311630420076186,19.49651441470722,18.94890017710164,16.57355561240922,15.189249034884595,15.855519809046331,13.180388779679257,11.698766947177145\n109227,13.930544196208002,12.903570817097018,12.68272513689169,12.700230700355087,12.25944030469932,8.878426996536863,6.885107759553216,22.142262851051985,19.616738199569554,20.269191187617043,17.796975708370407,19.23742509033906,21.00070839397713,11.941003496799485,45.300579687080685,41.97888081282226,40.4664545577708,37.29338717425457,41.82175149783831,39.43290965334156,28.794639995968264\n109233,1189.9165465710375,1171.903908115323,1158.2769304055773,1156.9376453288105,1149.7907795045571,946.8583323135347,825.183167184146,1400.83407368322,1276.7215540617801,1263.6039569320908,1190.7098409226649,1234.015721569533,1157.867397664064,799.4343051128959,2080.7056523416645,1895.2075670767965,1836.1884297453976,1743.205082729576,1833.913863140422,1686.1585492974452,1212.9490785291941\n109239,21.929020485380285,21.277493324917604,20.973272850392608,20.93085083094432,20.41905936166477,17.6130427162526,15.686377317421675,32.007865723554296,28.94828610230076,28.515270049352008,26.835049313324813,27.75747599199336,26.645644788361235,25.246968399010864,60.60810221940749,56.61396636757543,53.08086456686462,49.959855499641165,52.44581038414739,48.88468098074427,47.23267087680085\n109272,9.981089759375577,9.896139646512193,9.470869301858377,9.376336657146808,9.374891170277362,7.989004023814486,7.62370719984958,10.864436191131874,10.057881180299857,10.197349910497906,9.053622982071806,9.260058328696356,9.403703341206509,9.370004435619974,17.64085262730249,17.861796571443172,16.42561220305391,15.553730712838098,17.18232331510623,16.80927632497772,16.45439471930132\n109307,16.305505931681836,15.915929423799595,15.582336786092311,15.636581179523473,15.73792815041293,10.96275130196045,7.615305580984765,16.303605408510023,15.238271073367184,15.032019050494943,13.928500011345745,13.309333234864058,10.932003684791825,9.178188786034783,24.99809499494952,25.07063026628973,21.788807631366026,19.901774034915707,21.806848472415695,21.316692500610635,16.25809614219835\n109313,34.111686788119606,33.38846291366018,32.464977399447,32.24572505486976,31.572380962392273,17.857203039088454,15.631113746929039,43.37127114037318,39.168631531062616,39.94844441618617,37.052695056744724,37.06826325263332,32.84780733959653,30.258644648707293,77.5054872315398,71.29528060079578,66.64999406989578,63.24247539343697,68.70162075727467,61.124697919244966,61.006072708199035\n109315,191.0046988001337,192.71241119703646,190.30385330077237,189.26152097234836,187.20454856061394,147.99394944063127,126.9768355017636,155.24830597466698,158.3817328830682,161.23892243058302,156.09062966723585,151.86859534357424,120.12700710339902,103.64359329285931,162.5778104757305,161.14047417808536,157.4740201677934,156.449818115525,157.4165472082731,133.54321932816075,118.51617409067808\n109323,118.77477215431406,116.483920090266,112.53348007881812,109.92646549425997,106.30828844323769,73.53413293112035,74.31408193934855,151.14855078271927,138.68821294392794,141.31759862742388,126.2617931956402,127.99508467352922,109.7578384903201,108.25172113241017,252.4400446892339,225.58891036066615,215.5503286778481,205.1100313537453,214.9922943383723,165.2718267769216,170.28285677390787\n109339,184.02256999008455,180.82377837857504,180.06475232664516,180.4038314824099,183.81666735873463,69.91303056554047,67.30215683914103,234.26703707315684,216.7087793500925,221.19654163126452,215.77237836773216,215.76644944325255,60.63657115369751,71.81907008716297,343.0789277545789,359.26882250803055,328.384180846916,302.60317766724137,324.2147497837784,123.8621628292053,128.89631405537145\n109349,16.346005903065837,16.12324215599969,16.115915529618498,16.136567956327184,15.85772899820423,9.471254040577044,8.766025550485491,19.764371300233417,17.89135662873633,18.285766442197698,16.67673223250564,16.915494860543664,13.965559492623347,10.801493854989953,32.40052761973315,28.43838922791946,27.34152567296559,25.692215823825002,28.335651900507173,22.62837500581159,21.383444958152527\n109382,10.400816083841642,9.949261722544344,9.602511389235774,9.542205229800535,9.274401153719852,7.188526248424903,4.28547862014442,10.518807672070258,10.126301177566617,10.491816355673992,9.411505515263125,9.092788215167054,8.338232158833886,7.36344957644944,20.297061986238916,19.403385019378923,18.05641928570953,17.199975085435064,18.49424845861116,17.250628524323286,16.734010140268374\n109391,215.60410573443826,204.36384210282338,197.04369601053483,202.6700820377131,198.93062683396153,122.94120985003568,124.97178875021339,265.2262042274556,247.19425426623104,247.8333050926991,223.15863866167854,230.92522390815833,179.8580877933634,182.39497065307572,448.493478768157,427.01399646013283,412.6237112324054,386.84862069394575,409.5212193264959,266.4452346870023,273.3801851057579\n109399,104.38716386328241,96.99169007657213,95.2197456766761,95.62616227780683,92.50310423304587,70.74145003312782,29.84978165132566,112.4926063067266,110.14231135238425,110.92941253866867,104.91412361079574,109.96840597315936,106.9420081086902,91.92018535097799,194.96713406752966,192.4880219235969,188.35375922829073,178.76768227662518,187.90945881350723,184.44278330603083,175.33234458200687\n109401,40.022303622200496,39.82359223112462,38.3876077117957,38.106521057858394,36.61649468520725,24.885113962519657,20.67097277703065,50.27398597453111,45.998498859291566,47.50497934995829,43.620612544431104,43.77045811657672,38.889839361132275,30.031465059837547,95.06722006270122,91.42461194734202,84.00177208112298,78.92364632452134,87.47464723031725,78.02153188141405,67.81074006988592\n109423,12.413110941435862,11.807823739586961,11.344507145737971,11.164349112522327,11.072495669506155,8.00831425771469,6.0292931049942515,21.733899495721477,17.76618902134094,17.99167092423247,16.219326709050936,17.953253977695386,18.065129891474555,15.481293871720698,38.00701236969315,36.66206529456538,33.07351835528587,32.04472432461883,34.034618044194104,31.094533188537042,28.19460512645009\n109429,26.144169631640978,25.084665575022736,24.621730265499146,24.52539472837417,24.062756617154147,16.889657833212908,12.526409009882869,27.90953344038749,25.79010104656665,27.362792477181486,24.184650692022547,23.559481135725296,22.140512439520833,17.048001933566983,52.858488965442916,47.99035270454378,43.2744435742642,42.03938051945848,48.12931628635282,41.831384902334534,41.79908212125905\n109435,18.756207125205627,19.111680579949095,18.72067224120609,18.514566718789577,19.440058667921406,16.302150808255007,15.989658361335366,30.41827988407724,28.882056681287324,28.23353516299757,27.133803713515455,29.57793072547964,25.642091811363276,26.06804872990028,45.264272656466716,43.57703058420315,42.33578614745503,40.71262966242998,42.47651446253499,32.89288527342988,34.46059648389888\n109436,22.187848686318816,21.437203546457695,21.137204393949034,21.185674062908056,20.647949105369612,18.325487362300198,19.146076413495127,24.429894599012435,22.90783824477319,23.406255052166777,21.464429001992485,21.123965200266255,22.92250221801992,14.415313569703246,43.26332510989587,39.312714870290144,37.667211963435584,35.02471052398263,36.90452598255738,40.45321817077787,21.256514507749582\n109442,34.828055550890625,34.542939908157194,34.462252539696046,34.481901266416884,34.04975659077617,21.439303719510146,19.16676032047357,33.190075165512475,32.07605510358092,32.535580840401785,30.791815453521235,30.06070787910503,22.18739467173019,15.322107036869387,49.50530207505966,44.27055779670326,40.768566123148155,39.28514284758606,40.43363832201924,36.326785716137294,25.43875854009858\n109460,6.492819151488558,6.266123167114428,5.940258854030376,6.031140539807556,5.851433602049503,4.693362662925115,3.4985551163016053,14.474188565432046,12.361157428454215,13.01510756595151,11.685236814702288,12.667488132638063,15.282445132344966,11.521792756678984,30.21485186589258,28.426061572246823,26.218773729855076,25.152081795591265,28.838234623875838,27.092026314745237,25.999161497279022\n109466,89.29969738566959,90.22204858617941,88.17632561323829,87.81777912913292,86.19868260072457,62.37559585941529,53.649208214767604,96.77533524438579,91.16195786698285,93.48705510600111,87.9235667656659,87.73154335409431,88.25164036275183,62.83234603291309,170.27406428596385,162.47277209245672,145.13569014680922,137.47757477449656,144.57721522397517,153.44704375013285,124.99643493796982\n109485,140.85634176933638,129.44620189303697,122.89967970024715,125.21901276773413,125.44787160875575,87.05216852732485,101.18451777192543,166.8520025316542,152.63605508472634,150.0143939826007,132.6259013575304,134.94381741514272,119.39649718777547,132.21201243839562,256.4438363459186,259.73423777390065,236.8283593077823,216.6569999132903,228.67068699565766,196.77766489869634,195.31493992237915\n109487,15.159225274220951,14.52664510756094,14.24263273372294,14.188748891796076,13.803272757015238,10.313143642430214,8.071486656930105,16.57124672275754,14.92485519443812,15.108136732641608,13.74490379024684,13.633184071680926,12.777384400863331,7.107704795455394,30.287378483197987,25.96318804015688,24.049530126836498,22.58434330375473,24.490656983013395,23.3817763072527,12.521507863195808\n109496,66.70580602352358,65.67346938656561,63.64148982185297,62.066300345877636,64.5341826012169,32.80881197728631,33.35557138183519,130.08314405516538,107.87514680054582,109.96960872018609,97.65936557768619,101.04150784608485,75.69088954415734,82.48756508187937,216.80156500307987,202.6572751481694,188.86669339577966,174.86971373442321,188.7198008184557,126.11171120455374,127.99786721174691\n109505,12.829157804916454,11.913528846630996,11.739958961214928,11.763597981779208,11.556202376981853,7.2207376228569,4.819761189270298,13.729136579927157,12.266811921934549,12.580042402506903,11.288774575131995,11.140736068660072,9.455620721718644,4.62463925188,23.623056535340154,19.858792455110446,18.829149554317386,17.867303741146376,19.251173503195545,16.58175745370897,9.913512579975674\n109507,281.7328909689703,270.0944153520205,267.66415214112465,267.3682044869845,263.4208677246384,157.98877990994026,158.29374929375686,409.3735287172548,370.6311742409033,372.24416788556414,336.3972805036326,348.09484793282866,253.32035222216274,250.32752129732808,716.8353657729085,635.3957044264221,611.8823602567305,582.3634250857298,613.687196490164,425.16357226220674,439.44328423836896\n109510,21.758675960938174,20.679087934812827,20.35857712653476,20.365614082261484,19.873804171824652,14.075998554841995,13.53177438241753,22.65295544431206,21.625810888207077,21.938763268687847,20.00954199887856,19.679754210637206,16.776322718088075,15.050901583615506,39.609838056216155,36.52371097264628,34.94083517754869,32.29763790679131,35.002059374743546,29.841153878180435,27.355835669107975\n109525,2.499556273860066,2.4535522031026313,2.309751762993305,2.287231826860361,2.232903337874609,1.6452617324076,1.400435060762806,5.1783457369555475,4.1856241114757875,4.31204831138979,3.8313412073409294,4.0833920106837445,3.8771547539939597,3.495146570952262,9.703864058871332,9.579669785306256,8.602882783814293,8.069352295476596,8.676094691307064,7.581278759386376,6.915691792682896\n109536,298.3240373784341,298.0436606049564,278.45126837347675,282.67862401287834,291.61998830819954,133.12972066241196,131.74099239079274,526.087143512985,447.8739307362773,482.96184332430596,425.96562324247236,417.1486185555852,323.25701010733843,345.48503885627116,949.3341690066193,906.7104856555429,830.6590342284111,771.5888406617131,836.2186890857341,612.9981836449389,594.0040654898743\n109548,4.799227786310862,4.636365803846112,4.531910143108346,4.515270341763599,4.410407002272875,2.8685634751880604,2.43011010300449,8.703250033255635,7.756342461595805,7.593965016500124,6.998956892899236,7.522826085973731,7.66050986734574,6.074054242723619,17.116617297156893,16.082729159342996,14.976376327780395,14.187851791157035,15.079885289939208,14.43261934338571,12.353430419222558\n109555,34.88941917196157,33.0186681554496,31.91845381993,31.211892167343418,31.1202134001338,16.703050541003144,10.327868410554515,37.16536368987962,32.10388304222689,32.16713676096936,29.31782170096896,29.144971410685805,17.548664658942457,10.924549962753845,62.75918402776007,56.01476744462643,48.30759821350067,44.303032314933446,48.146778552992245,35.34280465344682,24.67559452874805\n109557,5.384357537313275,4.958468427843575,4.986845901526768,4.967499571239673,4.7867009131162215,3.0402714043505816,1.5987346732777414,5.8996664657873845,5.7206597461608135,5.756087351126398,5.294321538270045,5.182132191167131,4.494851414853991,4.453057189757648,11.214736856972175,10.746207800786165,10.23394310032229,9.550452395446273,10.293963631906196,10.115221000292232,10.692655704945413\n109563,2.190366970641636,2.141378803894264,2.0542553298207586,2.0811814315471264,2.109247096502733,1.4079984697435153,1.3801731027084938,2.7990879273457705,2.4850397019053503,2.3982227452571774,2.214208640215532,2.2842806791319266,1.6331638030232192,1.671503433320922,4.5007459072278415,4.50501208215805,4.284747387054935,3.9213726922135375,4.0660709965305974,3.345453148683698,2.9726580727467464\n109582,7.024818166404526,6.5939424957967905,6.501895248843079,6.523930581720168,6.293925776640412,5.5546866361737,4.9584089004015715,10.23670787722038,9.4998684424085,9.642777909146824,8.831681362142938,9.33666056189589,10.119330248579526,8.39199563755722,18.610267146725466,17.73882300933833,17.347697978834265,16.0347740028246,17.572612379273746,16.903034391340412,15.038668750597994\n109614,6.0874856188804065,5.906625853179811,5.810372542105548,5.781631691498619,5.841754397634599,3.474090451683231,2.600614219441181,7.07537143178587,6.149871368144996,5.980212615611003,5.455626402496175,5.2719012991163,3.8838163101892484,3.675651788621239,11.36233153281384,10.854354038428879,9.19736073677007,8.410198351078304,9.511621758013817,8.359516611286805,7.140750466229029\n109624,88.88840210137482,91.27304816779046,84.11286042273386,86.10675669500871,87.31147747203399,43.55536660802527,43.631214603544656,173.10634534918142,144.8252407458516,151.97362885571692,132.90213682247384,140.3214729976663,120.24550886842707,122.91030523274142,344.7486855379847,329.46387279288797,294.656628457352,277.0961277644522,300.838219237339,254.59326451929363,239.18794285328798\n109639,4.24166445772778,4.087318768613592,4.067303307260152,4.035439273543301,3.9623465395683546,2.0113059572317926,1.1911529689895644,8.130549949578267,6.533672211041826,6.01878725999325,5.550792268896582,5.749972535108974,4.241079430594586,2.328951000717217,15.236141509189979,12.877688724802274,11.619174969949338,10.796947664506929,11.759929713207349,9.765105668366397,6.074252874145964\n109642,15.44302861354161,15.193424644382803,14.523299443319564,14.486394140554417,14.489162170689907,9.798528225858433,7.721075398623442,23.61919590262086,20.08584175746685,20.531373188749466,18.26884923637349,18.95941174960303,18.65348678472566,13.654086225837027,40.55025575521553,38.200588234668565,34.85313217906346,32.71842918451532,35.599945800006125,34.31010034333885,23.482823425422414\n109684,2.170448804693485,2.124271589745835,2.0842094663681543,2.0861134519031532,2.043339274595406,1.475024320300453,1.3740900396990956,3.4913427004656814,3.166592437446177,3.002448595447915,2.722728372450547,2.949192999421032,2.822830755103636,1.959186057657673,6.848805222623403,6.1179776433136634,5.683017280502441,5.428396123823331,5.727283296696605,5.1987637283836134,4.100492877284608\n109706,780.6573199716923,754.098408731104,735.6701716649467,740.5019901438084,764.2745320888885,783.4758096957459,613.5476304660333,1054.263102033851,964.6888485136815,924.216276221868,868.535744882961,916.8513911416024,1099.0187600411682,547.1991494093204,1530.4901523257915,1429.7462317726129,1405.513356339167,1331.314790663856,1416.3657259819372,1462.623115336609,919.3358121353331\n109711,36.32011533504581,33.09273037884737,32.89194330182172,32.822841598851255,31.406842552098556,24.43288342775286,18.552783907632033,25.11003772789637,25.53893949479486,26.805129875291158,25.80861432649703,23.074044405927726,18.06322680016917,16.875771037711576,35.89908777852334,35.05155793843163,33.16042166355136,31.338679635127466,33.18651673383899,29.68943486028563,29.701398895810723\n109722,31.908338427043283,29.906240063702054,29.318192849890885,29.181141448571033,28.302834966897716,23.613177062456383,18.222775843346433,43.720264623653414,38.999009107977024,39.35927731143833,36.040207438649205,37.40320735411474,40.3935893173624,28.58364579949674,78.16877851358468,73.32940850324565,70.03352492216008,64.90036037715865,70.00644308014827,69.37669267278427,54.5800655584647\n109725,14.168302491344267,13.50014599106042,13.273911302155431,13.28309739979336,12.961457120979368,9.998964937570953,7.628353162777193,17.26789873508089,15.901168700688656,16.222128850876963,14.89843855013284,15.289466355690498,15.412599328687852,10.262386842657047,29.162067967102296,26.847762812260413,26.102043863508925,24.349208523562922,26.535511955354785,24.95068309189665,18.78300162443155\n109736,5.238187385946941,4.869404517090528,4.747155897595118,4.73923910865706,4.582420363438878,3.5642705435317605,2.7940206160634755,8.976272319572502,7.231663365105322,7.004237439807927,6.37849043392199,6.963293264939603,7.0407991768206175,3.5199274540020045,16.487007403320174,13.53114744888406,12.655013189948031,11.987749309047086,13.057498960659112,11.294040703136277,7.128562559995628\n109751,2.738399603652054,2.6038582497920513,2.550436810851518,2.553888066874672,2.4739911288368774,2.1028075717423955,1.8590747546430875,5.063283002070188,4.543656322774084,4.544009852563436,4.021343181381674,4.586259366786261,5.893636174553494,3.510009622006547,9.84752126404188,9.216014447188652,8.872353723734326,8.40762862308937,9.493787380235332,9.51094096454376,7.451596253952289\n109754,70.82804357473519,70.50945835400275,68.29012584975318,68.37126266982591,67.7863455810305,56.648556016614144,55.66534212083967,76.02183241950222,70.1488547467021,70.5266083069619,66.26873229573094,65.46913425999442,66.13490431398621,45.23873821546164,112.81673232260074,105.7424413001521,99.69777428919002,94.87171351418529,100.15063363983462,99.25949246689038,66.90470631323218\n109756,46.56776477748937,43.55138113954161,42.91717558087902,42.87533516244672,41.74871942899928,34.568022457884844,29.073427717070984,47.34465001480636,45.6085194927096,47.13630134400439,42.75684966765347,42.93967375941381,43.95755203712225,35.82792654689868,81.97999887286407,76.99884243182825,75.31727161080909,69.58796230218088,77.07849127973552,72.87663907614476,64.74781826840872\n109761,15.53161065053188,15.026341159154763,14.727949433787872,14.717674369611908,14.485001979319119,7.5110095745510215,7.844154691975596,16.561545747658176,14.687042954689028,14.745677858246484,13.51826584541625,13.377350315963344,6.455897852884579,6.782675505250612,26.79869208107291,21.723533616262365,19.29168908219588,18.179801891046576,18.858909725878277,7.999684760655143,10.352190153276188\n109773,65.1199304798807,65.05850790921464,63.64077888955608,63.53586445167428,64.61210051024676,46.37295967478313,45.99325230866576,56.87645001289234,54.61216981081265,54.91508564996051,51.746241364696154,50.949952163319786,38.11876130868874,38.305038672706864,78.54406095435193,76.66164734141263,68.71448006225978,63.366236798488536,66.97860467204194,58.46167141778921,45.32716572708036\n109784,17.14812950262973,16.766697908931796,16.69317411681202,16.74211571737125,16.354306719370978,6.954891500420349,7.296151518743717,15.377360026486794,14.535630685273006,14.970604068930784,13.71157688174179,12.97934952015766,7.240544529978202,4.610164088738077,26.951162825387236,22.830191635536767,21.565160093148204,20.33330129929069,21.800042529965932,16.354388676196177,10.469892413709406\n109791,35.39655953649031,33.42790109519078,32.70893060040315,32.82346132452933,32.014385300345445,28.375862818423084,27.389674036231273,35.97657562570668,33.99305109832487,35.52007964954484,31.032335843813005,30.730155241523843,30.099379151531224,23.164749286621582,64.3925954364423,57.648128022060796,55.77961600790784,51.51729260378977,57.223363440565976,48.14929680588256,39.7334047607076\n109802,20.763477087388015,21.052719482964886,20.372624822205005,20.211346950008572,20.156044575901845,13.465796935517753,13.650735312528738,20.14560705362631,19.31253227662227,20.14367715238862,18.41612685434456,18.10516378496168,15.229163715443722,15.865527835882409,29.88417407929419,30.913105071809184,27.838758116374613,25.917018986542733,27.762511594501344,26.607117813115053,25.623097734384963\n109820,9.104231842211405,9.274997521922325,8.815585598129852,8.854717830506502,8.69743884203959,5.537704738998464,3.8166533626573975,16.01641422643215,12.976054478173669,13.66753057671278,11.37477085160473,11.674052434393175,11.975475056762262,6.234321123392482,30.22874932837255,27.82925324378013,24.968801870803173,23.255348671178535,25.33108110154446,23.960326172703827,15.033121471075225\n109824,32.59920722334442,30.158198536955414,29.786954015623188,29.821535748051918,28.731364534005234,18.884244678507574,13.174154690153685,28.97665135930329,27.58914914256865,28.721138210713736,25.603722017462434,24.620739053847927,22.419309695871103,12.725750458407884,51.86491059871927,46.748756982387135,45.41448603763814,42.09970513888088,46.75307422925087,42.64376781153521,28.49456149538677\n109842,202.7737828573937,189.8926756644005,187.01043008268863,188.7479477756292,201.6581855701242,172.81046131282577,81.73599344100478,435.7952741956024,385.5852790087252,382.31783242108315,367.28879459212345,404.3083717361151,419.1801146118071,282.9203079703447,677.4411610930731,637.4332343155662,642.246629108104,600.6997248081559,650.8895115540515,600.5050078839429,474.9001583945725\n109852,55.06907819855135,51.069722190696496,49.48565703041353,50.27139715523058,50.49138636646284,38.30211704051001,38.71322499969064,105.45171389365385,94.13433390773764,94.56107255417534,88.92292618661693,98.68292999977852,78.70786922459483,78.75268377897554,176.3905427598139,171.49237939334645,168.61022532881364,155.3779575229734,164.96791617935042,106.4127790295101,108.00859709701567\n109904,37.23635697643829,36.73241742242786,35.69158915216427,35.53463412842201,34.933613922291016,25.622360220714988,25.852897355551704,60.6744620929574,54.387362825233815,57.52337768110827,51.56561002535268,53.402009792196,61.131582814472225,57.49868695950698,122.09228083664526,117.43987062150116,107.88075052487608,102.3715277268684,116.1972000623615,109.14107904593413,115.49525739070017\n109908,13.972528140489064,13.27727748733382,13.17851253664623,13.236403536009018,12.839431971040998,9.0709630473201,8.321181503848162,18.914030524480125,17.37650948428545,18.10671180073351,15.751144871488842,16.45816319236773,17.299288783212216,9.917024764289303,39.21107748893388,35.78962597167387,34.781714978243045,31.89509187941323,36.23886861615846,33.63507004030139,24.00108594910843\n109909,416.16449458533725,418.0716808316592,392.1253788481862,391.9396604019398,377.78891817229845,190.68381910973116,184.3467434080388,594.1161934251555,540.6232140202459,546.9797621544895,505.35045330277967,500.7051125637451,379.32171152773725,369.90142086857946,1143.0370835978438,1069.288921970493,982.8961661533807,925.0450033635731,989.6529684629564,742.4825250312161,768.6223650438989\n109910,237.05465482996587,226.41952696219838,222.07339245461637,220.44766194437597,216.63340262419862,153.67130561128965,113.01635476183702,265.8043768197634,242.62586917655503,247.63200164231043,234.47054151010101,237.31108620396358,199.35450258997844,117.74903045821799,431.4250506502588,397.3513706384031,369.7720821928469,355.23064463664673,377.6047303977296,349.5354435856277,232.21699501290558\n109911,1.691221038949638,1.6421572602714432,1.6325385502865168,1.6302264935386366,1.6017642743649336,1.4628951435074795,1.1171315338534809,3.525299135370314,3.1289025739227756,2.95686084535977,2.7952131219779535,3.1485668983263295,3.5750886540487636,2.57834408576869,6.483023900531007,6.1045916203827355,5.757634708148793,5.440390504551158,5.918722478668345,6.1322880446190835,5.032816292542984\n109919,91.25777934455877,87.68645430288036,85.89421622808125,85.92618787744694,83.80347073653346,72.98698663962809,70.83671207570035,118.45322035986705,107.89737836301737,110.16267969091385,102.20245893947033,105.06518359672182,110.83397571172833,84.1797975786644,200.42227579823702,188.6157525965385,181.52016888110373,169.42241762578627,182.10265491750604,178.8401373031423,141.78923964503943\n109932,7.797404923803998,7.34760463574733,7.3008194351038025,7.339960521923226,7.144856510303504,6.519551880244665,5.156719188988232,8.610311546662636,8.190477410974628,8.089203383029048,7.491686329047472,7.774918667479994,9.039555204870876,5.4258732944952,14.121464012339128,13.076530055197969,12.660425225698487,11.869043991159934,12.582616060989041,13.730869471565338,8.73565358395947\n109940,51.23414351900695,50.60996571721219,49.52643879429315,49.74084409412648,50.05751215865203,47.72170653044382,44.084794385037156,58.308984081429486,52.576617976779104,52.11318647430688,50.094768628553545,50.30139191361275,54.512005129227234,41.623529589864916,78.40194258119733,78.28622893244832,73.76576506553205,70.46392371649358,72.89264254223527,82.30762524521006,51.021258712116584\n109954,12.411441001591227,12.244231036562716,11.877581854348387,11.872921744192881,12.147343747109835,8.445574072294844,7.089730320716743,21.58568312181476,17.752064171447653,17.160019341364826,16.313443426656054,16.699622345794282,13.848115522809742,12.541059739836063,37.73710522036943,37.10738376110625,33.58688219178954,30.566546842963014,32.12898561543398,28.430339337544762,23.37879446098278\n109961,10.317240482354194,9.916504817416339,9.87610931529336,9.882921883462535,9.677920659991312,5.832662835726548,4.12619376401857,14.311543798973817,12.691940179925817,13.040664646948702,11.575110238390076,11.756895731708113,11.084852843672403,6.0412218467918875,26.632998562507037,23.180742151771124,22.250436128432668,20.663959637554107,23.09330147548383,19.944416474046626,14.768065679874185\n109984,30.53342924868928,30.16719236611661,28.541766062000537,28.175899938775995,27.744094834761224,13.786147140279983,8.364255785492013,32.57575656253249,28.78886725511971,30.15396114155123,26.554201754422774,26.233408935673303,18.89498108866322,16.61212843000526,62.54069865373823,61.287054622915385,52.86079284394248,49.22217436541948,51.959748758494335,46.41132548208242,42.56786799179333\n109985,5.669595787586144,5.390431311001692,5.3428166777578046,5.320570586478236,5.197596428082802,2.573492242641765,2.0803511925545357,8.129199380837687,6.938100894145718,6.761736313885363,6.237584984406061,6.240783272780057,4.398598897907169,3.836135498344475,13.942498974897658,12.156159287469821,11.192305598605172,10.49998954808987,11.10755757086696,9.154050243997991,8.703595005841594\n109989,26.189836866704898,24.206662134055623,23.707270720033375,23.664833680985165,23.04291917472815,14.570133050489757,9.71624801986593,36.1451012433213,29.244222976684505,29.581337880818175,26.805321612860727,27.29266401776578,21.583964953862996,12.024089725994827,64.58020735221578,53.20733784864831,49.801005078169645,45.84815715848163,49.777572580019424,37.032005798363535,24.22164445594908\n109993,7.1289910791685465,6.798683104751484,6.656721455590308,6.637862158337236,6.735404319226758,4.1582981524222005,2.77081922956541,9.236851971420167,8.255825874256228,8.038537938753004,7.323509889506626,7.26290496586257,6.148453284752618,6.958238687949339,15.684629660539505,15.985385127842624,13.662904621402067,12.610051361928315,14.331824123689685,13.306542118907872,13.30302945635911\n110013,155.24812575136426,153.68189312723777,150.318403157321,149.49826298165416,150.97337782249454,141.78467400650374,115.39275195483742,212.9196777155528,190.44699318058233,181.65432894304723,171.72525335136177,174.21435528379052,214.51660274531784,126.34721831211138,328.91952921697265,315.29498098486937,293.1831888766764,286.9621529137964,298.03750519421817,348.1775436712006,208.0606571857286\n110015,34.064273580241604,32.782007083177156,31.94822768109011,31.944137447918475,31.18632112138824,22.62151148090464,21.57609498676624,38.46917041302444,35.313755997042286,35.791301067841054,32.08816812975448,32.86269891824364,29.33312783280404,25.40819860899326,64.87612661530622,59.10092710710177,54.94672008371663,52.243769061273724,57.4555589656332,48.17030924041855,45.59255346027755\n110026,387.26681461269357,373.70806513051394,368.1038288660126,369.2220396364101,368.42152219532903,251.56299908632883,259.91392523623495,578.4407119983429,532.5587365158261,526.0716508708782,493.6753289258642,532.1488549927406,349.8282399018245,355.1819891154589,889.716599871615,825.9974671046023,798.608264462049,773.0823061852149,803.8410629406493,474.1465445494614,492.1067848760747\n110045,5.457671733088379,5.237865598620746,5.03378544572347,5.028944487150747,4.858170590255992,3.9627133753916106,3.550688493575934,11.448446936540543,9.593282075440325,9.771490069461937,8.883389267210598,9.503503148631006,9.820180379976126,8.045688755382091,21.026637002763394,19.697594542452286,18.809759598972555,17.71052557566436,20.24729582636136,16.77260207027548,16.051058494601577\n110054,54.904575496402025,51.57979658107051,50.380312675403964,49.75923120543419,49.18161477636626,30.21802859606869,31.55597991358943,100.47412760019196,90.71409980792205,89.37018068634373,81.5310954522157,85.55407094538492,64.63199070090312,69.76686023333015,185.78416491790605,170.69664130909163,162.29648453282252,153.84269501091487,160.3572351361842,108.41043648496357,112.22797909523293\n110061,26.40723140374583,26.612781765316516,25.575656017414477,25.392936605332608,25.51222959523979,15.597042883789594,14.544414624814541,33.07585298835565,28.89943851630958,29.352922192001866,25.060311390343507,25.773966027158846,21.015806816068608,18.673504870663084,63.2477369033092,60.27507013977193,52.253702036459806,46.94021563725506,52.923008972681416,46.02031487490009,39.196984765595566\n110100,22.954611393169134,21.93850157836171,21.651232416899475,21.77467933015419,21.684316059463132,11.771260616854821,7.540318370983651,20.87937536825184,19.959969173954704,19.728136852421578,19.069466807220596,18.779885494472715,9.903932944547122,5.009329877093112,26.254290372053,25.35463705960893,23.04632360480428,21.49844355683229,22.510795281463693,15.563470343469342,8.01438871994437\n110174,205.52604542712947,202.10603011315595,195.96713608115016,195.41524039466654,190.0198718984607,142.32193488038652,119.64505629683254,152.8202729577536,154.5324382332179,157.34783102516124,150.3756520474205,139.72972022852247,106.12478325770152,82.90201072240794,222.14959862997478,210.42106456620832,195.82407001981403,187.15744544023545,190.69779569669444,185.84210501752708,127.52058548045954\n110201,12.000386853451731,11.726331168845991,11.693063852514674,11.678180532855787,11.441120905453218,4.192061580481045,2.5125662125905106,14.20381018863711,13.172535284349603,13.362916115935935,12.208691776556876,12.365782052418103,8.062848439853962,7.412446045865973,25.701434443624912,23.79654379614766,21.259574058402308,20.262212474795245,21.523348381138288,19.577828186702405,19.728378093441734\n110207,89.35146567273918,87.59273867571089,86.16463629165766,85.37473315091107,85.62752326060844,49.391687250835794,48.14806764915203,127.93614429199295,122.02462054899614,121.71980534050765,114.58381945608437,119.15349181691042,73.890645650716,75.41889287511323,188.1675935232937,180.38158194287465,175.18708455296658,166.19200701060097,173.0173563233178,98.2111098645401,101.0001407139867\n110210,11.130757636844077,11.009555466861693,10.625246558462507,10.619249554221362,10.395375011587001,6.603338479335111,3.645319104375537,20.045885534354973,17.196096150465323,18.426626829003464,15.447821694579545,16.116664541548996,17.500365613540772,17.506698413015627,37.42110530472303,37.64315793347361,34.374193298642375,32.50961154623451,35.45535544699837,35.116712645789384,37.8987058289893\n110276,269.47495445289275,255.1192180527381,246.8782664865079,244.16875629648223,254.94267208131635,171.15853741196756,174.62650453046703,496.31260023228816,440.28196624300034,429.2207782634504,409.09539722489035,429.6473549293711,355.11559460090245,361.7477945428585,780.4067674983065,742.3849720670513,708.745532488247,673.4013241016748,707.0711401096869,471.47966930415964,473.54976864883776\n110285,11.787508825680874,11.877216843788702,11.616553202200608,11.661861023588594,11.843745480928616,7.980787739240509,6.934675053785804,19.036632071590628,16.379188980550552,15.853644136152289,14.659805201274756,14.546413924438378,14.243290542433662,9.546572919142605,32.07939642766152,30.949059410841954,26.695290989504795,24.239501545065846,27.359664349686017,28.076999019001896,16.946231291166203\n110298,28.109572080668123,26.799089653170036,26.04508135704402,25.933143144039327,25.4563751079449,12.379376602144541,7.822976286777914,36.5623504024546,34.567330461102564,35.19200664111283,32.42791034655807,34.3869922143001,32.129238434068235,28.155958776865884,66.60949000234098,64.90834123223445,61.93327796807007,58.79505617795537,64.43236842826339,59.115195094858365,59.37854250269685\n110306,8.511811542603064,7.608951492816021,7.62433132965999,7.601830853894848,7.273518116364765,5.291766657228935,4.149934203473073,15.989945172203708,13.993355930096843,13.27608247784731,12.201344795927048,13.508449130626232,15.122636296329501,10.020736573984257,33.01765322817913,30.3664691117275,29.24105348170601,27.22238796490525,28.930101671556468,27.91759189320314,21.694397264007282\n110320,134.6421378391556,136.03632819038307,130.67492815199273,130.71471500722154,131.02877003278465,116.95890667464802,101.2485556047974,113.49703702101252,107.51287527446563,108.6683133827982,101.27096147914489,100.21633970959331,93.6849115358493,66.72539185729961,157.7422360053448,151.1885558172845,134.51560223490986,126.60491179639737,129.53954349588417,133.85614199358537,81.97204983712032\n110337,158.97191817629366,159.78736512123618,150.79394447856438,146.48899797093645,151.88458971360438,84.53155839819321,81.55199602514561,243.15553727692637,209.9964275313347,212.30775202484358,198.08938659396586,203.04556016802948,144.91034240307548,151.0659164824384,397.5268145546265,378.50910605569413,349.77284682027476,332.04594545474475,353.77049754293154,212.46579928408855,217.84929498498227\n110341,29.759772341948548,28.316341192108084,27.871418856975023,27.81196215203103,27.417311987163455,15.849169301256445,13.255471386053031,28.490697689276786,27.54249426085537,28.183134514151906,26.397190448741185,26.134387849129297,20.79991753768753,18.524340606655386,40.47987822014891,38.37262750308354,36.46128263649415,35.70037729363555,37.91758670740416,33.976414240089404,36.63797931026313\n110347,10.588479122509336,9.8492370984461,9.685981722170153,9.790443172590068,9.584002866964605,5.526295934681655,4.691258616441358,16.886364015133044,13.275804170467897,12.910278943260975,11.965413819201284,12.605629659543638,9.40732557610531,5.9583174221268544,28.343538964401105,23.112326025680698,21.769311111370488,20.303842553119605,21.27547655278116,15.543821424582879,11.25422641979614\n110369,3103.369115094553,2945.54019963502,2935.157195270051,2946.986282354108,2906.518793612239,2611.2833317282402,2292.917792179741,2526.640827791326,2563.2171518922646,2534.7381258083997,2422.4776061503094,2331.9127971317675,2294.6419901744657,1904.24200891554,3290.4319864082727,3226.15205632619,3154.5256442184573,3003.76058263584,3153.8952211718374,3116.153334673888,2502.8468355812806\n110405,5.014451177811272,4.631748827453795,4.651907459865425,4.620080562153817,4.48689970421496,2.9622639810297886,1.786588248202055,5.701307804445496,5.112104902570963,5.065203700342813,4.633909509188002,4.516069669695193,3.658747222409193,2.381432536102126,10.31628487632373,8.657745190106137,8.20623027896496,7.616161085887257,8.233535360127984,6.560091790189228,5.751274633626136\n110407,123.36490047720389,125.61722550708616,119.94292193095787,119.22289768795447,116.72092063266071,48.80609713261046,47.97109570208296,206.31616582142064,195.65836472374522,198.6935893349381,188.14757561860995,195.4038499773517,98.83420942992271,97.72675147072034,399.8048243827543,390.67311177632354,366.14729182565327,349.19300963439207,372.4705038051655,196.7206972372365,201.2123792396359\n110412,115.7270713915915,107.94345086775326,107.40218011559773,105.12574873839206,100.46081467271553,52.383736437213294,55.41749104507091,195.69298270109516,165.12115282812405,167.76085767395963,148.23069531572733,157.0822329387616,126.69559348653877,126.11995068175138,369.0950701658126,320.96212684199656,306.38022801227334,284.80049398965383,304.39556451180607,214.7534505520134,222.15914131498735\n110420,46.25967512201411,44.01532485594006,42.5628579759709,42.90595396440466,43.76784007309929,39.5926458191584,21.75019644130968,57.77305318999896,49.09198955092701,49.011566507551414,42.84237113995487,43.493809214470666,48.954403586656525,29.85366666346919,119.73385576178049,125.40172760962402,110.48571008029033,96.20023974612138,102.54630006894584,116.02489630643112,76.19905440821994\n110423,7.344307715737197,7.1109657191537305,6.960373323814398,6.964852451057219,7.025412237881066,4.222995958978713,4.099875407766166,7.8294370372462,7.356178178160913,7.257282068020995,6.709062983324064,6.606040466000794,5.291640457450671,6.129133242064675,11.259596141745252,11.40035285728971,9.848049714211188,9.213213380814674,10.440051691890858,9.41310673638315,9.76738646528132\n110430,40.30189012066863,38.37345405213765,37.72993975101002,37.49202788863665,37.68021828510865,19.87036041421158,10.696070627732896,32.769412463307354,32.02853142596488,31.928818520300116,31.170609009374008,30.425445927667624,15.830300078225088,9.677917583317328,40.0159280630431,38.73176199260412,34.46716881611145,32.49578674132841,33.37878656307212,24.037178247759563,21.989830779045736\n110464,279.9947504607383,272.59563527730353,258.90928193457785,256.01499694267505,263.90835898184747,165.37947357753595,163.52489282332695,443.6905745830212,401.00972317620335,389.59819864363635,366.74621669530285,374.729842187645,279.3590622589335,286.35608811386845,800.6960573221352,751.8885084380983,703.5345314576609,665.7624439260753,705.0982595479057,415.60151364610397,432.3368122468824\n110479,216.86830156021182,215.42290811977844,207.25780833122118,205.4329700569734,205.38639677711652,93.88673412877213,92.26633081606684,345.7394657370965,308.1463679377871,306.6335077045137,285.7324266462663,286.5270189717331,187.06038782206622,196.26257543626681,603.9649606785284,559.0715108226624,520.8359343626719,486.57160789430344,516.0860915026002,294.7329810771697,306.06831572005797\n110482,13.47369459288909,12.981597931053557,12.836311250940737,12.742039914890988,12.43723071759604,7.99679999301848,5.4636939811338046,17.997487549355316,15.817297305523844,15.693898112332032,14.492166328474903,14.476262122424025,12.5176623511433,8.322826932780039,33.433349088381604,29.642581101758687,26.826227582818913,25.23040528297245,27.750760028616295,24.208559874794055,17.687081108238004\n110501,16.624279954198496,15.847397615255911,15.853242432892253,15.820183309772846,15.449458053937706,7.292321091630858,5.748357757728076,19.385465753435785,17.687332535405293,18.019742272586054,16.37294943388002,16.4655367290664,12.059905689522337,9.215556492427199,33.48508623827974,29.298553661756788,28.326153434042983,26.4774297351319,29.312658162576625,22.182356067955983,20.941948459040727\n110518,216.0492539364875,213.4310478245506,203.45123079840482,203.5960355259953,209.51289869398468,165.3529909933715,167.9517600140587,313.369658773965,286.07953959580055,284.5898092347796,262.5503754479893,269.87558759621066,226.95823465052936,247.758581388741,529.6615955777055,552.9722879852386,508.5934837395082,467.47227644846475,479.43107608407917,428.34987577594393,415.13900235196184\n110526,57.33145351611039,55.51410458726108,54.863017715933985,54.74373416312588,54.02838507309022,50.25612983089692,47.08205723023553,72.29760561177739,65.93190229520535,65.87765926409955,63.41302715009848,63.91649805903181,72.12850032209539,47.624851139450506,115.51393827012208,105.54903050702585,101.8314387215229,95.55005859984871,99.87553791693769,112.94096675031363,66.45703095674648\n110531,9.837899098402373,9.448965411174045,9.029689155951978,9.008742444220218,8.91813388548299,6.3354199430266975,5.536118717043821,12.229318045092938,10.238347128396413,10.730720555531173,9.01778684058783,9.828650252539562,8.95856984420648,6.979772421183149,20.728968393111483,19.830875063653153,17.152721466652068,16.645823763433835,18.008642470219396,15.106632599575075,11.977384775752565\n110544,25.86881388635842,24.678741783663963,24.33434088594184,24.231763675855053,23.833190758727362,21.889413651611633,22.9492858272022,62.565559648589925,57.01533940712848,58.46802393579974,53.83515316977729,61.755924300751154,70.70205993880256,64.34362031101966,119.8183443854488,116.23030074897872,113.13729672980762,107.62027871209384,120.41194792378667,112.49060666473488,118.06098288249487\n110572,13.488769590311852,12.511249479780286,12.17291213069031,12.107383997313269,11.859675527696902,7.672176903004132,3.9527239396338008,15.738237258575895,13.854184963093177,14.132536697527897,12.869317815374119,12.68351555616157,11.667408617954518,6.775432456550263,28.876565966057385,25.07578984239939,23.23913218605425,22.198804289368613,24.22047619944727,22.5155376530518,16.717656610979862\n110593,12.1491968033314,11.45615130702746,11.301317716335122,11.288148044967512,10.93569273747164,7.631298783503075,5.236628166118558,21.024625427671563,18.12705148946858,18.255550929630072,16.879456358108424,18.02395067167952,19.267767335040652,13.931348439892844,37.41467441428787,34.575495293445265,33.142225741939505,30.666681293848573,33.05236912652281,32.754303820131334,26.077845018889672\n110602,17.979797675474263,17.73034440075166,17.454645335677863,17.486498388838577,17.458410232258437,13.835127277324334,13.541562696063615,20.453416993661254,19.53283162394716,19.54750591790326,17.80657701073204,18.000134725533307,19.81464395771714,12.024202692362824,34.22926974292318,31.32862909850482,30.586778900041356,28.735297510207634,31.04970479538706,31.88481453085985,20.77527403052937\n110607,29.92873242058228,29.40120998553089,28.412705951155477,28.031633636702402,27.449345207379967,15.725653571841667,16.494665323202533,43.81662862779204,36.80999175039257,36.834952257524094,34.083718828841775,34.33581230450852,23.14012241038126,26.730989348620426,83.66532256210249,75.27162274460696,67.09745592359602,63.10409424679767,68.23079797461851,47.12105545522939,53.7003835706583\n110635,14.940992436163866,14.332643475182453,13.783671203712542,13.714387761010599,13.386977740280635,10.830216594730546,10.031136288501362,19.233601430354042,16.334675628046032,16.567224981115288,14.778030012771477,14.725224189256647,14.290040001205355,9.994757945026599,37.922261466990534,33.5727279477232,29.858714340621642,28.25974007521545,31.595103598944235,26.96076638189374,18.98386427590848\n110655,12.072162140616404,11.86164792199314,11.599272537443353,11.544370138382492,11.300349375247103,6.60410535258277,5.113748713406811,17.508261760734154,14.471510379486778,14.556675833320876,12.749873772686191,12.837185448041797,9.976351212794798,7.743515494997885,30.61460277761556,27.525461364809207,24.54802669398131,23.177453765946197,24.44839262198604,20.501492087856832,17.592604569026438\n110661,574.1930981586953,556.7677582261156,540.8789584093512,541.213533115784,542.1671911143859,497.4357376599921,506.63107290529365,956.3027567950115,894.0715067372599,886.604237977931,835.7941154058483,864.2726813124413,720.4602300666671,741.3427694446448,1483.870941842918,1439.4381969627989,1405.6421402280912,1308.915336668685,1367.0622765430494,931.5607019408557,970.5304703000248\n110674,126.10981354260952,124.24746704290449,119.50971887063893,119.88075577222074,116.2375542340293,45.165265600291285,46.95282801312593,196.87488211380182,187.27856785261778,189.55071565104424,176.72103869065728,188.38882454584171,99.18970795272689,96.79375094737595,378.2214225923963,370.5716799886658,349.3952239649154,332.7460030036995,353.98536603203735,197.2240805405616,200.31068155063846\n110700,7.917961842173078,7.673723897691256,7.496507287385353,7.448292053910358,7.261704066074786,5.826492475292697,5.182108898863753,7.441107628174373,7.137110683819421,7.2745500996099315,6.758782424146913,6.424037583226254,6.096430824579811,4.342586302223952,12.77806644808386,11.450834928346962,10.659949887830939,10.060427095081442,10.890964737370496,11.342117748484315,6.837015964383518\n110734,34.122516701146836,33.55001135624984,32.76669036127701,32.47936596490277,31.990641240024097,20.63696984927951,18.303780449018404,38.326046592482804,35.225758191734435,34.94748867738083,33.05170481342795,32.65538650966866,26.058293464949266,21.438935296378986,61.477603173932316,55.41485608709457,51.55266804944802,48.96538296933757,50.81964440327455,44.887608736513265,35.10296315379426\n110736,40.394916609393455,37.21478384178927,37.14360174108737,37.29258091104865,36.04980676454931,28.10217092941601,23.796529205982523,42.20756484090679,41.51852336415677,41.781240819264276,39.069694161053185,40.584258673400434,42.09126783897007,40.06726564422219,71.69815470736631,70.53031472738787,69.584602561213,65.3928873258796,70.33889090328408,69.84267411045907,71.5927099978148\n110747,2.6227345519309146,2.5707024601119417,2.5523738228649826,2.556709967657865,2.5056673718933404,1.190130592737315,0.9807940091645756,3.5047247820587653,3.2174524876993384,3.1251216555033956,2.8851804644093733,2.848902191358084,2.366472560582818,1.8636499430519375,6.161700327949407,5.590699776371643,5.501432422749807,5.227318481722999,5.40346870906697,4.919982515608043,4.19652243917203\n110749,11.291566045328192,10.8767524945009,10.438920767546954,10.31806334400821,10.147517378113612,8.461315035083302,6.451960955995662,22.176151142401608,17.803683656742933,18.805301328688998,15.805634964068467,18.697293200891327,22.292858437269626,16.832471719020496,39.929193307479245,40.47369084754517,36.24811803451113,35.067534801632945,38.69532325096707,36.16544937036722,31.095704043956975\n110755,178.11917670438004,169.20423330578225,166.64679469274637,161.99249303257506,155.26979656456473,78.42423082340493,75.79048106612953,258.09297049041663,226.59171322552896,232.3607022030092,204.55845217191407,208.78142293944285,148.49536415243668,150.02510748175024,490.310538238117,417.4162537841067,394.5143205836219,370.11422663300374,391.42841720622033,268.8647302210816,286.72308516114487\n110759,152.8397610835381,152.56133804014783,150.57349875559092,150.73343654649224,152.1044123274651,141.0791755042043,131.0304165101851,176.5656990623919,164.3096588834183,162.1297918193742,156.69208340366876,160.3469025050691,167.04921979102133,133.98403621621136,218.47686444395146,210.6555024621374,203.0607892903919,200.8189795333614,203.71858471223922,209.2540253934213,163.88243730349947\n110760,29.38720561261066,27.5207954277717,26.42799136265493,26.34220278553796,25.26651317008952,16.01888835034728,12.091124653651839,36.88424805156557,32.29877866314296,33.69598545561669,30.367865743353978,30.710307861656357,30.370231724497636,23.053338509847347,64.12981419947435,61.64310851275299,57.969731831144124,54.94784006336666,63.32344293273447,52.64093976333132,47.443243932356\n110767,528.4630599559631,492.35516441794385,490.12690381294686,491.64172812312853,478.0547789542932,245.57860924318192,261.4514744581963,814.9156054455833,736.4572954376039,737.90994228268,672.5662226176978,716.8517118545773,540.6440854819155,575.8107251354454,1340.5424978506983,1275.1338400309774,1240.8856814641156,1151.274598028244,1198.0336559500436,745.3042821338013,780.8634056237574\n110770,26.844992952962944,25.93911169664516,24.907194407407538,24.608503480686245,24.72601244979524,15.982247416113045,11.200454283388792,30.653787424711275,27.48374449773358,28.892874008836746,24.932149063201216,25.49835416015113,22.83924430054587,21.701082799346292,52.36317753971796,53.564689441112456,48.30145612837794,44.277038096569974,50.00974728226778,47.14034318456228,44.892127724119106\n110776,22.420710239237206,21.67499986201431,21.127165429699353,21.07006453529652,20.66405069585028,15.563478146063792,9.304006725176912,27.095868942740474,24.520796470254453,24.329548167522713,22.721916427085965,22.745884014614557,25.53473803989185,9.66360933802246,46.37733258047358,41.103826516151365,39.10837344464431,37.26763565214344,38.65478813311363,44.67605397793035,17.101361192870645\n110786,31.99148297197379,30.626747094005832,29.94137977767755,29.84066040989418,29.209431394711988,16.92528514927298,11.924723177850856,30.1637002501873,29.13586274049392,29.803955077843494,27.173415333295253,26.6001257773763,20.6722772620398,15.672951166501472,53.83458990285693,50.23186904015144,47.15743901938461,44.86312026319319,48.51226589746526,42.67980641011709,36.179422847202\n110806,13.653955319945933,12.695866231652301,12.433701329062373,12.368547411335387,11.913725723060201,7.542845136996939,5.757056262132249,13.836072390161645,13.459252483240455,13.766178462453173,12.519803617401442,12.270427522318679,10.460891814468544,10.103862183268392,26.68079594048641,25.97614483727578,24.109004324388053,22.710535915309947,24.346977119193326,23.225658796087707,22.693792106525468\n110811,5.893454722538335,5.646602301781792,5.608832706978438,5.6437343076693125,5.524014862547852,3.7942723256290325,2.5293820013413693,11.310340909343623,9.983899086406298,10.459619853537548,9.288939387085925,9.976411364852076,12.251044043841953,6.897952555894003,21.979420402865326,20.25408979212154,19.890611535237642,18.307669648055167,20.820147459978788,20.35644757872709,16.3277590159223\n110819,168.0997450987346,170.36760390742953,168.04808143631186,166.6257049122315,170.51547304179698,83.8293397705748,78.64072050450088,176.57577067722866,169.75713394975543,171.5042292142751,164.98935185637072,157.81202086287917,79.41191178074928,82.76326193883114,226.77946799105,226.94209517301184,205.62428198566943,194.81104518051407,203.08633866545142,124.67453633826428,125.14959043761665\n110843,202.65381365415922,200.33361256281097,198.4748320223455,198.47572111957237,202.49280960964865,97.43819400632108,93.0867846889783,213.13555022951283,209.15948958944338,211.21218107045982,204.31557947918324,199.04703166886554,76.98535704250594,80.9593170540763,269.1063437177624,291.0260712152747,257.31032644496486,241.24860403787213,256.4369011945741,98.66133839885937,97.12038565023201\n110855,7.808315553802158,7.076719882795596,6.953545628739518,6.973783505141005,6.737353651272554,3.4260140429822563,2.2319049532305733,7.940273846035459,7.764456554872237,8.000970263883337,7.004121039016545,6.920034885927211,6.11406517537841,5.435157339708808,15.212987611226009,14.617467022254916,14.354028062796935,13.306995473158628,14.62286320834773,13.326951778525618,13.418505274959172\n110858,18.589684345123285,17.69874669143928,17.435102857458137,17.479050188653616,17.01021769858242,12.645678061980743,11.247198260153015,25.30514558591797,23.789167310760394,24.732704898913017,22.554069678758196,23.396353907075767,24.45632388250767,23.896539812040636,45.35112399022757,44.17256025658431,42.8684857125539,39.70658918084556,43.66675862010296,41.639213239092555,43.355576453080175\n110866,11.151421074071184,10.941684962839757,10.539067939934414,10.505208762650327,10.272753136002326,6.741283661484832,6.97262627353786,17.75818999369787,15.000329927130876,14.901867531641885,13.810156836915747,13.743411459022173,9.013884289614507,10.351034553207663,33.09479774746062,28.87614389112457,26.852777790523128,25.173656305277184,26.878408681932306,18.054248901757123,21.25193511289867\n110879,9.147750825002676,8.684625994934892,8.48224770302087,8.425948604195762,8.18631954460693,4.694642646333752,3.9220755196854147,13.03593134024937,11.677191419163846,11.873463003753383,10.50211722309917,11.145702184726572,8.94531799834032,5.943869386020617,23.51933207865408,21.864767000949062,20.8814681787465,19.163647070663774,21.12813027595806,17.61856000587216,13.093318013288062\n110896,5.396727677998769,4.937613424951666,4.910027157196717,4.932913950127085,4.750175955993525,3.7321286078703255,2.408285608655861,7.763387719253479,7.422143557497575,7.26669235331799,6.7263628804743965,7.220099519776139,7.49342695077339,6.896213792233368,14.10702417817555,13.881481590839956,13.563747268204407,12.872197760498695,13.591659372674483,13.3501209851245,12.830228512071084\n110916,133.44599826894483,125.04852057151045,127.28095932654767,125.89073962704724,119.78315924624623,114.7592880173898,98.38642539311417,104.46790964463091,104.04466004600408,109.41583534950405,107.35601022094981,98.2638163393705,94.85413954044257,68.20152179855636,152.96745379401992,148.72217236640734,141.9246907570999,133.32684251067562,141.7175061306869,141.98903922663092,104.21222017979413\n110923,5.083664189627155,4.912312883936837,4.748929971858992,4.715696538900561,4.59456395299256,2.7421165740305877,2.145687625361561,10.126309860737202,8.360424541652709,8.22722283283005,7.757944899739913,8.2306672580848,7.245440620175276,6.693636215751343,18.179101481681087,16.96636245692636,15.456883165506534,14.532036857884997,15.626648623170915,13.232536199394117,12.453714113257496\n110926,492.6564241343519,481.0797028233658,447.27266140549256,459.1782936207254,471.71079807129405,383.1587455544829,389.67135423295974,805.5207895260269,733.3216192531182,735.7601396462545,669.6748975506854,685.2877327034157,594.0676217729911,639.3375614629696,1302.0878695331585,1336.218234009048,1250.899435374346,1147.6852096719906,1217.098081195532,1034.8445106787644,1007.9617782547978\n110941,10.159715589410988,9.615321964817678,9.375716074355793,9.417042161542541,9.168757705374466,6.356631957211596,2.7862097426723498,19.307540368833656,16.050187418781164,16.0144454704609,14.696914884881345,15.343675799463087,18.01703847655116,4.7008753168785296,36.311492574237135,31.390989270068886,30.215782755102,28.22564383727287,30.148545108995087,30.67194766729984,14.056246706093702\n110953,19.531708393041598,18.11712450529127,17.817993566839473,17.978042280439315,17.29387918439158,14.158050849295448,13.26179307221594,30.662905420171942,28.908644497494326,30.55273253576849,26.581803013344402,28.8541606012119,32.19750676442836,29.194763341335623,59.2522361520819,56.76749381842079,57.77084819946168,53.25617941583456,61.20464772624481,53.87819935256664,57.96919499946052\n110960,19.190893041923683,18.34842477613042,18.02208538577552,17.97120547444995,17.4916550946393,8.37312179730687,5.211998895167425,21.676758781116177,20.15115412731221,20.55468281711499,18.541346481611445,18.988985821968193,15.430191552389674,11.462895832158772,39.51446053301735,36.28721205689229,34.23281049948896,32.254555105869855,35.8257518024628,30.6790560818658,27.767927590938534\n110961,2.1156175139712494,2.0744480169457984,1.9879452315594257,1.971409870109793,1.935613598724934,1.4940666298672183,1.062705880989312,4.776390649222789,3.9718744012075042,4.146934809665215,3.566714394965779,3.8846872391611966,4.037100022208042,3.7038362460107583,9.268370937990982,9.195493121846722,8.480234605926745,7.945430756472924,8.997026349779608,8.533405988882993,7.727402346178594\n110979,55.32435455706921,54.690308608002745,52.18823864495905,50.730939844791855,50.99150782004943,27.552765919817055,27.560998995462516,90.55070226998497,77.99130491272547,76.66127700476154,68.17835139081086,69.99906903780037,54.59958596262825,57.56918359602918,174.13064423360652,162.38690319608352,145.21544963738094,142.09009442689023,147.634826552113,106.28069284434471,107.1188128338558\n110987,21.81666098385946,21.626592889853608,21.03612371508446,21.027518507801087,20.69160673989956,13.764845105806321,9.703190556564799,23.520276402045365,20.88856684266939,21.736420640035696,19.515918434321012,18.93790806399642,16.37454178631374,10.621818061410794,36.56510058062289,33.44229964556133,31.46667311972187,30.055481857233957,31.38558991252676,30.511422068049583,20.89253578513883\n111005,159.3910022089134,156.74318181430593,153.47802109889508,152.1258151827259,152.27591976347043,125.63831508297672,113.79695562021188,143.03139713864488,138.82278861842826,139.91330965444575,136.3658902975319,133.9845064195416,115.71818635460899,93.32380790643649,185.7421529121349,180.65962026434207,166.85448984756937,157.60737439572526,162.45060665689627,164.72342649099232,103.80450871620407\n111010,280.589753678649,272.76468635167635,263.16998114293506,262.059425258173,253.15148264975667,147.42725499876835,148.8501677052815,340.0955230006708,319.0281629393617,321.29557930205306,285.3982186055276,292.4858601093767,221.2975520901867,220.44013187376802,589.8191306329418,557.8422024648901,535.812339637927,505.0444899382098,531.8384878362073,382.71556356360935,391.44859272334855\n111025,14.124314052913416,13.199355834415764,13.00577206342014,13.022515995921964,12.591581574272412,10.05781713688747,9.089271797377524,20.12704001525899,18.397339484639517,18.478093390714623,16.994242175842817,17.78073990216183,17.984887691519848,15.406962246148728,36.96969173224935,34.32809055972419,33.827947565520084,31.587582286595424,34.08118292878515,31.837823054078324,28.83490002249829\n111032,13.831011545317558,13.62281100803108,13.340256580845555,13.315698290254334,13.052695466666911,8.376506769454359,7.400722070148623,20.578918129912015,18.349268703229832,18.42750833540729,16.854489742397465,17.265546930222712,15.681491914320398,9.429578221731589,34.27616758889925,31.94290848553919,30.868671150965536,27.99270750248285,29.96388109152678,25.032615356915713,16.961560430202443\n111039,28.191634598284054,27.07000210555961,26.481359771579474,26.388389359003867,26.214998841175934,13.29422549231184,7.470102641142121,27.98250389940591,25.28618999013832,26.232881868216094,23.8344082493111,23.399160447371028,14.362815182679665,9.231625777052543,38.908026494703385,34.93837644961925,32.00106781395984,30.845036098024593,33.14474156296791,24.601186532580034,17.91234995103729\n111045,7.87389583909731,7.943831511772055,7.720255578254921,7.677382698349004,7.677316164070309,3.7832176988697808,3.315072563063981,9.987411043164421,8.446714626785463,8.397919836024961,7.611353655693031,7.710937095531041,4.8905814617677805,3.5597504648090537,15.348173440359918,13.834246869344346,12.583619837849431,11.757217269814605,12.811650028946833,9.77243714847993,6.739397763214898\n111058,66.37103581529604,66.82743583816458,65.8967102951669,66.01219492354238,66.97659215012465,33.5220030073853,28.9181949859595,66.12997497967467,63.95773890291083,64.33557362494119,60.156956167967536,59.56468122973805,40.19167162643342,46.53963664526822,105.38409019381841,110.15938852819633,92.50476762844042,84.7936742299435,93.16012760389506,87.24598688476081,88.8894351953484\n111086,3.4398982592299094,3.3324478461539058,3.2412173614321285,3.260455246968327,3.2882129849538693,2.6927701946762133,2.307588304782244,4.375747537568667,3.9990571036009084,3.989607638273814,3.5882923764114736,3.6014017086867427,3.3936208980891966,3.82900531305407,7.651125080908042,7.938596336460434,7.478981434964058,6.692922884988484,7.7900570987711495,7.118386053698531,6.8322746798102845\n111104,13.537986397103518,12.444790472399646,12.349882701630147,12.421710497791212,12.185848851348851,8.16773818701731,6.561318300912887,22.310654683700122,19.81606259848101,20.89814330894055,18.53466337255706,19.720800051310864,20.566768762223777,16.643682069185186,44.41100203801791,41.21949220489508,39.93341711182424,36.65587884672684,41.64809408629109,36.831902353392174,37.31943148548384\n111124,50.52528307106864,50.030581095761534,48.17781622363557,47.921036756906524,48.141002626176345,35.75941069595803,32.06570374262264,43.65637802582392,42.03659153350026,42.881353703969644,40.276092532003716,39.43779895602555,32.04921545857237,29.762935059595133,59.068732933603684,58.721363929532394,52.34358002621292,49.07086527012899,52.602780826600466,49.41826411115655,42.73561174655603\n111141,10.019631643256563,9.4219432872716,9.152734059115389,9.094811569500589,8.804604214525192,7.219981775389282,5.625827975551671,13.942171019125716,11.700228819926316,12.36469841136038,11.274965586270826,11.206506745300114,11.376019386201293,9.438709097186438,26.08119672612803,25.68220347077012,24.13596234736311,22.609056357056694,24.841907021617875,24.081809435382205,20.500907860716868\n111144,28.97266354749546,27.431827123970432,27.405485683622096,27.32084274182459,26.806942454658913,14.219094428047457,9.886216647620245,27.404299306562713,26.334268348026267,26.950749837902737,25.84585997604636,24.939851113097063,15.598145765195298,12.557628678713018,40.681039972638736,36.981371875749346,35.30684620753121,33.70261339604541,35.73847753665386,28.175090150647847,28.111021191739393\n111154,30.157985470838344,29.321072541675285,28.246374105417697,28.498053109529813,28.704691785861684,23.207933869629134,18.862094356149857,33.72696224129903,30.91359098122975,30.311124166259205,27.81713053045625,27.905397301555276,26.609610952712345,28.992511254788113,53.77340077104251,55.53267390118885,49.359467459008336,45.208040276436826,48.873439675964235,49.12230878941684,49.265562896878684\n111173,226.37554408518318,229.39186546164106,219.16155232453104,218.0615739542461,211.46758383261744,144.80824126933496,144.58066702638405,315.6473196571271,289.7716083980081,295.2520317987115,277.0784010788674,279.57298825455393,163.03836673404152,164.55800986536357,567.7417618454039,557.8562553202548,508.5037463084844,493.88582505197337,530.9467934406377,323.67619627800906,311.09792129366366\n111178,105.65778854755268,101.11394926977543,98.24865998292171,98.52179905842502,97.23481157287985,78.31406884193778,52.444850193615764,135.73816758614595,123.18709999238874,120.38914682029215,113.59649633729312,117.65098836228245,122.64011309805294,68.3949607599689,239.02978053458855,217.3577723969427,205.14941620584287,196.8732141711842,204.26248944243764,208.3749929176266,139.1197668114351\n111215,150.90490460979873,145.00195075332124,145.05311714285637,146.27903122649508,142.17915772141473,105.41546088955477,67.22833699500758,151.7233052926015,143.5280283813298,137.48304643782987,130.77417595924658,127.61960942315746,136.8003425377245,53.41916713607402,239.24264977590437,214.16433378010038,208.18752020728945,201.0614748628578,203.15812464203793,230.775470909681,110.01150300341152\n111216,64.94604852474947,66.11472161103671,63.008897743225056,62.95754037845067,63.40480165312298,30.61563738415691,30.80959658464828,105.44707311261847,93.31723163295673,98.17646257789498,87.65211672209952,89.95998343427408,58.53557529251758,60.0004087930249,206.29861125211545,211.98010343220164,192.56522239816545,176.27774340698053,195.15413048534595,140.428581102117,130.2943711501351\n111221,15.65939396703593,14.947452111406374,14.642548112640409,14.760348928758164,14.349154154560493,12.107482929000582,12.573799601642873,26.73405312547491,25.012768286309946,26.725862931012895,23.812291343104153,25.72137137229019,29.11894578141199,26.856576316893275,51.19516272120861,49.64027700132266,49.648543299814996,45.80671264265053,51.91546012971609,47.15399608802461,50.16213583003062\n111229,28.834447007018195,27.61914431284211,26.809194717011405,26.605558332901776,25.722668173978924,25.14306814558222,19.64973878579376,31.492178456275308,30.293449120163036,28.969448756983248,26.099428722230567,25.734143141262532,35.47562948056112,13.288920193515624,69.40818769812245,61.471584227675365,59.17164283470131,56.15165317391331,61.39093008273121,75.04307590968232,30.53148931426322\n111240,95.66029566253516,94.43908880829153,91.40212083984541,87.74049748120203,86.81401572023704,27.329997717770304,27.46538992972196,130.24295424676637,124.98701098528262,129.45390715988412,123.27043419483458,121.0143430717077,42.136675576979634,43.33351669594964,255.18225895907662,246.0568894947724,228.0155908973985,212.88520022346233,229.04299813923768,80.05042564076382,77.18899292233053\n111246,675.0492962150488,675.4761905497469,659.778811948758,662.8137504473563,654.8847538901697,428.6539288584761,412.48114902429404,620.8415531332461,588.7207637517981,593.2066050388045,569.417014560284,555.5975997287863,429.56965508010006,380.4645030902415,825.8600443806984,775.4564306648581,721.4521870623022,691.0821001187468,715.6199084322652,627.107544848605,569.2214034818732\n111259,15.797241317921895,15.68967210715745,15.685685435945283,15.603005812342268,15.382083788857607,8.471054438041282,5.996106300614275,26.674973893044406,23.649492365783903,23.61306296131153,21.961667562596322,23.865505531272078,26.317861102113216,20.534797056117604,43.825243604101566,41.28012528836682,39.73350488652111,37.726203148111125,43.94324685467073,44.08326485765029,46.21543364142659\n111275,9.143735664428252,8.891232789014563,8.52807935940556,8.427732988030906,8.274210023763308,2.569606118605443,2.5795099182332053,11.293069325884069,10.317798544674655,10.579828580163941,9.710489257748302,9.818081633950602,8.563205567099923,9.188032189697415,18.97267813055051,19.326200303641798,17.52684540053274,16.332621280015633,16.927689375939156,18.554661166968263,18.88431008977347\n111276,64.5940583373776,61.37252266544607,60.49351446774445,60.796268907733165,58.68791063682442,62.30759320153976,36.13223268621329,58.38040624977411,58.37761944197005,58.86909822555635,56.04464969194117,54.90550492824682,64.02782098361378,39.060330999180586,93.21190546121889,88.82474642276487,86.08660725155882,81.89399626203972,86.32322880381174,101.32537825567826,58.978395422551436\n111287,8.351739116582472,7.665662893566208,7.599501792171429,7.563960220232871,7.264367451305129,4.985797539059085,3.0748230898708226,16.51310809506789,14.119380504969321,14.129185080008714,12.866384800841455,13.879365345676256,14.707167147402867,10.982964801758296,34.72951955098793,31.99531315428075,30.050107732832643,27.832117489662465,30.37118448687115,27.71033412365145,23.324949872548697\n111294,18.780611190785105,17.923066078136127,17.53710387000122,17.538679635478836,17.05754848248342,13.442518361004337,12.57589380242188,23.681621886779073,21.734435484487385,22.019440399507463,19.341823764093697,20.58842872447242,21.927398129072643,14.554435138605221,44.60695059586496,40.73006266954567,38.59670642679086,36.36531666665016,40.827857813436935,38.292664433535776,29.297467748523765\n111295,8.393695160936897,8.098557515308338,8.046860778990947,8.150010127269939,8.258893756814897,6.710240128115803,5.803369200841178,10.261606432568815,9.2001808772805,8.889422244012513,7.874062785907427,7.84303752289467,6.577070993564129,6.933262205085598,18.808910232488188,19.2751603452874,17.42639592471938,15.603394504189144,17.637076038356373,16.491087409114403,14.085821414107956\n111316,56.06791452030878,52.45540460444486,49.644606618897946,50.98764663189928,53.80939457111339,36.520346051719514,36.3692880297552,104.80750547411924,93.59032200338189,89.78311628268985,81.07887382356068,90.41740608058656,78.66826538884038,78.81806271009314,181.19039785831933,181.2533167620431,167.84607208850255,161.47196993489396,173.75148343987584,111.35423151796248,113.20463651476899\n111343,39.35858096881142,34.96525725871445,34.45337858846956,34.30313269095689,33.416948969185334,31.132538526941456,25.288663822132236,34.13646861229643,33.29928514761229,34.16817190618454,31.254286732244413,30.745283938780297,32.30112493776151,28.42839645134756,58.30135256088515,56.32695809445462,54.620294987021815,51.453119293248484,55.75458266761513,52.754963263160064,52.464318622101686\n111359,69.35766112833468,69.25617376877568,64.41854249647197,64.88919074765683,64.21116095913632,43.459641687181005,40.419657229816934,64.5295316561833,58.3488071385534,61.13065978239999,54.03072146268111,53.28052359117366,43.25144681306169,34.24865487800446,104.4647261908452,99.22621370477522,86.69441609906798,82.02520385783573,86.68352875572614,84.3412302094952,60.3951917162133\n111366,11.929087225017382,11.774795687400754,11.291076754346955,11.227061766301352,11.042868519128396,9.459182015184426,5.774661852625157,21.749965407511283,17.20317006797181,18.17926879007587,15.403218385898827,17.441038929673006,23.86006658338944,9.616696300794658,38.22623254929405,36.62705272405166,33.0260796599095,31.769752736172446,34.59043945732685,37.742788171456596,18.86082575341694\n111367,102.65027885876601,97.82554147926798,94.08282768030065,91.82046874034418,90.5703643339191,50.572732789332676,50.886521284050495,197.94087968789154,168.02863698417477,164.2499167840532,149.8919896412848,160.1752907981,137.51941546417854,136.74272595235354,388.9067370818618,343.29651276225945,321.23340169934335,303.3336191093168,319.9665125849321,238.22111973681865,244.201959778841\n111388,329.52605102626956,325.3501646258928,321.3743621198875,314.87800406141446,316.92539989831965,230.26943239054017,215.32801075112997,252.23559762064173,245.25512579580464,245.89326047594818,237.71453091462652,227.69431688568497,180.65985522586845,168.52492966804297,285.54082934822935,282.34510934635756,260.93715495289916,255.39056386588797,262.2034781226583,233.11135015897815,226.46434611044916\n111396,7.182198034194693,7.037678471911096,6.763998777236764,6.789273315853995,6.627165295673292,5.590596873313016,5.6279312707636,13.377209769792383,11.44530269614684,11.990567380828326,10.564982832936211,11.142001337277112,12.980156953784638,9.642142398597342,23.811743153648283,22.558610000167512,21.935131300768074,20.731895823491385,24.76297044642182,20.70652542768789,19.74721580451649\n111405,44.15502901872643,43.583547629615545,41.669873261780346,41.38861198654928,40.94178776967585,24.687176748845292,22.48586635442085,46.411451221596835,42.40480639961562,44.23441508473829,40.07979203908064,40.245851377764815,35.70067167775081,36.9659617429496,71.24660464047278,72.57927855286158,66.27660997617653,62.6947864631364,66.26032517307162,66.51701438260615,68.57253812832342\n111410,9.696051211404008,9.727708506982415,9.174076267047843,9.028160167425067,8.894901016073396,8.495389349406715,8.21335238150895,17.341086351771796,14.99853613371416,15.943536457060882,13.79276673917972,14.742002898284847,16.6319214341461,17.145890158884733,35.90044050418729,37.41455471669029,33.77184450925181,31.210638705386028,33.715742856972504,33.89904933593756,34.10031574354265\n111421,147.8948104004497,143.79179222156398,139.67773719155113,134.3848663361869,141.95513232694464,78.59705153563092,76.08865168098627,235.4404594149317,207.40187638557285,206.95029730279225,191.48700397239557,197.82895368604767,140.35288218353736,146.91407541213886,388.3310217339358,376.01744792548857,352.29897326477015,331.1208267428103,349.5644377301531,214.5410580950157,216.30301278376243\n111433,18.780014980466234,18.502540126868382,18.136451055691538,18.169552309669225,18.32216249082041,10.565498075889435,7.167343080452196,17.9292584628095,16.883629817982186,16.90013098773036,15.853700196719368,15.376447987935624,10.86412314058283,11.545450854682654,26.661223595268076,28.575629764772817,25.766346734177453,23.002377845063293,23.797922011870835,23.323497563953577,22.76851123908812\n111435,106.25327470042646,102.39178662411592,95.79444357452127,96.41313747205818,100.17294448543234,75.43662654393094,79.60905755456845,143.27721350724136,131.1813584706594,129.66381224832634,118.27536922439978,121.16638410058862,106.76099892103913,115.51338487757899,244.20913061556226,254.29716568214565,232.60915998730104,214.31927204204436,218.81077337404844,196.89751160805537,192.23078213802432\n111441,166.60279515021338,165.46042655875289,151.51039746862625,152.39169119710868,153.3831818206953,85.98020428794597,84.13658204344034,215.46852127283006,194.53138092602242,196.00832594961958,180.74019532534123,180.002067295043,130.09035543515094,134.29518846687733,372.6616504306083,363.3765587893954,340.4309174924381,316.1168381152157,335.29318855972843,213.2338935754436,217.3565627210161\n111442,42.4022445206265,42.83802033281319,41.191430052500415,41.13460834730382,41.47256775844768,31.209041998173635,31.29654996556424,51.43698469902038,47.77212983050073,50.544070227725946,44.439055028697155,44.33904800721082,42.93946207646162,44.293687069037375,79.20779970728417,79.19092171672555,74.63453968080982,71.18350829575233,78.72079370589637,72.8470618718294,71.56251505101638\n111463,7.571621953776064,7.375725705811349,7.172357085307523,7.21213782300851,7.114408666494877,5.943841718286199,4.968270755567533,9.78276736219779,9.046916736945857,9.030290883867305,8.171770144492847,8.36266307427334,9.652541277719207,5.045917888807475,15.752251368903439,14.48833386839436,14.073949044891734,13.330589906285853,14.557257966187175,14.660049474731426,9.345738538559354\n111473,73.08167411883987,71.478994998129,70.59406102091137,70.89312471842645,69.27617404574012,41.81915417632008,42.34954483905632,65.26512956983446,62.33735552990562,64.34673760252474,59.577286144914964,59.454648749060816,46.75244663261275,45.30172072968269,92.5356683093849,86.14690488997145,83.18831023816594,79.09910373645131,81.344771066587,69.26966808458114,67.12808309834735\n111482,2.0860572301302076,1.998804411937406,1.9270844792353223,2.011735938121122,2.135048217538177,1.4542982154466118,1.1962555816782487,4.897623016526968,4.112729217178563,3.922862438558054,3.813267538009677,4.282886738636428,3.7318631163726774,2.402539621931921,7.865236953908049,7.280877082190881,6.967360264542542,6.720813916980167,6.9962637641886225,6.137690606627123,4.534899620592105\n111497,11.207218305609704,10.306111701159939,10.177627462545084,10.213541981477055,9.920804211912806,9.178902892697502,6.04293763364211,19.938763086676506,17.55109818620068,17.561425959065332,16.185119340165254,18.43530316089372,22.251294329796355,13.68914533264685,35.05430537468243,32.57981184569328,32.31238426174006,30.130006247812265,32.59305459705605,33.330423839394456,25.27111213713009\n111498,519.8200918043378,523.9292773577404,511.83910372263483,510.5242072064069,511.0350627333617,427.46518192406853,419.76943210521506,533.4410734261795,496.2756317170518,496.1119872660625,468.87323915784333,470.2629198244677,453.98878577373847,378.0760369505626,724.6556797238499,697.6263908181145,649.9348457071278,631.249224932438,642.3979363650501,646.1173219845921,522.8765842109632\n111500,21.754763090595574,21.548560562166085,21.209196266531603,21.2571628350205,20.924520515984085,14.565517595762232,13.490377817907326,29.919237333540337,27.38265749021575,28.80280496160278,26.16621160102145,26.749140113665657,28.507362240335446,26.03913650479191,52.828093252018036,50.21376982480779,46.66641512281557,45.10189061834384,50.97083110392284,47.45425979598411,51.72366330446312\n111511,167.67563895433688,164.54046936639403,157.16073155288132,156.19910538221626,153.53022927074923,129.8396890509072,116.90742778310128,137.33818853258924,129.4412453537793,133.0626496215735,125.86270391347031,121.45401923208152,107.93214282396576,87.5622192071336,184.2877691697631,180.40965043447133,164.15005502433314,156.19368535912807,160.63954378019974,158.49270211902132,130.39691133911415\n111524,7.35436891432066,6.878570080635605,6.754169621757998,6.670558736744909,6.498380900243382,4.096799211409466,2.1744585218940147,14.539865783565476,12.97237602528963,13.207841158572323,12.085092663286629,12.696945192514109,14.287838951233537,13.99205941461462,30.955907981132544,29.87163103431769,27.6594190803569,26.26618772582061,29.023089096714912,29.771636435610972,31.16439534482397\n111529,36.95028129470399,36.33700563793226,34.52433290499263,34.75780159734918,35.41620494354503,27.692695059432374,28.617020161913292,61.847292489881504,53.90217764937153,53.3689640030363,49.501514081003414,51.06708166833931,45.806651775382214,51.45748201387975,106.48789562850307,109.19592617263876,101.39288043186764,91.73558870687165,94.94037790735668,83.17351481681668,83.32892114785537\n111541,49.04695942688986,46.82357332593883,45.851802866546194,46.19791367225978,46.27013274069337,37.05797266814757,37.82102125502453,71.45325592140449,66.74938332445926,65.4785067495276,61.430801588267784,64.67460705575802,55.00354869495517,54.88370867344726,110.23839590177151,107.74271800566092,105.0490272684278,98.18822855837091,102.74663243124166,74.1507670389594,74.78107678066077\n111542,8.804065424722376,8.672031153146465,8.37455481854397,8.484235762476986,8.577149118019591,6.713096112027303,5.327937449187947,11.882823290330125,10.190662652837128,9.679856507893364,8.684653849665901,8.740707463378873,7.684621143618201,7.0288778529077875,20.521495197277808,20.099657534935428,17.43016821961448,15.772863617669431,17.299799600991577,15.474388109866204,13.080458902093945\n111548,2.1961228907413886,2.1135040600315094,2.0486931462944136,2.028581276193748,1.9912389381324052,1.7904644673043042,1.4811944172182396,3.906909645769293,3.4302751089142087,3.446208399818129,3.127593618247837,3.389196443207998,3.783480571325898,3.7309930621192846,6.934687330431372,7.116719886350563,6.412749120780834,6.302816775086648,6.620169161421229,6.632461297663292,6.479178875145813\n111587,39.09793771356138,37.51133890506568,37.35581456505573,37.3333400903324,36.82088822613187,16.543891799499498,12.376033000884114,33.964432118978266,33.88087515631681,34.632204563137094,33.61767434553651,32.32090164254951,18.216950587934097,16.853881508126204,44.40789398332458,42.23603178790418,40.90341665322754,39.735632486637236,41.08094248101022,33.96183821521032,37.29208470793732\n111591,5.657294726134953,5.643015506199721,5.505125257290658,5.491017989577685,5.608229456398989,4.456142163698363,4.0926605837398276,7.387281814840602,6.845483476639192,6.594033296335039,6.141977849809086,6.071244401686829,5.774873598408195,6.497199299233409,11.797601014688619,11.982246137759423,10.607601990376809,9.876784991262568,10.896426909729792,10.543857694783625,10.58089577995158\n111598,9.829660573485631,9.794491712462618,9.559036038945463,9.617571776995177,9.636351142903813,8.510741590011156,8.541103619230823,10.918772179703513,10.447985506053836,10.313623645791356,9.610666179849693,9.333726695206094,9.292255106660752,8.18892417421414,15.796861534688759,15.004175340723862,14.068792030683397,13.25954978499643,14.54456007040447,13.921263948207166,10.941630799427198\n111605,42.817708011633606,42.10819030017175,41.3740720434407,41.2707879865676,40.09574035215074,30.599773053619693,27.47707512702553,52.84693451020392,49.44204011651694,49.18442635495231,46.89807334846014,47.237750287878434,47.66850749299947,34.040643594180146,95.89658587413172,91.87711075042999,86.75588278597711,81.3943917658205,85.0279357466886,88.036625993227,60.45847004709997\n111617,37.2185137593044,36.40712988824306,36.09579333487585,36.072233685292105,35.75191139030122,18.787304352309757,13.942988218607077,31.988560729912777,32.112891287279155,32.546732984687985,31.814059478910718,31.212235622908885,17.128614658054108,13.842049950051633,35.86312109616913,34.13347804516423,33.37539530537339,33.34077971144216,33.75810289588228,23.52283601628854,21.61644184630287\n111618,5.86665019137787,5.386049449936007,5.270925865979275,5.296795971304012,5.122440019365643,4.495411440860571,3.016224013568533,9.927587113390537,8.394763120508257,8.521707788275064,7.489472161748377,8.155335432210872,9.238189036086757,4.169137307015552,18.573106393677456,16.28803359985,15.809282978130657,14.64332062418337,16.593644111367052,15.031824161381277,8.88109668764769\n111622,2.4285230961055992,2.363018062609649,2.3134542603731703,2.3279367698122124,2.291913398683142,1.8499818824587448,1.808360920403893,3.3719113836902515,3.2784641598531317,3.31722473418217,3.075740751614671,3.1498034237985677,3.3547570062726293,3.262349410609771,6.1053962639811505,6.0187711335713034,5.91772849032521,5.585609707995568,5.895751334163921,5.745351278569873,5.748720004146378\n111629,21.176708521696114,20.09762369266251,19.083578507974174,18.895111722179745,18.96891011965395,11.292001208210896,4.493964791938104,23.33843374373419,20.97601739465497,21.90059906051155,19.285730954207605,19.225430686198987,16.58363273078441,15.233700302817942,38.665679826846095,39.59601119539054,35.0157860579227,31.844013735632956,36.048231521268455,35.60947007769793,34.822274918285416\n111634,243.11478700000617,236.64039416342655,231.49159095059542,232.11621969744203,233.7969426268126,206.90652425408896,169.9600721778553,275.819090173095,258.2488045515415,249.2077386854501,232.40015103980167,235.18829789730347,249.0807811413757,159.89833150954877,402.84995617579824,379.0330552199225,361.38095722168316,344.37901060800885,358.08902709601097,363.3430499271988,261.9304207058153\n111640,85.71988998376925,82.83595879713846,80.24466281569886,79.1317973007624,79.75070549730471,43.821801380185605,40.93803686571355,153.8993522742698,138.25435743004456,134.07780184988908,125.97611032970138,130.81700351916552,102.4256614976145,104.54674402871301,279.56473689985665,256.2281512118987,239.22613634338728,225.1366001227349,235.17755095370217,151.85648204708605,156.01278655579367\n111642,29.453451424447294,28.959213909741127,28.57371495585666,28.57649251962339,27.868943592801187,15.609571744763892,12.506585416917481,39.5655852640951,35.40284127927725,37.06062014170987,32.85574189738928,34.17290510193027,28.776840329934146,19.304753430070154,73.69701893253445,66.20419760205633,63.438775220849436,59.157148327284766,66.4042932978421,52.70511714026169,45.08331052028668\n111644,229.40171244577547,225.08524635525382,210.14784469703181,212.7282138070813,220.5187487878148,149.10575625233434,153.5349293240928,332.8227428227139,282.42922049256447,280.0900942835882,246.71472936824554,247.95747441814535,214.5910828334842,261.88647509615714,625.7350916841367,633.0625099730191,549.2909428515642,503.8056906699678,563.615897455419,492.65881362025027,484.3382125959406\n111649,7.315556589435528,7.081139765361738,6.968737058741678,6.946752520583049,6.828680521778789,3.3854678462539,2.680858607665447,8.31711696922461,7.751255474217174,8.219006855895476,7.291053447053948,7.207150141165357,6.965311777586512,7.096990902991939,14.878634340231638,13.748474112163988,12.418868405042117,11.937135864680595,13.176469700049365,14.937993413524142,16.357201565077926\n111652,89.74024029214246,84.8969242552097,81.15090998613408,80.5985460061419,77.71884504452157,91.14485717098313,23.205152124812585,103.09895910019297,93.93334341620972,97.21685025628324,87.0967009905945,87.56173355436566,118.0077394291225,41.00029140378814,228.45360966155542,209.29487722823947,190.98617991762157,180.07723918345113,198.94954049873937,222.78685124270444,118.37962701843443\n111658,7.730072375505281,7.468330776408513,7.434424874726469,7.44995348011906,7.329068993610707,2.9738112862927597,1.9161620818104368,9.145361902058065,8.416304151694971,8.60816439258518,7.819490238576274,7.829637665845969,5.886483961841456,5.080478775945178,15.31179587536056,13.963957435477194,12.926380564648447,12.192516719737947,13.158945430290656,12.163925156758669,12.203351864401139\n111666,1289.121336892098,1269.341717272483,1243.5398224976573,1252.427749759335,1250.4423255024692,999.7604209410566,975.5577637701489,1360.0800903342188,1308.9560967180553,1301.7973144082725,1210.9639482849993,1203.7968092116216,1136.2763525530515,866.8883514026406,2017.8552050863505,1967.2653533015832,1918.6805319588298,1778.3576789534438,1866.6535697645818,1735.804646568504,1382.012090175351\n111667,212.8268314815383,205.55885229906661,199.2754624018872,200.23130206030137,207.6699172007692,136.13986954312426,136.4783469990923,271.79183569211114,240.72465965751564,242.9234850675388,219.17848515252274,217.13208637061112,179.9414522022116,211.05405942603994,467.70024204385606,477.80562442230934,416.4599436352473,379.5307767821317,412.87269040062694,366.2868289126791,359.9008412197756\n111679,5.022080806228823,4.895650479805896,4.641885853082326,4.569144408778427,4.4515297311275805,2.452883893247189,1.1567012523871256,8.103773749003077,6.5998186984265335,6.741675211200472,5.744361542558631,5.971834574102308,5.329132836517081,3.649795838459366,17.30550786135378,16.31176059004045,14.017483381340043,13.049996769154358,14.047066594757135,12.814472118876134,9.56404136024576\n111688,37.7047646868325,35.65721280376793,34.99070931123474,34.909315910613124,34.23152831043976,17.003347056139656,8.58080514496705,53.72688073287444,48.360146683760775,49.758839649397345,45.010797753161256,47.45281526305231,46.18213822653499,39.178420138024585,103.73907718206405,98.82409307201411,90.01511009430307,84.97577340693071,93.98942679056276,93.1980602797239,94.12988940241526\n111702,33.60624652470724,31.25550438202233,30.929426989071874,30.85533706133572,29.85547791528961,24.486757605170915,22.608895975715683,33.645588326419315,31.58763580817455,32.30503018234641,28.61112593036992,28.553618487389507,25.441404536847777,20.31596090062303,64.04592973821677,57.02436552999929,54.281879912167426,50.22602998532082,56.04751958487982,44.153507146107266,36.016929085771615\n111705,6.841208634400523,6.658333156785336,6.564073426413208,6.581941711002767,6.411790644156025,4.167293314519661,4.345494003871422,6.462564156068934,6.00902194718962,6.138884991541282,5.489543916955324,5.309036618221159,3.897920135972837,3.0099480172250184,11.642944995186314,9.983601181888984,9.207976419450674,8.601553971083561,9.00207237107675,7.374544218914773,5.020004638875949\n111710,62.635653625433584,61.59837881624777,58.71176516954815,58.93614891417464,58.42736319350091,49.08507301198969,44.232281758565925,58.81562286949819,55.56888896235522,57.30474747904844,51.30517919420236,50.07065723870605,51.65637065887341,30.949507952116093,85.5186766551695,76.77922205093643,75.64135291960163,71.3987570541499,78.75020865120533,70.88139663158495,51.00524199949629\n111715,26.614710089820953,25.92471507697853,25.918447930659326,26.003267857556043,25.684156852467595,17.38457917215455,15.937843177282835,28.96853452150017,27.930325084947487,28.257700243867088,26.686179557005968,27.293242541215488,25.920464271212133,24.391935467584673,43.36712868780513,42.09318772224622,41.356646737944345,39.45617389569656,41.89409967871126,39.25755044796501,43.58868104522948\n111717,168.5686852415317,171.24369275971313,161.60358328014487,161.08537674123798,155.7810428356731,85.31075837331844,86.01580367063137,307.9003784195627,289.22382113486157,294.98338850750446,274.11996573712315,288.19324165619236,224.63504622746086,213.72859144601927,637.3965765756195,620.8104676747232,581.4921415999786,548.8999966726785,600.3375434143825,454.6906166015727,458.44447321099256\n111729,5.762404791528207,5.573977991781213,5.489542768177289,5.4696346397992395,5.295319075322298,2.746978636401383,2.1545788128926016,8.274817119422547,7.1632509608727375,7.042889636972332,6.384120315420019,6.680826385218256,6.108753774362326,4.583921748789451,15.777330211150005,14.185086604598501,12.988087393647529,12.156525328700985,13.129409876935089,12.098643180018973,9.838672605007629\n111747,34.48935341301771,33.54550060962202,33.10235451106174,33.23089543401398,33.13324272266558,25.009350512494827,24.78420580135256,48.66310105834567,44.1401487594845,44.43438479765129,40.61029705672487,41.89190807881765,40.895415533443895,33.71142497117157,85.16373079167144,80.1488370478072,76.44051082615752,70.97798193435774,76.16313764265658,69.68242385213358,62.89592034384437\n111751,77.67709520497279,73.67254872366674,72.65870376790363,72.68412308478582,71.10491925753163,53.08015444559098,39.21247294263464,79.40248162776442,75.61826216430944,78.16430372597112,71.62555168008036,70.6215452069844,64.92833761434409,44.22801572549955,141.64062849656685,129.42689093132824,123.68539189897187,114.66403833114764,125.87722380784349,116.99213323603713,90.49220433499114\n111752,42.62160061324498,41.404399035892375,40.32889130156392,40.2160530301522,39.21578245882341,25.20503481384756,26.953798329358534,36.14772603020316,34.954549596922305,35.9549848959322,34.112407144824545,32.3869799092457,21.183979268781563,22.034296698980356,55.739733653213236,50.82276280797182,47.18926205368244,44.622643850089986,47.79974874420506,31.84904962465277,35.07744950603086\n111759,15.783343347048444,15.167693233375845,14.696789607585517,14.686685656860234,14.881617811845457,10.732482354730394,9.504030867583392,21.939520830699635,19.20661862404464,18.718899903642143,16.768851110905562,16.7128268397735,14.512824431868799,15.888934602281347,39.25578691236418,38.61677411314799,31.82287928036343,29.624163801191372,35.42498053521153,29.667615938859367,29.145009019028713\n111776,22.927957486167333,21.315777034744748,21.288458502407085,21.200432893532867,20.756277542111935,13.896556214881477,8.551427902138146,24.49163809980184,22.88460794209572,22.746216067520074,21.424674508254263,21.682558349887127,19.746857956952656,15.430651258318345,39.00562867635934,35.96995726403391,34.881831650943106,32.973012771175114,35.12174153945623,32.151748995584796,29.217478780951456\n111785,167.34973085496247,165.8568920421542,162.3008095627531,163.3414306927864,162.59234745142717,132.62096646189593,123.28546178106441,171.38339494518775,161.1005435073203,158.36893624197015,152.08952917323273,151.78710233697998,153.01007056651494,90.95896480960897,225.74522328807112,209.73515352524177,200.1816660095637,194.42455844949603,197.89908446334084,210.47308939236788,108.88434910307355\n111786,1247.6606507751792,1187.1981648772828,1175.2471903957878,1177.078656452326,1165.0444985947702,996.6750443695581,868.7158607316923,967.6359706713679,982.3687292872606,967.9813112391884,907.8747932603104,865.7351251114461,867.473680037494,580.7307679689009,1358.8235438560584,1254.9014655051024,1237.5942445458918,1181.7072364541277,1267.0996501075583,1215.2651132564886,854.0536022116586\n111803,37.09453038373382,34.682756903397554,34.2910928675082,34.189289375214074,33.521656912316594,19.04894503231694,8.382503715759661,34.79975904729086,32.64227882372983,32.73338734007482,31.39277651338386,30.566031013317723,20.283645040930825,10.00888595710112,48.72321421472799,43.53307957533133,40.46259997527375,38.53879506310047,39.35140878667991,34.7556747891803,18.734539681240793\n111814,111.37169239809809,113.15905702650588,110.67645074127441,109.7894387270561,109.21575874892794,74.85586882910846,75.11962741719073,102.06164205656687,98.10456845179294,100.26738003670737,94.65583541254556,92.18128786407189,74.68860053834383,71.60097645380597,162.35938991582486,162.28542485806724,144.22392769640706,133.84759265068584,140.81448914613952,133.6945255820168,112.87565138186682\n111830,1.6649392200987898,1.6089560101150233,1.5649091956820695,1.5660046552646623,1.5298582318377691,1.0569872873286532,1.1187448148053414,4.313847583393238,3.655213534833353,3.6492032490065704,3.204891950224167,3.6256616067768097,3.0362436336930534,2.8833453472607133,8.293032469156795,7.526453338408832,7.191462593715101,6.7943073286590705,7.723721162313564,5.4779790690935215,6.223601296289369\n111832,35.59534570081326,34.03010198053048,33.60963054912054,33.389445386271866,32.93008650073644,17.566479413484025,5.599482560247504,44.71650261780157,39.0558978024789,37.97042670947965,35.897503812187836,36.54540227738367,26.391722553755955,11.594560991593847,74.0297410713829,64.04610593641975,57.591830337083934,54.65375665316193,58.91182568712012,50.11409592652479,34.90390404923268\n111833,2.233590006890606,2.2080718262260377,2.148699364159432,2.140350179573597,2.1108645201427647,1.4793537041477662,1.285209771365086,3.9187991922784544,3.423542332095561,3.388399192126186,3.0510169269804135,3.2835672077354245,3.442737262706039,2.7528865150820785,7.110478553576982,6.6906710077844505,6.168999340062511,5.87535146015074,6.254613586801966,6.172823453867215,5.013478342672118\n111835,15.669676619225477,15.480952684111081,14.958006814056308,14.916245781031925,14.978543753883299,5.9961915403828225,6.048845302154858,19.46354153324317,16.01804809734054,16.299711851039017,14.266825385179535,14.565163195811026,6.721198920553427,8.080308036645665,30.252241902967235,25.93852836490056,23.068225723269368,21.236001390134213,24.00950597724197,12.576681049914374,15.807802642707651\n111851,192.93449733654035,185.8288039544046,186.0674610099374,185.8338234733286,183.78422197005457,166.39564096183045,136.22599684217803,215.78130495069468,208.05102622511072,207.94050372409674,201.36990167628514,207.67843551648951,207.18921372360984,189.20033349851113,290.7384878445677,285.49488187125763,282.04510781086617,273.48632978956016,282.7484261707653,270.0879786697662,264.25133980042136\n111855,10.659785082618956,10.320303154485096,10.255674441999158,10.23574537972033,10.046247684262607,6.726474890060295,6.630724760042478,9.822060217107119,9.730211225871217,9.884787018411721,9.286546930553627,8.911338210152199,7.224520206732948,6.8056186333194235,14.599486229367233,13.625449730631738,13.193939367457984,12.33011013422502,13.114403331075698,11.830870694188675,11.248944539943086\n111864,313.20837785572337,296.0754327316065,287.8980393892591,292.0403343325878,291.302535090629,195.14166516785468,203.71263703990647,525.6426678302302,475.8336162536496,476.01419268609266,446.1548240085233,465.54892879184536,379.794584358889,387.0634398833391,910.0250249954045,872.4155896620327,842.7093111700469,777.9037931541554,826.5824384173675,552.7142372175585,570.901398303739\n111873,76.89613423727549,78.18902868040101,73.63196528411162,73.17762692613053,72.51841590663405,39.000564822319305,37.83584811958828,127.0441150688734,113.39143670484845,113.80483079834977,103.4498744192155,103.1097582970784,89.75844234110473,90.92543661703304,236.04916162226272,215.03750651707156,199.1443383154748,188.1894022304561,201.42024482722724,157.04200475007605,165.296245086933\n111875,16.056638050951708,14.921127986786642,14.577563872811503,14.56714880612843,14.108323685786228,9.88240809575,9.7581996259467,28.75741080728923,25.82112754292753,27.053798563438544,24.503867750050134,26.583954825468815,27.29792817824274,24.584180944801222,56.05193982998331,53.17515484924544,51.69100077389103,47.69433487120636,53.03969020272121,46.88372645235128,46.79133097020283\n111896,365.9004409431906,366.85902791474984,346.78468886876425,347.14447041489797,341.56173465877714,225.6856079249043,233.35298350321952,545.4840974166075,489.55270285040706,493.8692195608559,456.9011716122658,470.36117541390644,375.4239489723924,356.05198326777804,967.4597666603329,926.6803082622888,872.5287698036595,823.4362752953865,891.309478173483,623.1263068095402,628.9295680153259\n111898,5.983166230227969,5.815315211569093,5.84683178302736,5.851352094873479,5.68301876101089,3.530255697477098,3.1529933243474195,6.240198680623132,5.94481464881261,5.909723927951218,5.465340165216649,5.207614759159828,4.302238941655452,3.1216827186797285,10.873748376269576,9.657443771639374,9.420220502084737,8.814056326901289,9.298041334462834,8.948373464546874,6.649754486863162\n111900,224.2094303522748,226.3090346209028,219.95376841647706,218.6501455753064,220.47654178127638,171.687541602342,168.23541219430555,214.50593158797952,199.2979961179728,197.31646907596289,182.67754027495138,181.25842675703032,173.2129349128472,113.54259549218827,282.02316441077346,260.0467446880306,239.02333766630477,232.0798376282655,240.90127276998322,250.77334361340525,123.53128035788882\n111901,13.02173801697456,12.64587843095493,12.022082081592256,11.92263329807971,11.90454698023096,7.373725357359047,5.106305810387773,21.93512585472075,18.18855509479678,19.490848015491313,15.97340555618034,16.91726890688874,15.861814294446265,14.066702482999895,42.570083554066954,41.912273042422164,37.71523942007668,34.416794741068095,40.18082191987778,36.21686732792143,32.32364634675935\n111924,35.85215886370475,34.2789027569432,33.85897694995629,33.72624269086059,33.00032790368739,23.198337618208782,14.59631455244626,38.81107549474461,35.17109465939642,35.33365579460738,33.12028252551976,33.00140980977049,30.559624021783865,17.976363611237325,62.484760867157036,55.2988356580867,52.53945783497026,49.499054892671694,51.741153720317165,52.020003614041215,32.97251387204195\n111925,11.126029857371854,11.108283696427565,10.902954472147158,10.937094669496203,11.039501645410203,7.380438053711565,5.75162436924817,14.262719890215383,12.05306580820271,11.994078172536256,11.112337908022626,10.903984148888803,7.10307212647502,5.788762707519359,23.129139909792,22.539567348055098,19.67093843431,17.364403438716966,18.280029784752283,13.324931178569695,9.981766502326405\n111926,154.7123752037651,154.73273483011715,149.09556659403194,147.77674317341052,150.9101791407134,70.4838187037799,68.02143542994561,209.61318563928697,187.97852477371308,186.92688425016982,175.08254964143558,176.7527408200264,112.59049212888104,114.22053173262584,337.87042506112255,313.7264037691312,289.00124207985493,274.7225580673205,287.6782297135419,196.13644471729444,194.3232802809205\n111927,12.751428378460606,11.415173758797982,11.610893775926947,11.462353387394105,11.005284609771616,8.695379458489738,5.662524735083984,14.403713451887437,13.297968082701482,13.108051194868201,11.929487384098964,12.042001826447143,10.90701177136449,7.3250829053167035,31.36988730924368,27.058100839863226,26.13394219921872,24.141192219023228,27.06054125283234,22.05461772467956,17.7657512810313\n111942,29.809465306493337,27.571501526680294,27.32122522083515,27.515577102815932,26.69747297824665,15.25508049470648,6.5885618803757815,42.54615097890633,35.35177250427511,34.76597044422211,32.133202721026365,34.58165613127864,27.62451792479634,10.861884608885658,71.7509406920464,61.712712899668375,59.92562474068776,56.52084762225325,60.05829916424836,46.34305853515207,25.433680164906402\n111943,6.434784681044887,6.208407522552541,6.1130275867186015,6.0681763580588886,5.947150683915005,3.8738554111927175,2.6772430657096833,8.775659502965821,7.706932211404493,7.66403250646816,7.111113079166948,7.171103837739486,6.840752594496169,4.603686144414034,15.470864973975425,13.747256321318433,12.606203029041708,11.930802020578232,12.908333874588793,12.422588490481301,9.32380193226497\n111957,603.479912373008,578.8226713307619,582.0206150477334,586.656472674576,564.1337287952439,257.7445948079137,258.65475468855544,993.6712668385136,946.2417173473369,962.6824159961694,902.1330253192382,985.9929948893787,599.5732891411293,579.8162312701883,1741.8768956762838,1695.924006054595,1662.560591136071,1563.3484073635668,1659.6790076074342,1035.095478169575,1079.8519464669534\n111966,12.427319619082667,12.275953454498971,12.154669759241042,12.195692140638032,12.016959369843573,9.058676384497899,3.515052028100778,13.872226999667308,12.914490553322398,13.03846819260556,12.130104260589418,11.704385892334116,10.634945864530037,5.482496272034286,22.832619988347904,21.62519225847328,19.70914815853947,18.24916613243055,19.574252989732543,20.05500597717704,13.518897815330913\n111995,12.47717362588901,12.102977088111096,11.65487514254622,11.594136681285582,11.43961105557881,8.41145315844162,5.515865801826095,17.22250684581886,15.113779801614228,15.715912565397893,13.314653414376368,13.889682810009424,14.047286137410005,11.368039889071879,34.8160065327143,34.44413516066343,29.948951353920005,27.874117024070458,30.721072326650273,30.67337176301956,24.146312256871447\n111996,23.87563669081903,22.153267552328497,21.863504587022305,21.84573249055459,21.225342043818486,15.696479352910764,13.571112606688883,21.621564799022103,20.50650123055429,20.909199922503333,19.49017901683574,18.635758636992865,15.02368608089152,11.73853593968838,33.94783816408838,30.113918821665724,28.816634664322976,26.839203494117626,28.67038968230863,24.669607188492225,16.639879438147535\n111998,227.8731509482961,225.8206718980129,222.99356862021793,222.4281915721736,219.90370338569744,148.61185398641297,148.6595917567835,247.4082810837873,235.89377428581096,241.29033885056307,229.47090908681827,233.18677884984507,205.22302034533226,168.47175117798932,368.0407538192068,350.54331955745005,339.42455761336447,321.5979430382016,334.2362424593685,321.2013124891987,249.4910401845423\n111999,735.620813901741,697.2151386826512,695.365634177808,704.1157760470488,721.0989509056258,869.5052554410827,481.48315443016793,1058.219002897267,995.3007805173546,958.8790393322764,937.0110265165565,1032.657402783685,1118.1761165918067,849.5414479732531,1253.6079571073394,1227.9876037427641,1235.1230498634275,1209.3893777851727,1237.8302712878449,1263.888389633286,1066.1386856098393\n112011,15.760672559496005,15.228376312133323,14.840423823832854,14.85084820222771,14.74913270830592,6.792094804556476,5.464695318533641,13.628330160308508,13.222130504122392,13.047571822657336,12.36688932115503,11.965599712292564,5.637986414304128,4.656746058752664,19.44747378053061,19.299464111644927,17.573682519365263,16.055350948577278,16.308815770705767,10.596434708479952,9.129403748114985\n112012,11.145548745066181,10.829279139898134,10.326201967924574,10.284446059905534,10.028915672147297,8.773902396528284,7.573730836302559,24.47864002878955,19.18378097713867,20.65308623612451,17.922399601448888,19.014365924764693,20.792368319232594,13.308499759910509,51.33603241070009,49.763020342527355,44.80328921901302,41.77603217441219,46.37136880327335,43.87292928737267,31.212430987600904\n112022,40.98883019655125,40.69838613139343,39.39803033922662,39.31160734899664,39.328185446609176,26.306731222852708,23.586285262641574,41.90052098643158,38.1716116656473,40.04696639723792,35.61692654313603,35.024880171216864,29.4964585481804,23.959849489101682,59.02811107049542,56.0292993782545,51.08418133430001,48.44159752754535,52.906186526527904,46.52085790565192,34.36984800216868\n112028,5.341481442562852,4.934357353399665,4.830184523942411,4.833066015992423,4.682603016041047,2.8954046439726713,1.7439783334582908,8.190846471106997,7.0249815502802235,6.941505652075652,6.321292518515959,6.523683046360477,5.905398466856458,3.2321546804388266,15.046348447572065,13.306718027070541,12.818551971398534,12.005229186179601,12.745722291939003,10.853431383149802,7.678643373025285\n112048,7.162454779776703,6.481137823991478,6.49212152795775,6.41912869386934,6.2996618849359,5.309462040152864,2.8428585713873407,21.427668925301603,19.343008593997418,19.987734704638687,17.672840666197082,21.15298318287779,25.23346098671132,22.170170721639423,49.33156596059403,47.973938752547525,45.69740868870587,43.3393064614389,49.15170502014454,47.261504453323504,50.6174455094822\n112073,10.750937770651518,10.022349199755597,9.626468724249403,9.519335804825023,9.263287030539416,7.210576625796158,4.04424939448242,14.955592141415288,13.763427968658172,13.572088408748119,12.488094188980996,12.845737976058087,12.23089827545512,9.821429986536822,29.464696406947958,27.935930932773864,26.13955096230565,24.697810901238284,26.081586421130034,23.943266600406595,20.489968443590996\n112090,37.5570893555792,36.16272019987368,36.07356963262417,36.029473419563296,35.518168816659966,18.615223271815015,12.648591605114786,31.085589523707835,30.57944502439259,31.041897109745886,30.784842067373642,29.66330308135864,14.561219597294624,9.299351179177764,37.1492569174198,33.027145238468236,31.662009267604773,31.229355651870776,31.28917173997752,20.75808516049434,13.416253449309117\n112092,205.93229299246934,207.73541894019192,190.05450658767148,191.09075122259586,190.0559481144088,113.9551265281655,108.01620950629814,388.806497800257,318.29620299906674,332.1355396734398,289.5486210032622,306.10373309621036,272.4777292874743,276.18994539831544,758.0015134711666,739.7696637602195,677.0261308053454,623.2427823851331,672.4177704917794,522.5875221945647,531.6168400160136\n112093,123.8066132423545,124.3359474769172,123.477215790028,122.51471936287314,125.23386566516155,75.04187053972592,72.1173408911098,149.58035409236757,144.08175818541702,144.2075418426696,140.66114287196336,142.47362227928664,59.85638344989782,65.48163806536085,201.6072925086291,211.4910643712445,196.46032461427083,183.54452434243134,192.8051641072244,87.98035182449543,86.39475923880563\n112104,9.138770146102692,8.214758006365965,8.09536029350478,8.158316863977634,7.852087894396394,5.277275059465099,3.1967902458124087,17.513421772245895,15.970935103082484,16.066910288324518,15.066679252799698,17.38094828646013,17.549084459183764,16.115114102748596,31.282901901263774,30.712273816122888,30.260565805371296,28.24765418554108,30.302935203050357,28.29739005139644,28.674561579272062\n112107,110.56087747865608,109.53294852019073,103.74241998886939,104.35809472083865,107.13662973244871,120.85622857393335,82.89037723364675,142.22286500253497,130.58986494106537,129.71598928658491,121.85403466070053,127.50753096622215,174.92077333330758,87.4117915251671,232.77508004154183,230.55229324021803,209.5181048401489,202.89227501880495,209.0238395549294,253.85725666114686,152.1806649780804\n112111,18.79013236394396,18.17898720185469,17.58577653838445,17.406627970913632,16.869355883583452,10.660049475885566,7.114994440591049,26.213397270336348,23.53253333124579,24.250124860352344,21.83965040265369,22.26075925362083,21.72736650821823,16.78537321333468,52.137444068848694,50.273911655664456,46.13164458067493,43.1099782226827,49.01960066351485,43.776583362719265,39.139711912752766\n112122,898.4190405661396,875.8173070870489,863.6065910458897,869.3824427381525,891.3524713827092,880.476491574637,755.7284008313814,1441.1162701213705,1334.329791749801,1285.447365703723,1236.1545934724452,1335.623374035555,1419.9812123594247,1058.538833660073,2139.6239145446407,2027.687087913674,2024.2853789124283,1913.2522196103046,2050.250881695441,1966.7669448679976,1635.9370805344984\n112130,9.537552303950733,8.972715639481365,8.890556704174092,8.790981177836823,8.921194368335987,5.857431977823378,3.734545483561165,11.251431654598589,10.49159935788086,10.311929199295896,9.494084742125025,9.231471217455502,7.614707769723777,7.900355960844988,18.109711530638247,18.44806293333602,15.958174666989125,14.79031049554666,17.14330820831146,15.257906741767947,15.731169025113472\n112135,549.8724329545611,543.4806431769447,528.08637309066,522.3699654956569,514.6119934787143,273.78732980260713,281.40743807343176,737.8074309949312,691.3777188706713,705.6149134622627,646.3442193844451,656.4267422948442,474.3208439527997,475.37931269111624,1241.9412168439756,1143.970443670507,1096.4579365740224,1036.1006193454705,1094.0822011606854,798.9052449121789,834.9698851989962\n112141,64.23058544167833,62.82736823593289,59.79379553415255,59.66646136958192,60.30226155900477,55.665166188538606,38.96770873613754,57.270896290042174,53.19523449310465,51.83044536518091,48.52006018873413,48.29039308612336,53.36826712989366,32.6148449022415,89.40935005524385,84.35338198684744,77.42274301853571,72.49750685071567,76.30435393649739,86.65934899016597,44.80022664677178\n112146,156.5283984911412,148.11436577240536,143.82587210011243,146.81214352552067,153.42852626977503,108.72141775615057,107.90780439332505,280.58214631652976,240.83421884830628,239.71730863400586,222.19795201395758,232.43737777724448,206.1888367142162,235.1205430372013,506.6133246833016,519.0548073271666,476.39471881221783,436.22642013783246,459.2643249654605,415.23685353082044,406.53683942996156\n112159,16.295484475855574,15.810316169659151,15.571118195022963,15.528924929493128,15.28078943192634,9.095879996566127,7.256745216159446,14.91004010963002,14.641025696351,14.888640946151385,13.969085415228015,13.609036185104783,9.497231672498337,7.621682858565755,22.93794211810519,21.52000468609448,20.286138024396838,19.23132038675735,19.680817763048406,17.29998001333974,12.973056290690021\n112189,129.96554825289155,122.79938303127842,117.97935068333399,119.1168407413062,117.95823073665082,93.51855749139516,50.88645118219211,124.29643240736812,120.0550918059442,120.74526167974345,112.97027417371386,112.74767245751283,103.1553145143831,90.43590163363795,183.77320439225127,176.29732312649827,173.07575255941447,167.59667985588345,172.48615942528565,167.93112053348563,151.70233022827838\n112193,8.714325578978382,8.333814492411252,8.184540466870638,8.20987389674649,8.03536078206473,5.676057125335445,4.920756305882781,10.291236254004264,9.543485368174164,9.721223878919659,8.758660951960346,8.762494789302517,8.293130991568878,5.363158555712315,18.1603736844281,16.320087450933535,15.900334036600846,14.841265280458863,16.07827356760686,14.251242759831907,9.979247854126891\n112217,18.62337208663275,18.48611837852044,18.15040912859236,18.135618947629222,18.400722671214336,13.96596566623316,14.024452897257527,21.427702173433776,19.803862481132153,19.531727520332996,17.837942382478158,17.72914067644448,16.592257411806838,19.198028474761795,34.52709556580602,35.55706877336757,30.742241782165344,28.30767804792588,31.676656013808955,31.09524937764323,31.236292630748874\n112219,15.312665785331788,15.09788698243275,14.670073445234298,14.84088360564004,14.83251207194322,5.780634729580318,3.6993621506292804,14.940908169593877,14.21418232279576,13.970452134811113,12.868689320145947,12.18746477909705,5.694933199925087,4.100829660861267,25.019830287700547,23.959369161083785,20.900154263123703,18.35388477774615,19.527940699273266,13.94870274040431,10.300820687986697\n112244,18.539807046878096,18.538023376459602,18.40701659594579,18.34311069401314,18.477863261184183,12.097454527920489,10.656488766490552,16.670701470510487,15.871011007836858,15.69559239699948,15.066632218947488,14.470366902645168,9.476373402220858,6.880893917674988,22.199821646228475,20.58228520573858,18.337331247698632,16.812824619256734,17.478986332055744,14.5217177078701,8.164758044879754\n112253,25.67205353774404,25.521139438896927,25.21274855295941,25.163918670174976,25.50636296706902,19.861739782399315,17.210557844679524,29.03215039489433,27.94564692469023,27.543613976606057,25.336362308040435,25.15876582962957,21.355885579558436,23.516411067864322,47.84540867525035,49.22749171757644,42.24462553890313,38.60175019057107,42.83708018365416,41.547497943155015,42.46624244206626\n112261,117.58186845360524,112.39467478808244,108.79435628344511,108.32375647905612,107.95188622968278,84.07524230200615,54.62552155469026,147.2453342432122,137.08423062290345,129.96538149232427,124.56438867945792,128.3969372752616,131.12919658035253,89.96899891453555,251.29864877182501,234.2497445653106,223.3570718208563,216.50443354128566,223.9208126254508,226.6274963863493,172.25249529353883\n112263,87.27027282139252,81.09902944130945,78.57235548842809,79.32378658791924,78.58953466651053,87.91114361137427,39.50726784452214,97.17383518818431,90.63216623266796,91.0246673286396,84.41226291260628,84.36376803190328,102.90361190748754,52.912071329136566,174.93707477660038,168.057211304826,160.49482241912617,149.7953609848081,155.96960357504403,175.3877602105686,110.97509826119571\n112265,31.692097961480066,30.564288284905757,30.13156828848079,29.992056047878222,30.082163518804204,16.183429258402377,12.076933557510197,27.12507806235449,26.13303018325562,26.175121697638385,25.49056046569696,24.759145786983197,13.20468221212998,11.070194790664795,30.63784596784988,29.431316602936143,27.426200435672474,26.228028093884593,27.04666431591394,18.49580166189714,16.79644842443467\n112272,131.850591047503,128.52001624382393,127.89034293440345,128.0125715303287,126.466825211762,85.12373727648762,38.53328607552456,141.2063062703877,133.54199656824125,134.41715790127603,127.64200415551679,127.43546876840458,118.80853807844203,102.77657580634921,227.78285870637438,215.406849477081,209.04907774471172,196.84649879659295,208.2549054283768,193.03722200739998,189.91131394910263\n112289,333.43201427960236,331.8100099607646,324.7881281328594,321.55152024536716,322.18030485107096,229.8491274845809,230.790055629305,437.92803441856995,419.1888182929529,412.1079173192384,390.358882182934,387.2411818322995,261.97928015428755,270.8223226655544,677.4136885888657,650.2958923687991,610.6726932095019,579.4329346196037,604.8542755649587,348.084121527182,362.38258371918744\n112293,37.96218256000596,35.96250908942475,35.921237865526955,35.81071491241426,35.154406021261394,21.769710830958736,13.947737996510114,35.03711498661213,34.145146615393465,34.8925636054628,33.252033271611964,32.10343726974645,25.056286237525086,16.45197430064926,53.703055189263075,49.54900592343126,47.94920366571566,45.37167081296779,48.36523872503479,43.91905613601224,34.41702814359219\n112298,3.632430235791687,3.336993896313958,3.3326770233179532,3.3122299602200043,3.303919321883003,2.3617548345186457,1.34349862889943,14.145440234884939,11.660699844250516,11.401163441587801,10.23347062667223,12.21867152394379,13.436758325860264,6.669193209676017,29.284864665801607,26.593311518073552,25.293773534223643,23.928189802610714,26.88685265052587,24.296024737921915,17.464970979728513\n112299,218.58722594900232,213.23120934056047,200.17710719014775,199.66604126429806,199.91684679609077,97.41797318821926,100.4910636742554,341.1230664119566,323.2887659906951,342.4211025376188,335.0281914079876,351.2648210204093,136.411519892706,136.80125521816265,616.9816853445781,588.56184931461,566.4028910736408,539.2858800801523,571.6305810762314,251.70243255329223,257.1692931555597\n112305,369.27408764506197,351.72989461086036,348.6107322877983,350.12238655227935,341.6978175386338,204.87997065544792,158.47616925873916,295.57344938324326,288.99108567562615,291.2350366011836,275.93254567015396,268.36967241739296,185.643250444632,142.97298790835737,343.94057024299184,321.62389538082067,307.2284397132537,301.485006864387,306.9086080828018,249.56501583468633,192.14941883110876\n112330,3.3599471057062895,3.2591134815882175,3.1467816913818996,3.112290348518017,3.0927812468475997,1.5624824933250747,1.0501803891900627,5.240366272885837,4.542448958947169,4.683071916930417,4.104072837884975,4.250969332707908,4.20269246763636,4.308201998272592,9.640385333302813,9.810738696950967,8.825451366625988,8.153343838745029,9.266800487954333,9.70065923423105,9.548392669779853\n112334,10.979329373952012,10.383467131344155,10.102663442993908,9.99945017281553,9.680176390878737,7.000543314606181,4.340082696263984,13.973067301088012,12.68318510577999,12.929165462171442,11.577726781012075,11.813744726133791,11.17887214761203,7.773585567587881,25.606777629552358,23.56897065323761,22.613296309759804,21.236403122722493,22.547944363179457,20.346056293003883,16.562773074283335\n112347,59.67190456022591,58.804596423181074,56.96504522043705,56.86477795752515,58.23298498226095,37.270418751800484,35.662313681407845,82.93053861318592,78.53996915176967,77.07425635684683,73.62259606948875,75.62906362055935,54.55554925208799,53.94481864432915,128.63355497196463,123.28380549162121,116.32043454546732,111.71250216043798,116.3177354639339,76.39187498210597,77.70833253983639\n112352,224.17308766471695,206.20693665978456,194.9498844187614,197.607947784182,190.40052473625335,130.9545171591868,133.66144188091383,343.0400427432805,319.1834770705759,312.9177759448857,286.44525707644164,306.7982956495073,246.38354114994684,254.47416984771655,556.9205893414842,538.855712430955,521.4265548706551,493.4299483217343,513.1398605507122,355.81118870503445,364.5961063312432\n112366,6.924896118963623,6.427061151804476,6.410441289693049,6.424722815440846,6.321181029568049,6.1407230269606154,4.830115792086219,8.485709118437022,7.930009290433503,7.432510590715391,7.035392128572537,7.291701219782418,8.506935729947545,5.033428627918299,17.77312125445286,16.223075152005944,14.809166361608941,14.311615631861374,15.210084591158642,15.97714486292315,10.292016073579802\n112370,180.90168888933837,184.48093835453847,179.82162062221656,180.36816071991177,178.41872605600145,97.81873168106966,96.5661870997693,155.54404901510867,154.39579692880474,153.5549929378593,149.1082931795342,143.77918661107446,83.37737603492147,78.61178421104896,190.94242974782878,183.44754946803653,170.7258773688502,163.685799895551,167.3297826209682,126.41326321960325,120.35544954014858\n112378,427.1387569708776,406.2748773925466,403.12566284737164,402.50433832514,405.4906154233525,341.9812615851647,354.75476397737015,869.4185924344076,806.1566278645504,791.558860201467,751.5419551238986,829.1875285431418,750.4436247546967,756.8952108638832,1245.1073028464125,1209.8202537265913,1207.9714983336546,1169.372485165303,1193.7941257244922,937.288719608068,944.472932661873\n112390,27.663215869655534,25.701795075268063,23.961060758977126,23.850064429172456,23.222856893616367,6.860991749883984,7.924514175506001,30.94232291550539,27.555615923100465,30.173066394085467,26.161162212660376,25.857194570372865,25.923977167169653,23.572108816564043,58.89272230596783,55.065109443263125,49.72911982100427,47.849230191482604,55.221885599447724,51.033190510282786,55.12480418154042\n112412,3.791965655243973,3.5283706550099145,3.3799353529066964,3.2838180976454066,3.2425184821904645,2.5704248327797603,1.3831386318791923,8.978081543628177,6.998129734032487,7.056982369576069,5.901251919096701,6.899333124520105,7.556591784683103,5.359581229111402,19.66245844332774,18.87352523935346,16.396843858863782,15.826362067243895,16.876632859380763,16.074625216484577,12.77139467914395\n112419,14.548160800799039,13.79363860392691,13.645159573932153,13.52561885423864,13.146691362135789,8.72682693160074,5.2374373299231936,18.346482606969296,16.13657526008925,16.34067194001274,14.975302605825492,14.83596771058947,13.844001693954477,8.18797576595872,35.27285025380129,30.920078293881616,28.49922540780157,27.05888389283546,29.975591183009357,28.505528591515287,20.562977367488973\n112420,41.148588074363566,39.588719383948,39.39999817425621,39.42878542766193,38.67606369021975,20.769775718269248,14.789518362749622,45.48140333226093,42.21229845795873,42.82745565608107,38.78832764779568,39.33075319301137,33.47072808054891,18.75251837466202,81.10226889320035,73.84005728577583,67.26671459754952,63.48572402851141,68.93157744185707,65.28728997800322,44.98118164958875\n112426,40.39659193149181,40.351293026108635,39.47370189703902,39.436170901701,38.667802188858495,28.621050998168645,28.317029306186104,42.139813258050935,39.87296284155915,40.7447532134012,37.84488105038638,36.859080195356185,33.01604467250527,27.25168831926401,66.21256326300245,61.52925809254308,57.9845130680997,54.789587065235466,60.525749460422304,54.66636272666221,45.34556936739469\n112427,7.927885173925609,8.002488391128896,7.443210072472753,7.506441959244043,7.450165313627081,6.437412363447106,5.0715550767404345,20.505353250205236,16.346700532025633,16.572672287887762,14.343691802734892,15.695225852632609,18.17513561641748,9.971181247327035,37.76707638053806,35.25956353579131,31.87462526757988,30.12830705531368,32.481920162679714,32.388477199952995,19.88161765560706\n112431,15.926628212544841,15.345208756966889,14.970746588259795,14.951049862809029,14.581291157170508,8.501091390892551,7.587638086206253,27.672601313285984,25.40286968534396,25.924628874999502,24.00266423397065,26.25336377137604,29.159549843024795,28.012532268773246,51.5207067313837,50.369151960809084,48.064143489550354,45.43598742793865,49.76631171670491,49.54416086641159,51.09380999084991\n112456,4.607791984435185,4.416550683498795,4.387480091612174,4.380867263036014,4.295168683075295,2.7066123941616373,1.5513736870685422,6.224056613185251,5.953126355948178,5.91849894340585,5.524239188253637,5.593031428353383,5.624796871186842,5.816540061162885,10.783637955112583,10.530244916906762,10.235649546074892,9.673206091609071,10.168813720859967,10.301030369037633,10.933886832103926\n112463,4.673501077131671,4.445634171516949,4.471191738230549,4.458048993713084,4.40591738588919,1.290216638151111,1.4678003712085583,9.816162565275667,8.293588159307934,8.373804744801214,7.700083029061846,8.8754989244408,9.476526533560776,9.416762785884341,16.902851197062603,15.76008664627456,15.525362785122395,14.25658294151805,15.902320879941353,15.508679128704356,17.69646636807096\n112470,22.814530797091255,22.717531142346,20.67568312217977,20.486577811098424,19.906550113986196,15.560396293755248,7.030359777606954,44.925138194827845,35.445273734168396,38.42058909639804,31.395114762522446,33.95132195629365,37.61853193292365,23.714652678160242,107.69747187151377,107.4631408023228,91.91018516936339,83.85332415409621,91.19979918149684,92.34488156331663,67.66690283366603\n112483,39.24824730114559,38.29496625661182,37.944379939039706,38.02243308608735,37.201484027520024,27.08549679678276,22.5368591379027,44.19855033970734,42.05733657222217,43.20893892025375,39.410910597650386,39.49377025534616,41.0319822817417,26.439690108403244,74.13922150346045,68.6384347341833,67.21481910088058,62.73268919443464,68.60605192442968,68.73115687959027,47.834136026026925\n112491,9.569008169578586,9.019832534932288,8.927120488853989,8.898230709061966,8.739426497879244,5.162191797453609,3.5795168542413216,10.169091003198426,9.982119064331531,10.119423880964096,9.344415892301596,9.172508088956736,8.311104089464289,8.216708816653899,18.07428849276054,17.476421122492567,16.9912395746491,16.05778027638256,16.98350237551932,16.694770659961584,17.78215496126038\n112498,1131.5853552360143,1132.1457985230365,1113.5195518561566,1109.2459682675244,1094.6602914616535,859.0420894562164,826.6301540216923,1129.0059498145085,1078.4604855443458,1077.5959873446652,1023.0001226615553,1019.4730512192378,965.0388797480508,816.053329626533,1555.568246853804,1466.2336195397959,1415.8193668218191,1352.5431434769125,1392.0808141476634,1340.22135105848,1117.996995938455\n112505,6.123055503506848,5.548321411110131,5.658293228012777,5.612187205616394,5.390085505024189,3.8192995413180593,3.1486233567748485,9.654072992121476,8.518032014985332,8.298344923425057,7.226585424039257,8.06847026936353,6.855925929189269,4.7430203124895405,22.79419420306377,19.92155231004266,18.589817915045856,17.577987613681948,20.005163808119608,15.526966590154558,13.788040206394111\n112508,31.16932325235399,29.28595912591261,28.52105700102236,28.251406490704998,27.29187937823803,17.13053493761112,14.60929300254064,47.84490641627377,43.70302570622139,45.07053036145517,39.6859433470695,43.570106324495335,43.147729681472086,34.00722390297498,102.25661367065277,96.11789544034166,90.88489455655936,85.75552323375679,97.22025523819077,84.45280959411936,82.62467557753978\n112547,16.43313677580096,16.050092715800723,15.883284273862582,15.952718994959652,15.547085164285617,10.643526471075301,9.864599828125199,14.93691545450203,14.733719444728056,15.012492167065568,13.563496976586512,12.932044606610832,9.525702358752044,7.1219857272084335,23.835811812633892,21.437438284613048,20.892241243364897,19.45057087038105,20.875626922289626,16.14801388693401,12.797495868796341\n112570,18.25401093087628,18.225108041357263,17.52251716730589,17.793648771972162,17.880325373745553,14.579378603643145,14.856253826327613,19.94203492705195,19.929578789811742,19.7132615907777,17.103406765475306,16.06774811971043,15.793844384043641,17.59479200182174,36.60214705711828,38.08730486326747,33.10192313135322,29.121776679652957,33.96858557250027,32.285797475151796,31.036882458741864\n112571,1.9780591838066277,1.9471349691092394,1.8610309442739674,1.8677826532086115,1.8491013539733794,1.2328570545695754,1.2937188894721834,5.511283767055267,4.5814348878702456,5.072087267680604,4.069370162416273,4.172725371164285,3.7010180849794767,3.947567685199557,10.864618742296916,10.647417952875331,9.630800490844122,8.939214011467532,10.358054336044239,7.931956627156069,7.669760233248251\n112585,187.83039173562236,183.7996449471031,182.2337337626377,182.99072682856044,182.29337169656196,166.17384219442408,169.7441337921174,219.31553722201753,212.24706433608657,209.8438201502624,204.760463941574,210.99999469159113,216.35720498760668,220.94914825248202,288.8831275970218,285.18137082761876,280.5337845354135,273.3277468308326,280.2230009348361,277.382249968907,285.6029638218187\n112613,108.33315560529333,110.06844363623496,105.93421357895421,105.43351343535987,106.32690090928214,60.49505301955982,55.136256213566966,159.676831698067,145.30602590288296,144.71885868803707,133.07896285260426,137.90957099121928,86.45747572059975,88.24240691530096,282.2398757399751,273.984807240143,245.65986241392255,230.31904814000023,243.63454955154995,138.0250173677841,142.20912823579155\n112614,3.9671211191428823,4.005042146027502,3.8392326039262796,3.799361607632507,3.753480651639038,2.771465155635396,2.4870172911841055,6.788259398925431,5.6011202165595435,5.8326246079867525,5.120657858300588,5.318761344371616,5.237196041727079,4.446548291092532,12.542154370272998,12.319601757589332,10.921660743309035,10.167844221187314,10.88863703416626,10.142869207295847,8.90954126377963\n112616,12.776722881942009,12.047727894284813,11.760290625466883,11.876487361524502,11.84099587020114,9.079544706706985,6.630586406547148,19.99630828235988,17.92907417632461,17.62370876470445,16.7083740137595,18.201015181439427,18.262442472520604,11.748175886280935,32.09768625967966,30.741344696561885,29.647863973688313,28.190606731560877,29.978394055059088,28.10463643467196,22.1116922262794\n112623,27.584840966299154,26.546040250605564,25.915545796215554,25.684249973072387,25.04390388342293,21.793169857270065,13.325540174387303,34.57461945831209,31.942169902050974,31.949875648923452,29.323675931288058,30.08381365306464,32.98431554495079,22.123698700538327,63.39768548872811,59.591585038619925,55.36793754560816,51.89728967601255,55.763578148883944,58.74073221424897,42.89323628438939\n112624,37.83756629662614,35.75278639196846,34.824260611842476,34.68299700003681,34.55346834248718,30.720836014510112,18.477907266574928,45.68505456555154,43.83317570262565,42.603801082865765,39.645365387700316,40.57026423383672,42.85334933622033,37.64345658060406,69.8689099592254,68.56745147822095,66.88551378345234,64.30076907244872,66.23022501209574,65.08533523739855,58.62654404758369\n112625,7.134387219004715,7.051858249612024,6.780942565893694,6.848540335481134,6.796652727695615,3.343516137483545,2.5708124948448705,14.285698361219923,11.719859516377717,12.575220716046585,10.490065535859571,10.503765449858134,7.828198138051431,5.9082672778411105,28.243764157934578,26.248529259988025,23.12865080878342,21.832715979131045,24.57256767488074,19.98557325743282,13.713252867883124\n112629,71.10254904884543,69.69071086489623,67.50484309764111,67.07965133594169,67.27362671388278,40.334055655619935,39.350386970347934,113.26622722954181,100.76070003164166,99.30122212520408,95.10603775826051,96.96551850976672,78.34034184888755,80.37770360492668,188.48101591145658,170.59444429081984,157.07013159509398,149.52472416936808,155.93402366787313,124.4915239645377,126.72238735288465\n112630,4.486087702844414,4.053066432027812,4.088092460120956,4.113004145272303,3.9681503735469494,3.085483013360275,2.1797031664005773,12.741903997536602,10.750674941676932,10.803277961990453,9.55129611966508,10.914707431351786,11.4016026171684,5.866871126110008,26.550233040177048,23.933076369989973,23.652819398197416,21.841925047514298,24.633909491587964,21.43662462258472,15.06057715662065\n112650,4.452483086600357,4.2395078667522546,4.262458669048117,4.278237230270845,4.195339921819347,2.0134328143725524,1.4923058475486228,5.370659834040424,4.723107731842472,4.664242505263601,4.312445184014983,4.23951439352189,3.393590563313654,2.84902492050144,8.049235252504769,7.059627522675872,6.724285066859991,6.396009096649159,6.691858466410241,6.103831452753609,5.837432600080388\n112654,136.1205034650442,129.98842540051814,125.18719457706916,124.6372384750234,126.80322702387272,76.41713896847051,78.16798719121795,226.05917191233144,203.12905910806344,196.66606102257924,182.22910394022517,191.331902714132,143.4378143562672,149.0033759268081,394.68043770770214,372.4006357691186,351.09033428741145,332.61544055400066,349.3055691084085,217.04888227362554,223.62748601953663\n112675,24.933783502585865,24.855687948788738,23.953935167235713,23.946909971736506,23.64180068234074,13.183996060607535,8.409322505445314,35.3067016072115,29.186250095745212,31.710290798684102,26.15141930710691,26.769241730322815,23.99761375247433,17.252233566594143,59.36691608309423,54.82594098992604,50.82066029887714,48.83016739562425,53.54757506272863,46.770655416801084,42.28799193380518\n112677,7.698954955082978,7.273802974876568,7.225263239775893,7.245613356072906,6.928537952021535,4.9683813898038,5.315553577612962,4.803943123346312,4.903625198147159,5.0539161166970565,4.5166124657062525,4.097956007930281,2.8504206959610774,2.5854477836054484,9.553144927112635,8.648841206581944,8.070351532787438,7.531794559802315,8.07532495898729,6.350763103480599,5.424639401386732\n112685,591.5562749358414,576.0640987645124,546.1479195757362,537.6679052866882,557.505919101395,303.8959009079024,285.8688877165449,903.7183607571953,805.6457423280131,821.9276565823135,731.5228227645844,739.8393239373918,615.0338578261304,641.8217397157462,1458.4991528835565,1401.735341239839,1309.8088262834592,1222.4091235223805,1305.707864850613,998.1563654694553,995.9823402544106\n112700,81.59270243767689,82.40731955165029,77.69465427673562,78.27735952174403,79.09925032321796,44.40176094002666,43.89967831542161,132.62822410053198,109.93214838382949,110.48586241951291,97.00859267790011,103.13484519783572,82.46274538728967,86.74775251484769,220.34314428419572,207.2069903875039,181.18910820189413,170.21168850328624,189.25863680337912,145.1141055820015,145.7998172918092\n112703,14.681916588465354,14.136494381592879,13.689626223417534,13.730946746667184,13.99692411692948,8.899758541554574,5.743153322186166,14.435399915198317,13.612785682134916,13.55725978204852,11.996013117670792,11.529855239355511,9.253668177464672,9.58841134587192,25.57509081716212,26.47963138111361,21.391073254675295,19.344257185131205,23.00595690359097,20.39871453654793,20.14258340863229\n112705,32.21345989718975,30.48157828097192,29.977629721473075,29.915440113145273,29.182670196440952,14.750070124341882,6.9559544770929085,34.14810682511097,32.26109262932403,33.26769981920362,30.32060291421637,30.44416333245123,24.6118808227327,17.584423362784584,56.258330204085304,52.71747282475613,51.25647181873802,47.84769546995308,52.32109631117087,43.146531757336604,38.4453811775457\n112718,14.398230754010493,13.646079349759408,13.35821491494442,13.364515410353514,13.03978481573455,7.530837489810393,7.5840656623308025,17.725175303037457,15.489077594476608,15.345916991808005,14.205083112784017,14.217127986151008,10.528208822496017,9.92257086928803,30.088198751870475,26.462107455211648,25.01154409528025,23.3708450519218,24.363391595511263,19.000077295964726,18.51179328946381\n112720,30.43528535803826,30.005691584226014,28.89991023965971,28.680437850031566,28.53887188156959,20.808769104223327,19.055238595280272,48.08308627192729,40.43907244247271,42.51093993599995,36.571107862911575,37.08827424815332,32.405683227719926,26.732221496696777,84.06164885664359,78.5863563084435,70.68151770252209,65.49398706478406,72.78468820895021,59.84950991281983,45.518177546646385\n112721,280.88698502119036,284.8430792007713,281.6370389961925,281.27031927422615,280.00098074485965,212.12272589893658,181.34666790326176,260.3069735414526,265.53398683106997,269.64381694365227,268.75050138232206,267.3733154478887,220.81184374467279,160.0960660241961,279.40277833779624,283.14412139702193,280.1973552560081,278.5077119993591,281.85207633601465,265.24845233566765,160.8774000793791\n112722,665.0381958361306,644.713974945861,648.2545128644421,650.4879480189929,646.474440407871,437.63382138588037,344.56870316469826,708.0558198796467,682.531814774857,665.7692987298069,648.8250905991767,668.9602246325534,581.0565773402227,466.4135507319851,939.9632033321863,917.3251542430644,907.5746811168253,879.4601088140274,899.5841278029436,789.5955591620602,656.6657561559203\n112723,355.6550818098397,350.82025296903555,345.7595100845005,339.04942281690677,336.25061871636746,192.34390209720692,186.7035968015033,536.2536673659281,511.67979756975024,535.1225608195107,529.9894256102988,556.5776182383629,192.66705653075752,197.5600723893513,828.2007207555775,801.1143363788623,786.2435131972967,750.925263174503,780.4655592405128,297.0136150239831,306.4393127971273\n112745,16.79185141260582,16.676589317144067,16.25072593507632,16.25902355462456,15.825702034762243,12.209035111705782,11.93225189656043,26.364618269597145,21.978854106302986,22.303580828824803,20.23813838412096,20.83476500910694,19.323281088742267,12.4157794145154,48.709903767768154,41.2539674708699,37.837853470353465,36.39010097265372,41.08524314098899,32.0963614940314,24.226765621094973\n112757,110.67943605587216,108.86405810094482,104.75671280519144,105.11867511125733,109.34618137266226,72.63855302052526,71.58900949657325,203.91576827623928,172.84964872605997,170.82801706519083,157.1725126858645,155.26310691316752,129.32590064020906,148.49396173499326,388.61288318533855,389.6677044276198,349.8745080828494,315.83152259114104,334.18764039894,289.8398334969642,280.8094632759356\n112769,49.070777359139846,46.41347079599553,46.006582389813545,45.87768602035972,45.01724558171497,20.092265942544643,14.948202416709716,44.83187986019416,42.39455259874326,43.022988812492606,41.44400095017688,40.57476040944027,21.601313004214052,16.26595218949201,59.36097821521822,53.131946434514575,50.13104299653029,49.27754587689933,51.925085892419254,35.53344584713234,32.5523360531083\n112772,264.2801263369129,249.5537207352974,241.51944407060705,243.2961786292183,234.7612589198319,154.91785237331283,159.4643438432659,446.96113019420915,406.2102669795968,406.6397953548024,380.4705103925206,397.14452357463773,313.7610745153413,323.89553598369906,739.5219425538371,715.0444529148016,692.1983352184402,634.2323755772148,672.803113893778,440.8574888684903,455.94148631100296\n112780,1.457296608956428,1.4098927585430745,1.3783585225404176,1.3573060439191078,1.329220123023383,1.1989790979399402,0.7979127431419246,3.0092358214557215,2.474802116189294,2.4453358062577313,2.1066630206966916,2.362795001056052,2.70582262332185,1.8099675890424183,5.802744266703186,5.556375282808037,4.8537987767730995,4.729174236747349,4.962660972779216,5.064857028654051,3.595768212194089\n112781,211.89885204396242,207.18402749697034,206.1652285205989,202.51782986387158,196.95108478489567,166.25150061489506,151.43959437284326,209.25228388565006,203.50330659893757,198.9050459574282,188.57593480674598,198.09264212795378,207.02545383158395,101.31911803025685,326.14692074495065,302.6669119210627,297.51662649775994,286.216916078644,294.8495981335227,321.81970127093444,169.43622195250782\n112792,508.8009861593055,493.23065831766735,480.90914397180387,483.07672791059923,472.29006241972405,260.5864251424767,268.8721140931587,734.1950501500093,676.232764932444,681.6405687865866,618.2765371324872,633.965074494084,474.8012850106228,499.49944016416583,1228.9250825462334,1172.3927745541607,1118.2568768190147,1023.3617574741472,1077.2917832039095,666.5012362670092,696.2563002013485\n112816,9.325144539643121,8.826453571995984,8.737161743365196,8.654557772638105,8.453174456900756,6.031411771094743,3.7386714690543306,12.817652118506977,11.444281281684486,11.102904761291857,10.116389676097251,10.243192692621157,10.264930043914756,5.582185285434429,25.73290798188345,22.859301296923128,21.179550924485454,19.904813494404387,21.00610973702299,21.43902289146266,13.898886094124851\n112832,35.50438302923636,34.02384999810009,33.5520167442891,33.60878005330107,32.80965321083642,22.40950625770283,19.294891550826232,36.20492801325524,34.484994197564724,35.15087243716645,31.7407296315334,31.523315952187534,28.42031581389637,21.861096697293654,60.90223420374203,55.25347228738493,53.47557008078246,49.83465276611249,53.92746892261861,48.99607684944378,39.58633174014045\n112833,12.236615964490696,12.096382780488943,11.678585973100764,11.67932522768701,11.3953401891034,7.529355935280826,6.375229974555892,18.759251108206712,15.970301361765381,15.66298598264331,14.288316567888748,14.230112582598633,11.630072788902757,5.963363672612768,37.388435198567734,32.047553079293145,29.043484182518142,27.303167787485254,29.159022161899415,21.686854580068484,15.127277616889595\n112836,13.067621599899605,12.605577854245283,12.361623151680375,12.334123819820515,12.103777592916076,10.064730515706685,10.529575645286874,14.506203477250589,14.196725512714867,15.152769054454822,13.055060937004951,13.399802263361584,14.192547522045734,13.13049254065078,26.558826135443635,25.601938788807445,25.41364579824122,23.366951957891946,26.676672942478312,24.0242235706827,25.498036993360788\n112841,14.985778980962962,14.440817205856915,14.32349207286115,14.379516215463774,14.073545564809248,11.527703172317853,11.717823694375427,23.849114154357338,21.156330234164354,21.12327858880715,19.400681974461207,21.84048103899506,28.28439259685748,13.090286183526516,40.290954200258376,36.65454309128814,36.47339140556779,34.14857287742766,37.03361748761715,41.247696511117525,22.496054383361205\n112848,196.2990457440826,206.57452198179357,183.8412164290974,185.2993361277096,187.22886676944597,98.92658802566702,90.24924844134499,299.4953968621582,247.83841826061592,277.60542930673483,224.64581050419523,225.47815063261353,274.3527722641355,304.76913383591005,730.9842827252858,750.777621247479,617.5246527204959,560.7501359778913,600.3624810881718,757.9684630674869,768.0245012294138\n112849,183.0436929303502,175.28068141529465,172.61991395921657,171.78100032352492,167.99372692680626,107.32881752454779,104.76956865055702,268.5518641820693,251.80769152613135,249.8950557625582,226.3697423818365,231.19004355750553,205.5057106432946,208.54079871744275,390.5494799004579,371.10578031397404,358.71770494040095,339.7899346584491,348.00519345517824,275.3771571729409,277.8979753872728\n112875,5.193334654327228,5.233775056628762,4.977404124325377,4.931411561517162,4.892846768335921,3.189512104340322,2.8972568561210506,9.210124395088544,7.942512553591595,8.377042461347605,7.376083002539631,7.75032082083691,8.113512031058066,8.332592540590392,18.599455015749868,19.35779771894081,17.182258761969983,15.882994731598023,17.030116858944435,17.40763200908107,17.330507165616062\n112882,17.227819127494953,16.316600412223288,16.157432871539285,16.17577162958686,15.728300306642986,14.015840938434348,12.2086254700552,22.50172532965526,21.67379531351184,21.957469528905953,19.784520026200262,20.571367041252064,23.715414986350307,18.386360454388246,43.62633299762937,41.206489605267834,41.38812341197274,38.346891264501224,42.528189005106476,41.59407518427264,36.284452116226724\n112896,184.05292590545483,178.8804657884406,176.2238437838767,175.378896432781,176.14558039320104,86.2108492298654,81.22109714192459,235.95260049790053,225.57751521364165,224.6474547976576,210.2501011363853,213.35945974515727,119.13694802130674,120.73935939106046,334.2582293752402,319.59651981206986,306.92142742651697,291.21503091451854,300.8012798391283,151.43506503296098,159.00247281469842\n112900,5.965410503331957,5.499500277437176,5.46441032406981,5.433111136903164,5.164096152708859,3.632635786653774,3.115656647568085,4.6608536655792,4.700924625544515,4.664472043652775,4.32355829165439,4.107067090849172,2.877710022725987,2.339915777127358,7.035457555045262,6.491714506742149,6.43668594201666,6.042253613516777,6.508468192758619,4.945192303297623,3.6563963565089104\n112901,23.96705200487054,23.457496405293114,23.5129351249844,23.630415382614906,23.204507841420504,14.243283930966387,12.11068397977668,26.936047655919737,25.003350916664825,24.34911828541349,23.034612817842802,23.00752984534524,18.759122844267928,10.629944561899874,47.3366213826483,40.40928566513639,37.64510748207798,35.617386446016994,36.672587483452034,34.743623661450535,18.327210716556017\n112928,11.088130319137523,10.508643176291503,10.29508932961319,10.282562782425485,10.039937611736748,4.587970685035395,4.942998321148851,14.16881116038944,13.183428401126813,13.78254565258317,12.536074888410873,12.895380987069355,13.44377169579966,13.277026211915244,23.912271739016994,22.99047825885674,22.47693379831466,20.8831096388111,23.087144413145023,22.588323346066357,24.09710987612807\n112940,27.12060312056918,25.956537455536473,24.796149577597788,24.48828569067994,23.66084380459044,13.382785940857566,10.116584969587176,36.10604632930921,33.58003733579147,34.27792274996068,31.88067967359445,32.189888014538845,29.547882893816176,28.077953544516436,69.13383005652814,68.96489320604245,64.49779069003112,60.72314106064887,66.01361917401306,60.6522784745032,60.11650029793751\n112947,19.59384845519186,19.292124100791646,18.75308717549812,18.736252892457756,18.44869417467982,13.153949694201469,9.975712136505948,25.739424265942635,23.044802903191012,23.623676865602945,21.638602631230697,21.726411680658835,24.688362633131014,15.098044043727624,44.17162083557034,40.241583787925634,37.558675816657725,36.028350566047344,39.624158761294076,39.47998496413973,29.765831228825153\n112954,239.74915391459635,240.90639110598187,243.62369061591494,242.12500700895282,247.87424955839612,247.63310313348055,240.83849392234984,357.9849095183191,321.4666680538299,334.22364767473294,322.4715680839978,331.1476484158894,355.3541786052125,345.4778166307087,442.0043541562939,460.49689737561397,463.3342546363472,444.1198482761992,483.0332386970027,445.16100922746097,460.6367390322709\n112955,128.60950725435177,127.79715831234009,124.7170263663779,125.18495611888423,124.67557607181334,98.41243138783827,101.45757294668638,130.99075613498724,128.07270127981585,128.14545588498805,122.84709414404958,123.32641810812125,115.94311271267773,119.38650869741294,184.01535611565947,182.70064052079732,173.12309699211565,166.8912553536173,171.98320614907962,165.03752340548485,168.7562757310403\n112958,51.663878064127545,48.95902055278254,47.86540607607018,47.480294980678465,46.25817828177037,41.155544700645585,25.77173878163812,62.89585193046556,55.581277817207415,55.411788118780635,52.040493015466694,52.72022733141912,60.560540485614226,29.7760698635432,110.47231131130043,97.42791872680674,90.54463284070115,85.42785795133919,89.70974149874382,103.94367794192819,49.122376823131475\n112967,70.9699633757298,70.08648007128427,69.23985175781735,69.36209800034915,67.91581359140217,41.56471915552669,33.02257210654332,67.13808115235483,64.94041175577026,67.52325456539889,61.88412540257085,60.35680188700183,46.790416779671425,31.241462392832474,111.77731093545613,101.94931715267158,95.6586075863377,90.01398226753645,95.52673498439538,86.86806587293395,61.09446818952082\n112988,1.1091194827646549,1.0441025694020503,1.0070408393815902,1.0030042023680728,0.9846897060753241,0.7967796995849746,0.3705843666376417,3.0107318469709576,2.6209157481962735,2.528179100499123,2.349607227110891,2.5484674343207936,2.724915442426595,1.8286086666139563,5.412987739751936,5.055390354814979,4.871665378243433,4.582045046898594,4.813701451803458,4.600233628227477,3.4597073783524936\n113001,157.10067020022768,158.91520133343582,150.30222404866657,147.481917000285,151.617554435392,88.26871151770067,79.02010509378636,229.69464877507633,209.67322552698963,208.39301784711733,192.27313764944012,200.84041325431906,131.48885306014384,134.62133492038583,394.0501383847558,383.2657840152178,350.96540804891185,333.83203568631484,352.29354983989447,203.58072175628834,209.78681029808794\n113013,42.21492353963342,39.73032679257701,39.89109711628707,39.992083340052716,38.33595325122973,29.143636506914586,30.557287982289044,33.01062809528989,32.786694676506244,34.28098436806755,30.86624409133316,28.929287083171847,26.327345805686075,21.945184360173627,58.420007617573255,54.42352596321987,53.11520478812469,48.921789704065645,54.38792786040147,48.15369183681732,41.906062081123494\n113015,33.522293228477885,33.78496483593909,33.78231505656148,33.75207810059061,33.47041793367765,16.539994632909835,13.813834805717363,28.73002496522983,29.170948795399916,29.766788463994025,29.48177740487159,28.570707059964278,13.459292578960543,11.71722398899762,32.940208534279925,32.01515263389303,30.2402113709719,29.9636064342542,29.68410729914247,19.939742982989756,21.805578295577163\n113021,10.961243677401628,10.805087985497916,10.785175402191003,10.78522275455155,10.640411287832123,5.701888002762587,4.134562562522972,9.889651700792998,9.924792543944035,10.047584961402608,9.692566225028369,9.35266090921219,5.861925849699612,5.7670911370947495,13.389510469352627,12.774151230806062,12.430872628172269,12.013906297338412,12.324352935944741,10.409340479935421,11.233368288698262\n113029,112.00260447834074,108.08769187480995,102.39439151724115,101.81960057844087,103.61669381794317,77.47063389104808,81.53674437797741,165.9010830936814,143.77453854685248,139.6052202886988,127.2151803622521,132.1942530967468,116.84445895660876,129.455794652301,263.9890109442675,269.39313123410716,238.23488795756393,220.54972276685683,237.79690186524135,202.36686407501367,199.83661565138289\n113032,28.210047652260613,28.173964099688714,27.568265566650542,27.67431502232564,28.173310089873894,15.579703168489246,13.211709360617144,29.06713130110339,26.464966427750753,26.082979776806585,23.667812405545302,22.703795720422,13.621096589756705,12.00405615418478,50.38881855308082,48.84432088924402,39.61039321639308,35.10820529273931,39.776192798812865,30.17653122359728,24.39460466304746\n113063,7.258566512060432,7.118601586495564,7.0336098146655175,6.953710213046668,6.814399850080426,3.7573603622189924,2.9363805689974214,7.009078996622274,6.358730012771285,6.397606235698309,5.969472558988688,5.749441488511143,3.4766164073632524,2.2185298596863623,12.248109364929531,11.288831323543041,9.664819571071865,9.081756368498507,9.824354051024617,7.059446119519336,5.099225035687605\n113073,264.19266374438473,252.37495532518707,251.52953112119158,254.43358465702616,246.7144117131502,220.1072426766916,201.6463348998429,248.25195103803944,236.28080706609524,239.66540327845226,226.05001905448412,223.04197037527038,220.2706297091645,166.24963326785934,374.1766753743384,339.1088114189039,323.13548519975575,307.80985785548677,319.7818173099463,317.56656890927354,229.1924831021038\n113078,70.91648773308589,70.69268152236123,69.88065824094396,69.99761667606462,69.22446332498349,45.75588648142015,44.083805395115604,68.73947764485717,69.14586989347418,71.2159443389791,69.62330141081631,69.96489373992488,46.17863380944787,38.216311658494746,92.70411361891043,90.62075684985642,87.63257306419578,85.52818032329463,88.55887573506938,64.755029093731,59.756365003374405\n113092,1.7176294704349329,1.6483097230144945,1.603557520409153,1.606339295416388,1.5730714478764924,1.4080038870646503,1.3214167165547903,4.287909178995528,3.7185290809814187,3.605657804265755,3.294760849772779,3.735085530583501,4.139272465672601,2.8198131074602646,8.125054642521267,7.489827313996344,7.217334967797963,6.861876387473725,7.454805939816199,6.890299525574257,5.792311031844013\n113103,4.630935050600564,4.389600426517989,4.191344295241632,4.15206213040656,4.02435714333149,2.782004714998393,1.6930266063161516,5.408130482361342,4.69465416505923,4.7870146109515,4.3577405872861705,4.339937938024723,3.83627066008963,2.2640812039477924,12.078909291749351,10.724706299782643,9.415015581681864,8.72014191498741,9.580024082992546,8.446308359701508,4.93474745761999\n113104,16.41010630950152,15.51214598426901,15.168174101215964,15.097559851248077,14.692557870917062,9.636718067986331,7.793637261661682,28.21593598404599,24.447069825012303,25.32325672731077,23.029036278035026,24.865027062460303,23.31940521933273,19.32078664768565,50.65051429470486,46.70553059915421,44.96948198039353,41.504600367948015,45.82246203548412,38.72397756915445,36.20215945425381\n113120,9.807980768795955,9.630944053246603,9.634077777675646,9.647163290696998,9.481279135460285,5.618559283207326,4.7733056040663016,12.146536193306986,11.764044687732179,11.928393435616686,11.087078529227494,11.455032206284681,11.43933488993851,11.714852121046752,20.33378476628021,19.7891422381458,19.066768149203387,17.895556666831386,19.264093266521467,19.80320675091283,21.429263768866196\n113123,15.552519785412604,14.968014039587656,14.828070304033096,14.771043609477303,14.808998703034852,13.782448011968897,5.841153937603099,40.16660725596623,34.96601002272394,34.462639379412416,31.291529035385103,37.550834365833346,47.03160983261607,30.326786335951873,71.27374562609296,67.4212172472218,67.02526323376337,64.41603615131453,80.08164201403974,67.85273464104039,74.41128329010957\n113133,46.33585840283979,43.65236669849881,45.10752950582775,44.08754806707927,41.22769951288695,37.53235054386555,36.66420891890473,52.81487532149291,48.364062229630754,50.885355514930374,48.171681663931516,46.19879413986651,53.93738734233463,24.116997523919895,106.63412796062113,96.57225317506006,91.62447301466055,86.36068367557004,94.69920452079978,103.64166473491198,48.886151316332544\n113139,14.00707602599285,14.15687002088757,13.386047877248826,13.408692878460533,13.120210847307828,11.610614716167163,9.447200932900653,30.15884226707786,24.123684664478123,25.954876542828263,20.867300016634097,22.511866541970452,25.224163653476225,19.487271311746337,57.496341513742244,55.36261259727781,50.76631015082777,48.0972659961921,53.183505968085704,48.66168009557672,42.5436393641207\n113150,13.755992288420877,13.03843150549948,11.98562328841499,12.030139892755098,11.806418097217202,6.239372356036987,3.2139535422009016,27.064522535568223,21.550908574438843,22.503468579782286,18.5401974357095,20.24111615069246,18.67942331501908,15.790035106802693,54.8511073209606,53.8772240685059,47.65974097181047,44.55562832776653,49.115507550611326,44.4644147856581,34.708471945398976\n113152,20.395982369759235,19.710946800336067,19.548022267687276,19.530997322061168,19.290909629868388,8.283102195808487,5.6409467153575354,18.33247610611614,17.453100854207143,17.57631196403977,17.31557314826981,16.844350636854106,8.165661556610125,5.833273990472705,22.58482472156826,19.822321479040852,18.998211677181743,18.632259686897978,18.72028525904265,13.612522530085398,9.946874828252328\n113159,16.75702528193234,16.924754582927974,16.26525498152863,16.19323123873243,16.110733114187752,10.734668619775489,11.059032883304926,22.183687777958404,20.017269744325382,22.026929985420093,18.210311241799317,18.06361506290399,18.257269915554716,18.634262950387022,38.24380414557859,38.59872926452498,35.314833111440954,33.252806486201955,37.88777550695315,35.573616968942524,33.41196795732322\n113180,10.449248187527008,9.917462577636483,9.75723620266055,9.642943592718458,9.527636590434149,8.92002708318564,7.990957713345605,23.907459999123787,21.92412588738981,22.23744505130133,20.47971958982328,23.50048197412252,26.049269921649962,23.75337534994918,48.56026701036598,47.646736361155945,45.69459273470599,43.45413317033388,47.54560123829249,46.04971612459219,46.59560456685505\n113204,5.909936422534129,5.963225328019374,5.903233937906765,5.918420889224733,5.951052482241971,3.8017862046133795,2.1216804540367766,6.934523985650355,6.611523734831088,6.503703951792233,5.922515008268898,5.750586158626046,4.875579863408939,5.534083825857259,11.008019956286738,11.131491631596093,9.389227185458315,8.46856762589055,9.570910486339761,10.643295378080104,11.596364806477018\n113229,25.38922766877991,24.842279505401265,23.418363636646443,23.230777286232595,22.520431365538972,16.924804488672102,5.37389362729251,33.28276997761243,28.409127649194005,28.14871203787069,24.508594876892566,26.01455476905291,26.545583819907105,7.114936248890269,65.7990237869032,59.974434389646696,51.603263969901775,49.541940105394985,53.9914561833422,54.26230185847322,17.532488351959294\n113235,153.16014553065867,151.96889457529053,148.89880535997224,148.26859484244417,143.92282537080325,119.50648236347949,88.48280042705336,141.41882969658297,134.8154383539255,140.4040637537903,131.81511264767462,125.03199452173362,125.61326695998251,63.7814049195562,227.95086810727358,210.81168626536473,196.55472649279935,187.03577407321302,202.23535383324523,209.01799061154503,118.523487669295\n113248,2.6523326947680825,2.6556207543685555,2.5716068836028843,2.5336853280076244,2.5238771678290335,2.0224525132856215,1.8952345642495487,3.7582767914070665,3.2545741633101195,3.126074949136035,2.736696516539664,2.8976275647622893,2.7393681510461,2.1585376826280984,6.795045411189705,6.206374141054488,5.5478631715371876,5.289892443343758,5.918540020885485,5.493420073370268,4.098107722244389\n113278,4.481838820179526,4.291807215623806,4.127335666741557,4.073944372929286,3.9803153555386594,2.4365216827468723,2.085963901509059,6.950161754481907,5.932247762927485,5.7927872418610695,5.331242663698223,5.330750202441989,5.056618897284697,3.1619952642465683,13.986230058173668,12.30497105249671,11.17463453746929,10.498277476147832,11.093021604887495,9.867790865017614,7.217773806112923\n113300,5.299307035158579,5.153710017478253,5.069491505650932,5.061367439348153,5.092761041670856,2.9968273322759393,2.287732614113484,4.9501995030439545,4.729016553159306,4.721009410347229,4.565675415249202,4.369958692196527,2.8474672591069004,2.581612387419881,7.434739397857215,7.724263428378828,6.89585392455919,6.187560800461104,6.1746649599801335,5.50777650810811,4.915386667673559\n113302,58.42655474402038,58.52829207892738,56.333269028042054,55.99431634132579,55.735093850208905,48.9825643688576,31.863167643439812,74.12309720268662,66.24998857634995,67.55396693306916,61.7443432595266,61.26136744312893,75.4046229049117,36.34056613611591,130.09225497018585,128.8517260319657,119.71166897982478,111.43058105795402,116.92270273259062,135.27816077471894,72.51424379416481\n113325,31.222775846965003,30.8468255736038,29.48779505781307,29.348238389972995,28.890354649054544,14.453315841429163,8.403980294766123,30.533004652237178,27.543688923330993,28.25021181632787,25.70086080943183,25.126406531004463,15.55262981012859,10.385366857542767,52.11902155626951,48.688349211963,41.82085732823854,39.08004876083015,40.43306614359667,32.8177392960258,22.307965460153916\n113328,479.6309577732155,425.698887593668,425.8438031187087,423.9733016901305,430.6007368219775,291.5472647472191,288.1588922875313,764.88922030497,688.7821254119782,685.4562379823072,611.6196965414659,641.2372563759848,548.5494028141034,555.6721146751985,1318.570012967351,1246.1613026365562,1204.5982874704619,1109.298000132001,1182.6810386736136,814.9456345367688,834.7756400988186\n113340,22.853009292071658,22.796593605915525,22.754579479337806,22.78398224640554,22.48817497524797,12.870341898015457,11.26148063522221,23.61616171175112,21.672975571850024,21.819853171141084,20.39174144662302,19.959223742638013,12.091465487579686,8.310625952758762,37.49059507919032,31.108707372730347,29.61365780827065,28.447473578442906,29.921535923179977,20.543477171418907,16.029710554441596\n113364,112.9726917681066,111.00190491544319,109.29103532303698,110.35502002265136,107.34409649695571,77.1763798931247,60.86973329981861,138.22393958553417,128.19432594300926,130.84026697215535,124.32295996527655,126.46188106469087,119.51415144595154,67.94217889784942,243.1567343857179,224.88650402511104,217.608927244482,204.2079751044263,209.75586871933285,218.96374002390425,124.59760842804678\n113392,5.282049971097948,4.997405350577667,4.961592688788062,4.9816809817074486,4.860582806785639,2.793094043716261,1.3064312307983659,10.147812453391774,8.563957948665532,8.74333774731854,7.761992883315703,8.271168577111371,8.223773511670855,5.094317656410962,19.273577312538514,17.012582991552936,16.680949386709884,15.332161008338488,17.29274912641182,14.833179466823383,12.859435902068833\n113404,117.01585449853363,117.65395028778465,115.0328478213011,115.73163663186963,115.29315511582075,100.30206080280051,91.73808069326599,90.4288232686763,92.17015395442832,92.8105839719152,91.48147513455727,88.2827052501601,75.83161429415848,67.41458131131715,95.83663903326571,97.01598145677977,92.27878375600197,89.73503280226981,87.40786626655539,88.98390799017467,63.220794985141445\n113417,89.20068420574935,89.3087192367982,86.77331244581482,86.26562258556754,88.19847598876687,73.55254946036784,73.31730860384059,76.02295414698825,73.60422144907466,74.1613307058893,68.69477184411052,67.19187530054094,59.042077743353914,66.34859315069873,105.18627110173693,110.42366923149673,100.81147084574225,93.99632221186866,99.87476348713851,94.05394920088004,94.20215971622147\n113419,14.749764187016225,14.059077660426519,14.07689002373305,14.14748392970769,13.986423258401711,10.533010499512498,8.770720936404471,29.726068441149778,24.851828599453206,24.14490182354226,22.462280465834244,25.087627914499084,25.950792810134153,16.904561175860643,54.34532112705563,48.22263029481406,47.22836210239079,44.292720065020994,47.54284426979151,43.214635838917765,35.64640184978745\n113423,143.76916969217862,139.16393970929798,139.20671962396463,139.56887212408736,144.06897701653074,71.21392436263794,67.20900071325507,176.37278124548334,156.88798059104062,156.2416023754872,143.70151817233796,142.5601731990092,78.98017542477741,85.23151171609187,267.2099986582876,262.30202586747606,234.7335417864073,220.81813389329096,232.91003324486906,145.3187322573827,143.9092340056818\n113430,22.534270222293017,21.134650524757827,20.799257828136817,20.72917703466685,20.024433748647592,11.171947709861794,7.851800991345623,26.043795480394092,23.053488574110386,23.208625761839535,20.91723206065877,20.709133115709424,14.151000926472674,7.276478650830592,48.87218038285695,41.116592076623185,39.50535937047349,36.71742207260661,39.32006907142293,27.507709943623446,17.089297259270005\n113442,32.22307651092442,32.65316223529157,31.584974592865148,31.340890737505816,31.117293310095082,21.899667102622974,18.840300080692433,41.83743573785255,37.271022462459776,37.03916280420059,33.9799266235388,32.72859500085485,33.82810893598181,18.819781668668025,74.93997270765907,67.53869275211345,63.375535336531286,59.4690744396547,62.592980788801846,62.60268338706852,36.32385045931056\n113463,26.1373250417531,23.610733873062685,23.579790650476106,23.623219948659244,22.36877851297632,12.594951974405033,10.761233030631718,19.094007509643596,18.72013269866924,19.157558117502525,17.099409857945385,15.884351830314955,9.919908095239657,7.559591215692561,32.69606366662501,27.909710082366306,27.21177214318665,24.713553608005235,28.134210268560484,21.12309795152489,16.102565074315603\n113464,15.713989149499952,15.908117263580369,15.256591379271867,15.11350836553541,14.950273306075394,10.69589501802327,10.947915515016351,18.233719988248122,16.306754059906194,16.696740979968936,15.202420535202716,15.20660051775804,13.61577343039786,13.426971915056198,29.92454275166441,28.92903140827716,26.299466627949084,24.75250002553597,26.06124844435599,24.785398439964073,23.553784290012544\n113466,49.227712531568976,49.306328587917044,48.44279864786055,47.40855160464383,48.84537019313462,26.873335122736496,25.717003974409394,81.0919867302183,70.96743912113887,70.15919744494315,64.53912672387212,65.8007452603306,46.85452476481296,49.7578508858993,131.9448629044945,124.75474266224828,113.35580402492,106.21316644282916,112.26838447912488,70.98872778792133,73.99207298467456\n113480,4.422904405564899,3.9817372022541297,3.98677693800535,4.0100209278579815,3.8699384066576807,2.608415789066507,1.8854314946055895,10.037745842083465,8.199205124364617,8.004592398318865,7.419401979461613,8.783716791635799,7.864324822117069,5.182504010477692,18.89337374743591,16.591671116641308,16.223307776513614,14.99980028772431,16.466262519007035,12.98673349602433,11.093656956719053\n113497,30.584813909072526,29.465284203689496,28.84639155157501,28.69223690101461,27.727350667671498,19.202783427346887,15.59418701663261,38.22233759128586,34.45743820395705,35.83368906488633,31.23985559934522,32.40352647969521,27.179672868075034,14.339770087838527,82.56981806530541,73.72168797637917,68.58830115214793,64.40169750115909,73.30095936146569,55.65231788881961,42.39415272137181\n113503,4.30053107043877,4.198969172395282,4.119988371927959,4.158151927661137,4.096752657880063,2.2995643064485485,0.8653502401944918,14.66107258846732,11.894304757344814,12.228446165895297,11.204843049943943,12.408604817422098,13.55732367781536,9.458301396555457,30.17761565345426,28.033749035861682,25.45700117141496,24.140396509355458,27.228203346006676,24.932374282918616,22.286218826539567\n113507,252.24189533057165,253.1649555053093,238.05886662909376,240.68668226944735,248.3551766354018,153.21821075022507,153.4071233486832,435.63089301760476,385.65537805899027,398.11177471990567,371.3920938326432,369.723393952886,294.6458507368162,313.13463990301545,660.6832745972392,657.0310291757431,621.3858601266725,589.6070122357405,605.2474156047158,430.1849552870881,432.71464194725576\n113521,3.9417159159309305,3.7778603872457492,3.729959317877317,3.6975938738221075,3.7048289981980047,2.091613624734787,1.4135989620563014,4.843504436176615,4.0909235572761515,3.8293532122116973,3.5906404948282082,3.459704449795627,2.2450287798856716,1.3628779743046266,7.1066763874900865,6.221113979404364,5.459858539897072,4.907058611790922,5.027183539215374,4.1997718135633875,2.246427205147776\n113524,8.836581961709404,8.559735668421006,8.350267420750034,8.381049957742276,8.189321882909141,4.398197523093697,3.359747175059258,15.227684352697088,13.560485655013954,13.737467214334613,12.566489597869717,13.925767657225455,14.142103542827192,11.13908223521326,29.899059464103917,28.660022335476107,26.38615305007396,24.86663880115019,27.343235204638056,26.408048104917878,23.70991099635106\n113532,13.057319387294228,12.267252892905871,12.190700541096668,12.250733599489555,11.938901684317436,7.949572562615284,6.983907675099423,16.903223358178145,15.336481582609215,15.017096153003534,13.937112352029441,14.564313783990379,11.771788673434767,8.007951869489084,29.63618023796292,26.48692784707433,25.77299003063661,24.27938620782329,25.801630672779073,21.039692385023926,16.1682698171696\n113536,1.8634969731111586,1.7872862595466956,1.760723043662736,1.7620994491015036,1.7584664064016413,1.2796520849893114,0.7863343298968712,2.665316076117328,2.5024413267668013,2.402213550666233,2.2378847020431127,2.175742862731235,1.9850671793124517,2.184088655932052,4.122523435125957,4.238930004318951,3.9157278081331923,3.579805376903689,3.8341736843770478,3.798928599962711,3.947076493183165\n113556,124.71897012697929,130.03406371431961,115.79419626016276,115.37171525484511,116.37053728185768,60.04011740820998,54.51712765001024,237.17607010137127,198.70661303436282,204.84097703838748,185.5769051861447,196.26188220507078,159.97241829291968,166.7491288728659,435.50695113515843,429.9728127845527,399.8360439013374,370.7489496846338,397.8842236218121,281.0506404331396,286.71334144468096\n113571,16.37765337206669,15.74347619664647,15.262329235406682,15.188782608041485,14.834115629561282,9.644592764088397,8.069061754025347,22.866928914051737,19.616606078602487,19.50974484825751,17.90816720267949,18.304023083793695,15.628138386064027,12.147996874331099,41.1551788755972,36.77436034037034,33.334275836313324,31.53484211012372,34.38645353831491,29.262047527399336,24.239246471559365\n113581,203.33712871563264,191.23069654696545,188.13685502805674,187.96024897158057,184.8910644257976,118.24603562625414,71.9660776253192,162.6755569664385,161.25276593403294,161.4102873305588,152.9778870611451,147.65280318545254,103.55855462543256,69.12245707838453,192.1248485307843,178.32278551819078,167.6970423812829,161.61151627591397,163.9079050211219,137.78744358544196,93.48254801378411\n113593,7.696213436636243,7.754403598236449,7.503156869545285,7.4066537357007585,7.450936598971619,4.726482975274841,4.44292814483093,9.708571884700406,8.259098763063601,8.517101387623965,7.3791801687581415,7.576410495878717,7.208723760213879,4.238356878871119,16.77344731316104,15.710780655594338,13.951683146785497,12.877220614706612,14.61594207311563,14.42934680879077,8.255151721959303\n113595,20.912181670893176,20.018782596371235,19.719103179369554,19.73005666519711,19.288501502060463,13.41478047450689,12.824397911434973,30.584917356976447,27.60091146201322,28.99221463485487,25.560557872079123,27.21139672107861,27.13519198965294,21.194034973354604,54.082415239014814,49.404817627094616,48.87536617344595,45.16053733287176,51.40258293058952,42.40618831230538,40.628074495535444\n113601,19.444820998336237,18.908130606623228,18.67132225930376,18.558225714270925,18.335428474158846,6.775649582486482,7.2738368957821695,21.917913289198307,19.831370773417778,19.652019002724426,18.456860999710436,18.140458866388272,8.79983531365475,10.407794860203516,36.79111921267729,31.430088109150375,28.32944171972227,26.671351704020758,27.384991187506124,18.549021928830093,24.37455378457814\n113602,13.024434764286895,12.34293146181642,12.183247833544366,12.148725090223907,11.929435162598073,6.587416869127463,4.417348377009325,17.5388318074707,15.58608638464278,15.31978824056555,14.425381662188625,14.729785217787985,12.005145953974115,7.566946612585281,29.754483588478877,26.564047940962517,25.203447391933423,23.546675282736576,24.40202798063851,20.864483958433823,14.166671352757172\n113611,11.383426063265842,10.755155220685968,10.610562714828577,10.568514516192067,10.254905567075957,5.689939164237297,3.4247727081608654,12.349396466822357,11.841924462096719,12.046288230928393,10.773641886462153,10.600861481870703,8.248132581975039,7.425659737604059,24.31118414691594,22.201727029548515,20.844286011425556,19.222514177012798,20.999912404940734,19.169502518132997,17.883053722246167\n113613,3.745650573857072,3.6565181008360037,3.453640823304097,3.4243431355828484,3.364221270856611,2.966957676915493,2.6113408924136894,4.7143538456494785,4.199870022144795,4.38464063454152,3.983573993930032,4.051899634611064,4.318416095914748,4.219975310464779,7.963227265821714,8.157787103507866,7.421398508335956,7.081779332772823,7.543666666172323,7.524861422846202,7.602346915181382\n113632,11.473931336960039,11.038853131216026,10.686303079326354,10.699795272165607,10.472408096480674,9.182490582133083,7.147862249882445,25.667883010782926,22.93168238643366,23.020782809618737,21.121361754057315,23.527695940297598,27.010228472737925,20.39028494238272,51.38422122508073,49.08867142814966,46.51783517685162,44.00133878102444,47.92075969389119,46.56535401422692,40.919788297321354\n113636,11.275525178590984,10.886302864847208,10.593279196629595,10.556149617221143,10.63543758841012,8.921789637690189,7.681808144006072,14.131591982016513,12.567507073614944,12.122573160652822,11.263006737260978,11.113621129837226,10.210302754275622,10.231571155537173,23.73549500345091,23.40719353741093,20.95776541103205,18.9286929316238,20.348586411407403,19.327761830513776,17.85088535813676\n113646,877.4216864071805,829.0157035589749,797.7483505801038,809.7308713042178,792.4718918071112,434.6722247525847,424.2203017722012,1165.283990236058,1060.7119878398248,1072.8753756841966,967.5308893517364,997.8889377028044,736.2755036479969,750.1195629203177,1997.3315272131506,1906.510031615662,1828.1791704292432,1688.630469088598,1796.610457426075,1124.8525311542476,1159.0255759810432\n113652,285.3490471403728,283.24438433847615,267.8509237872304,268.0460586164489,275.75588045106093,223.2507731289659,209.006811763021,624.1216809826933,527.159465298336,545.5622607134632,487.28922734869155,522.6221339748091,418.6191562241537,425.03371654181245,1086.1348599231815,1046.5165153672917,981.6386194682731,925.8260777727467,997.395061598958,657.0938506981056,676.5612567318624\n113653,42.992515739401846,39.93337827616076,38.89049290345248,38.78120506926416,37.91425477065614,21.225942841499883,17.149380915954072,36.93558385226107,35.48101415956746,36.94187653266364,33.47620059120965,32.31749504677142,24.063699721708876,19.78315523594247,58.4075350736212,52.98331024292082,51.08659391217698,48.25078417434855,52.958982980533015,41.451490789687796,38.644324747538825\n113660,144.251759770875,141.24080995491587,138.7476652267448,136.864838001188,133.93537510108854,79.89115459443605,79.03610985780514,198.06801999765287,184.40379168692374,182.37002129821616,164.0105962494883,166.97571328704402,114.72860732714538,118.40342241918316,336.98485362897173,315.19151154373714,300.58457282623675,281.55123397301213,294.8768910673274,172.11149280466566,176.67280084296573\n113665,41.77006211530362,41.26825534345332,40.30369016346584,40.1344365924073,39.34205949838719,24.232476713290314,21.51163969040542,57.25832341931787,49.452223744454464,51.00100070235033,46.29037126147757,46.753866764074985,43.612338878655365,26.56278141213588,102.37887934487705,89.01413336223784,81.85659184968391,78.66450132708466,88.60028094619395,74.4854288277259,53.41843780587337\n113675,4.708235766152401,4.424965334634585,4.367694298845585,4.3980859457050565,4.311303276649105,2.3233839557160056,2.5569348883771505,8.941683898235421,8.365546707182116,8.755174919787331,7.821997587274042,8.587588553144785,9.38483839871588,8.776369510089966,18.614833210286243,18.030679913987644,17.78206004723993,16.354462843424507,18.575654363608784,17.353177103417007,18.14663341845102\n113678,10.768761317837937,10.447173190879203,9.805092633066355,9.646000507676806,9.461839176607326,3.3727094768527155,3.4892422251256154,14.097382245297647,11.727991666525377,11.423559594372527,10.242472599131519,10.185262363661433,3.9080127376386575,4.629801168905991,26.494451665283368,22.870230728219006,19.953877516214607,18.497183869736002,19.171453346271658,6.443401958631477,8.942710343430058\n113687,10.928278934412528,10.444561222826758,10.128571248818066,10.253595693880056,9.998614385215667,8.283949277334479,6.060054163763467,28.697771931206884,23.655046464574028,25.08847949620388,21.92654968427884,23.659159639263734,29.12771232975185,20.176033277189113,63.483016271498606,58.20470593568998,52.95522500059947,51.06490158140475,59.777275069651935,53.44069168357905,51.1215899143055\n113711,10.948641773319574,10.524014800949919,10.25789405106014,10.110323025538083,10.045557889227972,6.256676507522224,3.742356853421278,13.014296751562618,11.817026976441937,11.894465528709402,10.701047848099096,10.715043748599259,8.878615549778322,7.81392636956279,21.05296746188249,20.316727087586134,18.342191695802693,16.81065449863537,18.20250032253183,17.711933504326517,15.960058465827833\n113718,133.01106532365432,125.06757913512551,122.58916308369564,121.68612613045151,120.2189995083032,69.0209090195329,68.12191648142158,179.97628568026272,165.642666358365,164.08919849563478,147.78510314448928,151.05081941093738,107.09574171587786,111.6388038798969,297.2222717185471,270.3801380542955,265.21474530414844,250.28800325783058,261.87619772588425,176.40862846019473,182.32595369876955\n113728,5.438615163668908,5.294157084777938,5.26226596594317,5.246965445492934,5.165848085287629,2.1781146384561505,1.0829606476713227,7.543667404215345,7.025359819012632,7.035604999466524,6.585260215747386,6.700999574904397,6.1615550869557865,6.503053625556339,13.128519993103613,12.746781501198546,12.025594343037705,11.431271761111764,12.04669316264984,12.295731886123184,13.3458725408651\n113738,14.172997194115835,13.030749925852911,13.065326097646215,13.070045771044295,12.665090633720988,7.185227290512706,5.0503146174043145,20.592583904024586,17.110803890748873,17.104640506274322,15.042071907895492,15.381955103436395,13.279591559874541,5.715337812641639,37.82605545872281,31.676574570555303,30.781682366202222,28.310475396819044,31.880492893788677,26.056777849479552,17.317838032167828\n113774,1020.2883692269588,1015.2753570018367,1009.088152333653,1010.2015010400538,1013.0564854441899,844.0734759361105,825.6093505547717,1024.3674802151704,1004.2158862673057,996.2055138834415,970.268666064821,975.0704935880959,901.4067040326944,819.9168825227476,1217.4148123721195,1181.810762966614,1155.3489175133407,1132.5490821897897,1162.0685905646624,1058.83042812887,994.8439252155855\n113776,7.2625362841932155,6.912762273312007,6.871042553801278,6.863126462855919,6.66926075965288,5.309234929763256,4.484423626505272,11.00048704801446,10.098952498343344,10.125081890026522,9.23774251561337,9.857356363171018,11.105644809219706,8.088876296799361,20.587352625753145,19.181005849761764,19.134505371565783,17.614110885664328,19.478196515058432,18.99765387212697,15.95125732743908\n113806,2.513538040474058,2.3784643513665134,2.3405559012332517,2.3545291118227816,2.293028039923422,1.8202925659585294,1.7785732238939422,3.189710521440578,3.0817584042442965,3.2091638929212305,2.7995926673598,2.9263145032639692,3.1298846897855266,2.9947370704084078,5.891053137993197,5.687981478790462,5.584706182620244,5.141932239428934,5.799006412979796,5.34498989520454,5.637233953717002\n113815,14.405289954921471,13.717832857543423,13.325169877938205,13.213286034014311,12.969954358536183,9.385261799313623,7.621197555724215,16.298071524689178,15.014333715452949,14.753354930706152,13.686189780653317,13.953954184782592,11.712293183290885,10.342105324340677,27.829344828332964,25.11294255465288,23.469152577357054,22.186557948222426,23.851159071699318,21.208664293252973,19.178530879396664\n113822,4.228020490236264,4.161729750420056,4.020107419711861,3.9346917573361875,3.8928535361582317,2.121773697989643,1.4939659022946556,6.216599081215259,4.854700506402333,4.901370029157009,4.299472199018751,4.298855674503447,2.916146913878075,1.656639627409096,12.250965020810222,10.340680342371972,8.6623885244972,8.044820450114692,8.458753816807459,6.383769477696517,3.918678535795717\n113858,215.94439382190674,204.65578112443632,199.74053578940666,199.84916602467536,195.10548862684954,108.95948092166817,110.23645104959493,294.4006132530446,272.6457051372297,276.7401910382796,250.17022623970516,253.7269943125418,194.40742470773122,201.34458778315732,505.7517460691895,485.2324095510131,464.47312712360923,426.6361056058331,447.9414975256142,284.95605403704116,296.880495576256\n113883,9.74108524250374,9.312291843284783,8.994277236446452,8.94863785669625,8.938158155451028,6.0863954697989735,5.497033459697949,12.907595887437436,11.207569385823085,11.78394798628226,10.196255847878852,10.20897613057644,9.283353916316601,8.278665539957247,21.256536846237683,20.53781150096012,18.537842538624698,17.259880133646245,19.430016194878505,16.90054965670316,14.086483085575601\n113887,2.9390819812285978,2.8309246375242525,2.7827419903490376,2.7878186381798113,2.7405282094935535,1.9636106910420932,1.8484803685338296,3.2327250593665524,2.9570200676480933,2.9609679953139048,2.693840487066125,2.6607840786418224,2.3134591693327953,1.6726948168511746,5.251286160845873,4.612764439753283,4.389656023712913,4.138159646187425,4.369888964681221,3.947271180211358,2.7275439556282293\n113888,37.88356030790744,36.834685234207925,35.76159423242943,35.41213400146847,35.17124376673449,29.84485570465525,22.288426330682547,49.71384030591071,43.7605384101706,46.677927627663514,40.803161387683986,40.50955500117751,42.11700596955474,33.85053626576128,87.6616877093393,85.47623928805878,78.89933180634557,73.62219196590847,82.70122484327504,76.15773596228078,63.974527729447125\n113909,5.0909841173617245,4.8783169155820065,4.824146210815807,4.810396175151132,4.712114544200689,2.497059197318137,1.6574148341016244,6.66523332599695,5.815716151189107,5.64840537000761,5.220148868478053,5.181164348248655,4.259448570203018,2.6643350168893565,11.82208476126103,10.136912599149284,9.601581968318566,9.082138835923619,9.395702617795672,8.601125838315959,6.155093035092416\n113931,3.2761454883702483,3.2707513422216943,3.186915441595918,3.1405528929419497,3.1441331194954323,2.7061070482789775,2.6097780797505825,3.9294951898929167,3.5414897704963106,3.547150867198724,3.256894915804491,3.309549719491995,3.2066798833756276,3.1314806829158757,6.034021971018065,5.920898335848013,5.377080835512549,5.049907994820274,5.49373436404502,5.209124036752394,4.809435370426019\n113942,17.09374150922708,16.668190330620224,16.122867838333576,16.02445840710649,15.809027450434108,11.553409301187438,10.392059122059255,19.947199940624213,18.10115194307733,18.089327722342087,16.67814546469827,16.794277486318993,15.342672296186612,13.294182959597533,33.073151252519054,30.647974047975556,27.953616691956608,26.670798014566557,28.072860728272996,26.645365104251812,22.235310611122035\n113955,103.72787283692578,99.91637113263229,97.93090903371281,96.73358113030562,97.12828832653905,56.71160967278729,54.769189090407934,190.0506521091766,166.10126566929284,163.90240133462564,153.0368109993251,160.88553212157476,125.22023561673072,122.72289118243624,357.30660333524617,310.91576102285273,289.690717521986,277.10919352341165,291.70298743358154,210.32347308426998,219.64630867188316\n113960,4.252094214043451,4.044714645413947,3.9056369967068783,3.9600911282811553,3.983374033445634,2.1970295831217874,1.0700832718065356,5.978925964624885,4.640049522237045,4.4905890587469734,4.071393027245883,4.105432213543109,4.011777183425285,1.5825059324960848,10.494796120954247,9.875034024264183,8.967515128578265,7.8989318720622945,8.192667169991324,8.990087838960607,3.3943229891301656\n113965,23.53387217721042,22.006520245661193,20.8989268263396,20.94458086109276,20.4703443550243,12.39116972183776,8.439012642108358,36.33685590463578,31.044410316956018,31.750389775196552,27.698211536004074,30.264937747925575,29.28729342497287,16.812037890463014,72.30474441284976,64.25398688351164,59.7628628048352,56.70660854517908,64.56956253267217,52.161074294532376,42.15866651187022\n113967,25.688652900449085,24.631702869088556,23.685610376051695,23.602183277657968,22.98171622891422,13.5647580693949,10.32916911724914,44.03301524458387,37.49009073395481,39.05976585605913,35.19462376850416,36.81897527970752,35.55252638926225,27.818383969370753,88.69230756077029,80.96735191742715,74.25973713090615,71.08412180580477,80.85670078934788,66.31989721982141,64.89373682864965\n113969,17.289757604668683,16.16751157344226,16.1600282148779,16.06150799364963,15.635633321689795,7.499304902897523,4.922620292920608,14.524193156472823,14.311812694344772,14.616601625590533,13.958429523252777,13.184812382779821,8.380601491471548,6.648316080298098,23.63263651267918,21.661466848692054,20.368950859492102,19.394338087519365,20.880222984868333,18.505873153323318,18.464969823159397\n113970,13.465204536805356,12.829018060087241,12.470383728658817,12.325050562379795,12.030380521012248,3.4899066357007222,3.4267794121686825,33.3501656551949,29.370732388155893,29.529232708307546,26.504438075565673,29.68871699850063,31.297900990575677,30.62504186156237,72.70873239412614,69.57845284617751,64.26672543341348,60.369361354455954,66.88660977357343,64.96749170744651,67.97130408193766\n113973,30.035939475134594,29.478885349841864,28.608610524822534,28.249802540280022,27.707865864751806,21.49910240039331,18.92086131020544,54.549605683042095,47.2574711731787,48.1877907659935,44.31992169607286,46.55531547124128,49.35077939164498,39.20831996267436,106.33110867298377,98.95330803140563,91.05021582399993,86.39219927804797,95.64006078357502,86.78044166305446,77.56361900043116\n114003,414.338509105864,422.2003574272528,403.00580022061104,398.5452065040803,385.56016052221344,153.94333650566747,148.10567765590156,533.9893573421964,494.24436033034493,503.5985998360702,461.8558600297964,457.06420357075143,331.199541294696,328.78200702587685,973.852393137496,897.6669839410304,814.4396473495873,769.7453710861195,808.976731790645,590.9825290874153,624.1060910591526\n114007,183.12437942106655,182.07544877830173,169.40998570971524,172.1961184004954,176.91050139212825,149.3178360794857,139.87986852468373,319.7518025451044,274.9718623707245,281.01515605290194,258.5681349188313,271.98335779071516,214.38640637674743,210.62305879378783,497.8509588532598,478.3736400540976,461.00039439719836,433.63797324869915,462.38914554682475,323.7389911007617,325.07573343901254\n114033,1.756865979918911,1.6945425385210657,1.6640643437691238,1.6693369167074477,1.6437679553252191,0.7046060421311269,0.7564666446614997,2.4075207255350852,2.3268455227563756,2.3565055436237548,2.1692182376151465,2.2114416574403535,2.111294258129109,2.109787819508119,4.177076169487641,4.070740587709714,3.9765820976966912,3.7430228560760233,4.0084175646953994,3.7088227786067507,3.829670209836171\n114068,126.76761448698784,123.57228784854802,117.56234446385825,117.13403707232969,115.57994583221932,104.7587931953959,49.71496039533731,113.16116491616741,107.46158737565004,110.67191525131645,103.86579419923758,101.89700543774474,94.48015841228896,68.40285063302868,156.0821727098749,159.3428901370362,145.2669224609894,137.79871743106443,142.85804321951622,140.15004018626277,122.98537978440183\n114073,797.0969265349789,776.8342597899127,753.6654815950258,754.3270133492654,734.7175536484386,486.40755764486534,503.2462425738754,1298.6924216522568,1176.0908252774439,1210.3263469523906,1135.932087733077,1158.4970482191948,890.8094461215354,881.5136156030237,2485.4776817218876,2248.4296723879897,2060.0874929286365,2004.5454174692975,2190.9530095817195,1604.0276992954427,1634.2952145997183\n114082,107.51989956070791,97.77407265906781,98.05259954545608,97.05406816794411,93.30766834945241,62.93517021615561,44.310143982499454,78.27396029158051,77.63222037068678,81.2816815578674,79.97318819252892,73.9540130325004,48.75529719884887,32.78520648963073,109.99025650893189,100.95882774301414,95.86422624277652,89.97637790258433,94.54677615118469,78.11920783192474,55.57085440028286\n114083,6.868940809644593,6.59260811831133,6.34138345701318,6.230927778753487,6.236906925018246,4.589888549499573,3.7207118792973226,11.792229390425495,9.422396141488766,9.611108087306455,8.0835599950374,9.24057829216661,8.936612252346452,7.6697021369632115,20.913664553478675,19.9282928846217,17.526074406613766,16.82852895598335,19.273364850748152,17.604500674097167,14.915959470643989\n114085,33.52270505653289,32.688719927354526,31.224443835904584,30.98375099210779,30.30567489811078,23.26073234169833,19.421175791241104,38.94591023202741,34.06198654780318,34.61450240617275,31.255376700693343,31.25102782309304,27.15669665305026,25.63270942492998,65.37136377052751,62.27879970993297,55.69792918668275,52.27722000568337,55.33327361990257,48.58201552871147,45.70493005181295\n114087,51.85411128173288,49.737131586224095,48.96574318658604,48.83813843013667,47.98892476202913,32.11497969826345,28.02292275004631,59.03489248287988,53.344070848337275,53.66497227039385,50.55076924463653,50.40406112361142,45.229275918091254,33.875444572995136,102.87033476444317,91.58013151281583,86.29396456397359,80.04251352803453,84.91314225512771,84.21455842213605,59.68790516604651\n114092,260.91798777088195,258.82528850312474,251.39771787791557,253.37393686962298,260.2963286908727,172.84833897121987,153.96994676350772,364.8739843256418,314.43213904443115,310.3020413424557,282.0408822284365,280.77282631043914,198.9634624383137,234.4057549699113,704.8726115232496,735.6651368746038,623.8407094950404,563.6427937779905,635.8754151781915,518.5156426071954,509.90113062750436\n114113,8.713018644453557,8.516711063428072,8.431766653817782,8.442802758384255,8.285145580524421,4.962146103967037,4.2772321107311795,9.895361319139768,9.303474450269198,9.459773623684681,8.716358866437284,8.794011272172122,7.727651292509541,6.598275008076761,15.613317806819744,14.555311890832039,13.889867623146566,12.936552123373461,13.845950765262597,13.217603333836225,11.459934480222202\n114120,141.32866525330633,132.7217601574645,128.20950362698568,127.17610232914802,130.56018561247444,80.92475906173543,84.4785646678518,249.19676165304188,218.68557449551045,212.80641607043145,195.76744447787362,207.54976083705492,162.91335312151554,169.3213696753596,445.10290264998633,415.12159768204305,390.4632738783532,366.06299002730316,388.8978803455025,244.68171230936238,251.39179873208423\n114130,86.2648618771348,79.13945589152253,74.7842319529732,74.11322915202325,73.67883509337582,53.84748404630414,60.1030512023517,207.02685297652394,181.48305531870648,185.37586052003337,170.70488665990973,190.64127041572593,167.22074775367489,167.69350379621318,377.946226881156,357.924951864378,350.9554007600216,314.8021343848912,345.339247011209,257.4271663986087,263.8876577288239\n114131,11.50821662407357,11.236089452888802,9.994451218314477,10.086465863687994,10.509759562573144,10.729619397248133,7.842734801405761,21.04609932868099,19.273096765598623,17.97068501116469,16.249951195286158,16.466956536980522,19.071308082496135,15.815987553119236,43.778530793415065,41.062652160145845,43.353607621264075,40.06660246180525,45.331981559244575,41.7689606286005,35.65964335982784\n114134,9.567499612770822,9.305463687342103,9.175850402535481,9.203933491399823,8.949524572889665,7.279145105003049,7.647856866545341,9.520936369884536,9.544899176716012,9.817340716367589,8.8149834922384,8.746950503373373,8.688074843866293,8.649162721742465,16.16091276086938,15.588678154816979,15.42017688708768,14.118381338901248,15.679092586080472,14.843884536248014,15.096014126577476\n114138,6.5123081189118075,6.1322618863307135,6.084390721710788,6.095395585121021,5.8600143732761145,3.6248237149610962,2.9749773084324356,8.759225617873543,8.078125296281186,8.045050940440026,7.043396999760195,7.308251057179156,5.693817560263971,3.736416747779497,17.21666502837322,15.269564788285031,15.065346545136412,13.672717373140264,15.331469551288235,11.8383187328827,8.563033352960423\n114149,29.14553706528136,27.45300975695176,26.84561145395322,26.938524238336818,26.042805854752928,18.445884293156073,11.60656635174773,44.691238054470375,42.506370421288935,45.01076438140029,39.71781918321263,42.51427079630345,46.007866917154075,40.89761940093839,95.4498064423358,92.0616139370493,90.52333402487545,82.9872746673391,94.8663171371786,86.9334227247242,91.37565663323717\n114158,2.8439751158284836,2.773182137958715,2.674460380596542,2.6373073428486995,2.5990781620596772,1.845618960137035,1.377955548560199,6.607457372595909,5.058458732503466,5.325527068043431,4.517862021333797,5.411733812393065,5.5161574378307,3.566308150128049,12.236744555379994,11.964747647143126,10.679288431141416,10.381256620372014,11.37344547320606,9.758405796461894,7.459906086253847\n114159,7.1216304982881535,6.776740212823654,6.8038609815005975,6.773730226284124,6.44454262640005,5.736623799861648,4.833743239533833,9.370109227306191,8.492910787107386,8.623642477353279,7.936462725242206,7.98464519796987,8.302488736786193,5.988870767446889,18.91474675438894,17.765732228831272,16.784246678665614,15.547139375861686,17.811104161785043,16.5367014433499,14.364648298349232\n114190,19.414180669277336,19.153150818795424,19.1502942008286,19.220354543061774,18.830091940138093,10.164472356014992,9.76672667810835,18.674676053790954,17.688288669569257,17.91625959652011,16.962310909997694,16.40621438898212,9.10693436277123,7.975587631804375,27.749848236116545,24.276479678925597,23.297662565784176,22.271206973868715,23.360493511865254,16.388420376486113,15.255004631013568\n114205,6.476270003546331,6.05258479235226,6.0815183656628475,6.10007359373961,5.93358953640581,3.4844537508769386,2.6125949521331115,12.582303124383001,10.82629490506545,11.088352181950508,9.789488523476349,10.726586285261025,10.366208021620107,6.974505188721451,25.28956443393345,23.08119709758936,22.302410923095923,20.522094836788433,23.421513395522894,19.68290036413315,17.33839297281535\n114216,5.996546984250003,5.90472073116264,5.8685134199893625,5.858091850354965,5.753413789054956,3.8489582157149838,3.757037390331689,7.851046832498152,7.42284135707795,7.630384300056878,6.952717331925983,7.275835020939123,7.692627541062206,7.22827139253326,13.326205181125674,12.841518390159685,12.232620975258644,11.663416713409239,12.838417092084095,12.927338168130364,13.91267428824954\n114218,28.652775819217283,27.34376232573626,26.887971149462256,26.686236322425593,26.018988833155813,25.427777269899416,21.619398682404867,42.67519624280311,38.990907196497254,39.14812731304948,35.78612013387325,37.64655464671057,45.46351014616561,29.328427888516252,78.38725201242411,73.78485266686326,71.40697366201118,66.7873037354707,71.45578474986159,74.81166081791103,53.920434096057996\n114221,19.507448770239954,18.031376734630392,17.85935120341998,18.08647093595399,18.32638459507862,16.806654396944925,17.466237046155484,46.2125666217846,42.13831119724749,41.046018307908135,39.779525152073454,45.20615734414949,42.04932527574761,41.82917043936523,73.27685401040429,71.13066997801708,71.79713106346279,68.2908931813779,70.76065776664744,57.6562629342692,57.733874705421606\n114246,23.444226319826257,22.507871271294828,22.209803426895824,22.115022016077216,21.75739763551775,13.47905733771081,9.639536602969848,24.254844562342534,22.785500647837186,22.96794640785922,21.364650631169045,21.094021332144667,17.463558397947278,13.212820944886046,38.24770178506577,34.748666545394705,33.11424206194699,31.631858341442186,33.68814044943384,29.875826227908878,26.53126797476487\n114256,108.47114060882056,103.85266668294398,102.63133500313455,102.2936270048361,100.32088611028522,70.44673716721117,42.00191703703572,121.15253473731771,113.00144668085115,117.7556498407653,111.46528769020613,112.66919870691491,102.80116285609311,48.67940939720338,187.96892222973221,173.50143608742786,168.85587427736306,160.651254610705,172.5946993881726,161.74833746502543,97.07001963249888\n114270,38.91887686627602,37.36207387994659,36.569617122453366,37.134233655905156,38.057633017574645,28.093149210204203,26.941360417746,58.5369308910083,52.28111874718484,50.57045662980948,48.482654916714345,52.201270685645596,46.2456061853498,34.4586249590116,90.0260304728553,82.37631321868588,80.7857117026794,77.58023784886282,81.79062121630074,69.51810388555482,57.74854751658827\n114277,230.31646223514875,223.59931205432187,212.90987038882093,211.2885263712461,219.67394326959547,150.13515030871727,145.29881538147913,377.1711788763291,332.2171629858208,323.1674569196403,309.29942408438245,322.2295072637409,246.5279668076363,258.73457813411505,717.613849593331,669.3349952275794,616.9902528311424,591.393811104884,620.8700270075105,378.7985470250886,386.26590771830524\n114283,8.17043656750864,7.913196266551075,7.637029214385061,7.672799755964049,7.487524164156114,5.242330194088587,3.8225317124493152,18.422143056444455,17.001708678784585,17.154062113956204,15.757912471570862,17.474910594132602,19.12959058158823,18.0669696988582,38.88416568491682,38.492286353145595,36.2380961059108,34.344687166763016,36.86512175969024,36.52940083690578,36.57002290872868\n114286,8.804513228244632,8.223992850753913,8.072414956369618,8.077004019926266,7.814448528625967,5.416563243545319,2.8161938720845927,13.495117025489218,11.8835570498645,12.140165217016223,10.714769236149985,11.185514855475548,11.13347056700835,6.721431872509354,26.948957049527685,24.2864425230279,23.300457123384216,21.36840178498998,23.767364409713643,21.124492643529514,16.27606686427355\n114288,903.899568813937,881.7788817434525,862.9253891094033,871.2288315349135,872.6553873540354,570.0876814736545,309.08410329727354,877.6138352043929,841.1817848653552,847.1684163301793,786.9939835271674,762.5619040729848,622.7164509356847,305.86571600729656,1486.3571277503397,1389.1397442203802,1315.6265828011697,1209.6108946521013,1264.584700302903,1150.1104313114097,698.4061538654843\n114301,39.42041842914519,36.55161741866345,36.52872451632794,36.437402838509215,35.44841547676138,20.95253066912074,16.26547120872157,42.36094169985432,39.55192165774184,40.74501316105011,36.49054239317349,36.160551420236764,29.552519869448766,22.26591479075096,78.81916437398621,69.86103245733398,68.50351230143295,63.22053390693092,71.62601679057916,57.57094623920637,56.22202205604879\n114306,39.258117514886294,37.07733570563992,36.641499481940514,36.40309379153609,35.94047928079258,19.923033042371802,13.475637591215808,35.497542429244696,35.00263610561363,35.454320040324546,34.63917768177338,34.03379111404568,23.907340730088055,23.111688110452917,44.56110176596378,43.29108043831464,41.39236665399259,40.05491697473257,41.00131788236655,37.00401056783678,39.86482264690231\n114310,14.70854579616976,13.82071553650226,13.593386287583321,13.58515906995978,13.181981426075351,9.775217423481209,8.761761741454752,18.02582102082151,16.401310674204087,16.915195385395116,14.768470288620858,15.056060803747641,13.378533870069576,10.573534198397024,34.127282102161104,30.40856475809783,29.373695686326077,26.952380156561393,30.120673692969415,23.7079460773983,21.02886362623859\n114333,131.1587044953773,132.05069635352842,129.75069156159861,128.80019403909978,128.2311107872524,108.01103368623288,109.67291376963004,103.21751379741853,104.24163974758211,104.72945046285366,103.7665230160057,100.53985516304469,87.16780247077709,85.99834586933731,109.45837788887738,111.76572060303084,104.97610674969474,101.52669089413818,102.73801758267726,96.01772835001047,94.52549716249003\n114341,20.44845051558139,19.404911410821327,18.846563891395036,18.71712768865112,18.154647394795433,13.914943847997582,8.343165671217328,24.737575751128688,21.881764871878413,22.84486518321074,20.22027662106153,19.79126245215477,19.361456391496514,11.495949506939164,50.15220598891962,43.96893679658835,40.233849245018774,38.30772398260404,43.27790936781007,39.36168919810594,29.661933170836825\n114353,308.21853317638374,285.8867110927766,278.273165987582,280.4146194619084,276.89185943379573,184.08533694388478,184.81250025358835,491.6561498513908,446.21409657794,445.6543792460583,415.83730942872074,433.71702031315976,352.8186290245607,357.58009155934104,812.2254271743925,783.7770953748106,759.0138035183387,700.9237178539995,740.2269976811139,509.816809580262,520.1490243095078\n114367,6.565576924161935,6.377718157316013,6.211911271458816,6.211331491379023,6.25547142732279,4.5199369866391885,4.1897624475453075,8.032135311258513,6.83419947290807,6.572075915764883,5.983846309408393,5.6741936096560766,4.384026737676144,4.517326781879438,13.029311678263957,12.24731398445908,10.76786325033129,9.723460544005396,10.66396226133348,8.666636592818918,7.5439969224137\n114389,2.570554130257455,2.549191954268124,2.4584766126370003,2.459092134004039,2.3931016471721427,2.1041029064868084,2.1347980241668103,4.799171448439599,3.898746718938392,4.152871079651672,3.7154078212252384,3.762547808684356,3.7430740799568865,2.7586722695909462,8.99775702076135,8.705776400856712,8.109310096031571,7.56248951430369,8.309631878011567,7.726572629906685,5.5835100384567875\n114390,16.372626773512536,16.330154396134756,15.315473671130146,15.260837546564304,14.732671282757687,13.006878609665241,11.437313457862139,22.518780553070425,19.180487233606343,20.34998649083516,17.73795228058268,18.485828427977555,19.15886971234775,17.74749519543437,40.348012518226525,40.205845087784446,35.87698976080612,34.37896087348363,36.96429150438827,34.77589499292835,33.3228321778699\n114398,12.642547190856957,11.991847782983827,12.09292692318154,12.162562565058913,11.769949117841655,8.2330050476305,7.627746976746107,17.319576935951012,15.641069210299882,16.031131160369696,13.949122909303874,14.273229783498058,13.93531083841923,8.111419681477248,34.635218064843755,29.94257770676543,29.606551070629177,27.132712145131144,30.902974932312993,26.956570866076675,20.087895564559698\n114415,25.911271134816054,25.232920857915694,24.97787470133685,25.03695964826262,24.999282301689398,10.399976544805568,8.372307879137933,23.058854546326703,22.264963219413826,22.514877749936854,22.08997263504789,21.528010610393466,8.683249726003313,6.82208085054702,26.069213884350532,24.860146848227906,23.523040366950713,22.339238729976053,23.520206833801286,12.922538917992807,10.499907538795982\n114420,1469.024214098636,1404.4721759854926,1401.6622439683156,1398.3919920816254,1381.3137967499088,1161.2414864945133,1097.7406984858742,1290.3284462886031,1281.5389037906443,1267.2516830858292,1188.8482940820152,1178.6417259835036,1086.5508801264791,822.64407669481,1853.456643999567,1758.3201632163252,1745.6422286211216,1653.0908111123922,1779.135230852925,1544.4862608632427,1215.0033918874353\n114425,91.63127185380127,89.7214966805508,83.97606100078332,86.88502269825517,89.32943397370694,67.17670493833735,65.76775701923899,196.28993919325893,172.13134266281463,169.99917787280026,159.8640693692853,175.9911460211561,146.0790498466586,148.4038025797633,323.5333906106363,320.37755153713766,304.19470420431577,290.1425688192256,306.8694915046032,210.21091732155773,209.42816905838217\n114434,38.431667884567915,36.628620215145645,36.51136246442297,36.87865807697313,39.88278980699515,47.51613629869632,26.182247906272035,78.6454285168482,71.63650129361297,71.2671363228534,69.83338329284824,75.81469427973506,89.74322252841684,64.59234839171593,116.04833951176383,115.70942911290858,114.65273511078344,109.27987296235182,118.4203157371557,119.6100618965962,101.65962333474384\n114444,2.2226531795236566,2.0499522555461898,1.9713699946707006,1.9544033157887946,1.9870012858923956,1.303927822074864,0.9375079036557823,1.9707090691820734,1.6903600016127176,1.6342435338475543,1.47624339528619,1.423174395879823,1.0874212000183494,1.0781097771907902,3.5880412198587366,3.536353461284951,2.7985477745520915,2.488956354285815,2.9281493079313825,2.354420160639451,2.0652902200824017\n114449,209.52463409725686,205.11698391823452,202.09357835412155,202.20187384489896,198.57255463117437,170.33114544503775,134.8113144490567,319.2292731290171,300.39195474561814,313.502172474088,281.5427700596632,298.4043331762855,320.45573572956795,265.5133172164585,574.2822409048946,557.1197760429366,558.6403927981308,519.8015058705973,577.8810469551313,527.7814012295951,482.4202124811367\n114466,14.896748180986066,13.885079486285258,13.686091421664408,13.720062444631802,13.418145630136863,9.59025035612195,4.316544823076734,16.190718768768093,15.3819470069063,15.731991935535673,14.195857872562382,14.067367175257527,14.80053389696473,7.102126884253751,30.541307090164697,27.598337584403588,27.160920600715148,25.20440600856485,27.821550276988287,27.611230711978916,19.375677473459355\n114467,12.55732083860312,12.49893376275027,12.033209695125548,11.926000608350163,11.609987946401832,7.70991447902231,5.9459471150524585,20.516267394046817,18.781556863252167,19.366123541438764,17.56939333164727,17.973319218547864,18.855370297013064,17.37141211791894,43.46785070184039,42.312553182699816,39.457075709068945,37.24027434703017,40.894939196882945,38.958502063281095,39.147557926257925\n114468,5.0552832586311505,4.83024594142011,4.891244366339033,4.920148927270299,4.780851956945874,3.6166387482613325,3.3914779538680424,12.577681270037145,11.481851523913349,11.509723337662821,11.086379668204055,12.857027523325968,13.881817921339733,13.681883419655108,22.171359137152212,21.77666914445303,21.645017291977986,20.19322336478782,21.74582617290977,21.275609657968165,22.000464408280788\n114469,13.46443032180772,12.759527224960571,12.269384109983921,12.076814379715051,11.885555144784927,6.804518792138224,4.130318009018764,15.443491723903774,13.081669831687448,14.020900075111523,11.654259147791295,11.024164833240686,8.967500911569957,6.721897415348894,32.31786719745739,30.765821868210743,26.330456643429283,23.790221608000785,26.861747060559708,23.019585837212265,16.270595320066494\n114473,21.523462459532343,20.853377853937726,20.552902354841297,20.483562831730367,20.013751424025006,11.860884051058058,10.124382423843437,20.69896155208217,19.205341267123444,19.659980108377432,18.89397241632488,18.16064929198686,11.302472673185362,9.21577239874648,32.247340729432786,29.909947758029645,26.970677871364753,25.755445277909615,26.578108363518677,22.456556805162027,19.5956833720377\n114475,18.51634468655793,17.853341127974375,17.506478905921764,17.411425721025253,17.70880100006854,11.566559398753665,8.58350641360832,21.152740142246657,19.60398989870694,19.29742176578297,17.567198126860646,17.33269926362761,14.32575426615454,15.156583642590267,36.18430756681793,37.03862188081425,31.257604515221228,28.991445447017785,33.708326405827,30.45765416864885,30.004784414750773\n114476,42.60267405496688,40.14911478133311,39.86995387151407,39.83117007014791,39.0080455973455,23.30667950126867,15.648760756400016,51.040302929310485,46.89354178248985,47.793632980272115,43.61889104189767,44.08516324048056,40.21184761586894,25.130729999054065,93.32134267564915,83.57386923031548,78.9017315526395,73.26280118377514,80.31270725861083,74.57138178408759,60.58602420087041\n114493,462.3847663541151,432.1528921094405,424.7002819854396,417.26920958831823,425.6417093187073,330.07710846741935,341.2662216604557,958.3776793624996,868.4003757921507,864.4898959342439,835.6490393741993,871.9426370524369,733.0844469178471,749.1841853457018,1599.9053595228238,1559.3332731227001,1521.5346867678766,1408.5754759616143,1465.2412347891927,1035.3853544416852,1061.2278147716054\n114494,34.1949793704829,31.320060315152794,30.585651591751738,30.600627325065137,29.5070443373664,19.460004612536096,13.939682882108368,35.309492461070874,31.891484851056674,33.595624444576636,29.392247636413117,29.64967203069238,27.167708689023776,15.666806603852919,63.86655254643336,56.27780640849271,54.37663970990001,50.48164161673256,56.91784685990664,47.576637855500636,32.77691908433019\n114499,70.86067416820156,67.46475057322849,63.78355778983816,63.38697567498613,62.67944776878634,39.834373638282486,40.074260683099936,124.01868218616575,100.02820435082833,99.12762534328873,90.3115268740746,95.86023635978829,83.65887080573484,84.80850696471755,222.34184214709708,199.58698995376668,182.18416840006014,175.28713337504007,188.43660750334854,144.13518191751737,143.37681374074177\n114507,4.991157754206483,5.0045391413736,4.871068276284392,4.843514928541773,4.833841283535186,4.097504311383835,3.697131664245071,6.817370655503424,5.925009729492739,6.239213778481111,5.514271312086918,5.354732907649911,6.493907939475381,3.5152655641983066,11.938758624512582,11.440722645088494,10.32341443434962,9.435100068350971,10.311913188847859,11.887538265717927,5.438913031158875\n114517,5.09548891901609,4.880130724838714,4.778111455648622,4.820042259465684,4.720758972545514,2.4722072332647196,1.9530816078653948,10.311825933866347,9.215336348032922,9.430947927538282,8.051873330245709,9.136594968686003,10.326732434858467,7.960186955409237,22.55069734443829,21.063104558646568,19.757572431867747,18.56042550085286,21.567884992417124,19.83966809513886,20.020902165310385\n114523,17.41445025558936,16.152649147995742,16.040360162811528,16.002727982228667,15.611945249200138,10.107826721142862,7.0002538032907005,22.328520130493594,20.48441205782904,21.29916467606716,18.949982722880023,19.859464902992546,19.44088071333804,14.727516678656585,44.68206292311414,41.70616197817899,38.94847317195334,36.95932389627432,41.596798564327266,39.21204890558839,38.82083890207688\n114533,34.50152373919398,33.871258736695765,33.13130410569912,32.85851363531696,32.242293191577,21.716807097320835,21.361678634321752,38.03544684787386,35.116590846343506,35.83353288594139,33.334050147070066,33.357304616176506,31.46031525122954,20.061683696366504,63.63937313370508,58.02376476081226,55.64584857225008,52.06344208816614,54.96094657019434,53.4137984195267,32.88764659629238\n114544,4.718105490450963,4.540813708584414,4.489328386565774,4.510651553243774,4.399970860585804,3.1073171652139817,2.7289646568719608,8.206204460134161,6.589146432192016,6.58555103505031,5.998291538979128,6.351275143416817,5.350303401905242,2.4732077467117737,13.72291005582876,11.304376217317756,10.79088647025078,10.012639119426265,10.954968862466393,8.271196239190859,5.07473286244595\n114554,4.3288466782436075,4.027101722926341,3.942638305783455,3.954534279570755,3.8407215337511134,2.7992626902157762,1.1912488292852208,10.37404102382941,8.772304822919208,8.77804449979729,7.818290185103216,9.021394588639474,9.693273049450593,6.286128524817377,21.164950090380266,19.66766945511205,18.396601251239623,17.39925875016645,19.40460133626888,17.67725100981775,15.38389521912815\n114564,16.931258537421332,16.54722517586109,16.354941369340793,16.342723577013462,15.871511662497996,10.060913896727705,8.676163252854003,18.78357998680693,17.889789205686178,18.36394385550804,16.602175989880372,16.78036088553446,15.883298251171167,11.80922466968727,34.47255005578689,32.491701956365304,30.729417573965993,28.942751103890142,31.982273688332306,30.75553154586695,26.213176071383607\n114565,39.66293366308141,38.45592570365181,37.683056697429485,37.46762966638152,36.85150169543305,26.61832300510817,24.24803029233204,41.157102116500525,39.18325390999912,40.38207661692673,37.45585667396282,37.57878129578613,33.90818558770277,31.27280780058964,63.173600084711886,59.75233170994341,57.886980259078925,54.09736343586857,58.1564963926813,52.66530314962652,50.97675039773052\n114567,28.571760768504692,27.1413401926836,26.71931194451641,26.619741561561817,25.958089514757223,23.299067832035867,19.458601675597738,45.2754436411421,41.2540123661387,42.428422596256176,39.165287833495945,41.53811249124355,46.366454439428345,37.646279221185324,83.11964628203407,79.74075220826688,77.18769486045632,71.47061881390532,77.84226386592495,76.31254517120995,67.43682637573461\n114603,61.878940992975906,59.264595806213386,58.5758375201563,58.43500509595988,57.333762783539974,33.90969919603018,28.68108981416523,56.25586536473871,54.10136852810567,55.314260002787236,51.69985500121296,50.25944786652521,35.48585802661312,29.48164002418327,80.09154139581683,72.31681532397918,69.88977390921194,65.87226321458121,70.39258567633381,55.296482899974386,47.567958089235866\n114604,103.56886942902837,104.02707699067734,97.30052175966905,98.4133355728968,97.38982770673354,65.40311007202757,64.27560899109082,144.37222428710618,129.65179576909554,128.39028855644506,121.21581385984219,121.88486244490828,97.22370066720246,99.9698829792604,226.39977006584274,221.38764704747402,209.7215532814642,195.94032965272322,205.034651729672,139.93047878431054,140.06494426920216\n114606,18.132083262893435,17.252479475270302,17.150705957059582,17.14773261995834,17.875270337170534,15.732449621632131,12.889856513530296,38.174380694740016,34.369712191936706,34.75445412215075,33.208598535216936,36.16283798935101,37.23573821362309,26.128604171728615,58.29426294315739,57.690422252542014,58.19015768701317,55.174491090258606,61.17506818761648,53.504261969908406,44.237564960532495\n114612,175.03134283000463,176.84120947502916,172.1040497242666,170.25408592288622,169.2454302927812,114.79418780645429,93.4525853202617,132.46751442067963,131.9742643645162,135.06829442895344,130.76604885078527,122.14144363751383,82.69176700499324,57.92317808927364,194.75198002686298,185.26020306890175,164.5100431033761,159.08107398496176,157.03828855417513,144.9108120282596,82.30546895695588\n114623,26.102586744906652,25.564312932515723,24.6559183159092,24.333974571640788,23.772845031945902,14.384878382462842,9.044285244658639,34.00115372861405,29.80017869969743,31.891511184645896,28.81385166370325,29.180741970083456,25.342965719504942,22.99812081469685,65.24222907406099,65.87244091306583,61.64847138599957,58.00069440038179,62.789722291426806,58.620452613544636,55.40689932244762\n114629,22.317358729382295,21.988396920634916,21.216740653308563,21.083166255986807,20.455360310758945,14.659208355423134,13.086510249332049,37.311611959940805,32.74555353411545,32.39371966720409,29.705935150542906,30.35931023993955,27.051941367921863,20.506425325207047,76.45217271127433,71.34749126670054,65.60704934105729,61.67676614941316,67.0658744230144,56.655075705457946,47.26407168904391\n114630,31.233843982834706,30.04047326552368,29.508325830942866,29.428071390569187,29.135274851753458,15.874549966302757,12.027220277814603,38.97959789878264,36.90303760543453,37.64169905347392,35.68824319979273,37.210788918349294,34.71357586864282,33.726179510970454,64.48446855699225,62.23152535081363,58.799977255567896,56.00787239438943,61.877686523289924,64.01773554670393,72.7906636350108\n114635,37.20691696522208,34.98185845687022,34.82137572469182,34.72428691219302,33.9309195827996,22.04304790110752,16.50360620216824,31.146061658782415,30.81234705616096,31.53142647305053,29.796266137799556,28.582476550120028,19.784747985947053,15.59595987873101,43.29931179994287,39.594374921032575,38.09944652769988,36.75746387915493,38.96733731203222,30.70842759251307,27.143278663893867\n114639,22.659310036874178,22.266706201301858,21.86168288170894,21.83282818928561,22.128149555146337,12.504513454263524,12.877672916736948,25.734368685164533,23.94664432480079,23.72846475309782,21.653956189272975,21.134374129400232,14.883208491948537,17.899216973122854,42.35119324742617,42.31280631322132,35.096455211619066,32.568396223853085,37.946651169203605,31.646696544326545,33.266424185261464\n114641,41.73843078986011,38.83088074335938,38.918157286274514,39.08965728936003,37.8199829114641,31.920492788967945,26.856284528488672,54.11978209027133,51.97746341865833,52.00958407191615,48.63028684833279,53.18367545502914,57.15166424257952,53.97603242584648,95.22506125714563,93.09401978080564,93.14208570509045,87.14071401358908,94.44234598551522,91.71009941542783,94.8515378261569\n114660,23.938174558235,23.61992008209674,23.22158220443921,23.235780908549568,23.365518908313636,11.971022061405213,7.222332686117123,25.077201538874615,23.55675028996881,23.40648032519939,22.353238016042663,21.988407232722743,14.266965368832338,14.19904597195175,37.712925037913806,38.66301620218913,36.94412215558948,33.6447572355596,34.833141426266586,30.251407610720573,29.95136895120579\n114665,9.587802930758153,9.319047358932597,9.055159237264787,9.094987936142607,9.307498312451669,7.745930525249784,7.338062134871018,7.6672343817209025,6.756806968020446,6.726829952142409,5.965815585406009,5.615146715332526,4.1015140989968994,3.7247776026385924,12.57068239392388,13.24177357482181,11.009311898944388,9.642305439508483,10.548353912638042,8.066916189579388,5.03140074967876\n114680,2.7589910433572835,2.5277475060088217,2.518521732902352,2.523814834203819,2.4274827118581888,1.4203083009160613,0.7946136502392757,4.484751386774873,3.9784655609754953,3.988390425535024,3.604590411081492,3.8409330582686625,3.8684181721983837,2.9537601563449565,8.60056929129303,8.088044541265823,7.984239469026669,7.345784663025036,8.139426980065824,7.5228471736980715,6.757348143258193\n114681,29.051645956876726,28.330090880515254,27.80737170164607,27.65934626774057,27.28559667327684,15.79167456390355,16.3774468280065,38.51763700726711,36.481520396625925,37.74242770339644,35.05140758640742,35.886646409454855,36.972197058072446,37.01703710593223,66.58186877431461,65.01936253113763,62.911714427504094,58.67022858072254,63.528559820380586,61.85854028609101,64.19649914451693\n114717,5.475204412418355,5.10773571667061,5.078777704082041,5.0758991631218064,4.886004366412731,4.679152034320832,2.241994889524191,8.141738465105314,7.428633974674568,7.425521959487001,6.288592300494997,6.675640845315466,9.18120643421555,4.047914307403611,18.04582487367678,16.34057437081647,15.571058805899154,14.668808916969853,16.7014097082484,17.777712261690862,11.72826803892533\n114724,20.119415777271985,19.640351803694028,18.93965480518359,18.94737882778071,18.438614389281785,13.307000236722004,12.691449039216614,23.046523194525488,21.65707995171653,22.743510857541203,20.142914622434947,19.968389886778773,19.583679575378454,19.169081078862792,42.19439115363687,41.83602989650042,38.749213384875645,36.19679479739147,41.25199522926749,37.78013324294218,39.20557707784429\n114726,2.1051819496626343,1.9725647340230599,1.8962221755923911,1.8994934013515061,1.8384695504795108,1.0661041117525758,0.5570066067810705,4.720656213909069,3.8553708467191408,3.895679092325954,3.571515628532825,3.863710677436732,3.5892154172400548,2.610988421297394,9.277174970956146,8.397681417604119,7.78296316501763,7.407003832429549,8.273119846908333,6.760681526440938,6.244442507210576\n114730,18.06437815771817,17.28368599208839,16.696738469260914,16.48069362105669,16.270818725970056,11.286201063872873,7.539773945549463,20.54524083644445,17.83434593173615,17.889491309016485,15.688066927345997,16.42875459432881,14.827598692135245,8.473876964859832,36.48640094831408,34.86470022785144,29.728284776785866,29.051473706937784,30.40757390158532,28.79199530822445,16.426023953184238\n114739,14.536532170275942,14.490668813578884,14.125597569388821,14.204307990353833,14.32540708500783,10.942575548051305,10.884328064816929,14.796305639087686,13.959288902118926,13.618172405126886,12.62559502389831,12.334214071767889,11.219442639433334,12.740977682454375,22.601653559535514,23.52901196642803,21.099914740881005,19.044962418015377,20.27014898214943,21.026627515034633,20.87691901031419\n114745,4.903665177079103,4.685125401834502,4.69068010919147,4.712839585344283,4.652297627674584,2.4075444112627014,2.2403792305832066,7.379892811011295,6.266428951310278,6.096240458922421,5.409575735356888,6.047664064254766,5.785889861358648,2.848062847986098,13.607801375920342,11.648848571178394,11.106752918983583,10.504020833556197,11.530276106090703,9.734390227232902,6.8590205446618295\n114747,54.99237808340453,53.008642407227896,51.79675730738669,51.652427244353724,49.30445940209945,32.31832413760956,33.42650926192982,80.96518794701824,72.97974374134841,73.86623733036197,66.71027421947525,68.60309699254373,62.94711488338352,63.584330423418685,138.69841588983945,126.9730258119842,120.70523456543442,114.568988946723,120.03802912639689,98.18673771203309,101.12038270875767\n114761,5.46000532384184,5.254997855181925,5.1377055777900615,5.137286200350629,4.940476919906837,3.429289880686986,2.5182269204535834,8.673011362918771,8.048879757978685,7.969890284900902,7.3586869869805005,7.577298973526368,7.615778967698689,7.1447669837429055,17.61934513829758,16.851163961383982,15.799986225874418,14.970150455097064,16.50324597859963,15.826176178789767,15.465725168167992\n114780,16.24476917785708,15.726599833863213,15.433547166705253,15.576343369797979,15.776619058121769,13.101479160528758,13.245705094493088,26.159366640124862,24.15815545286612,23.717549364269445,22.880325527944347,24.760306429775977,20.623443665975337,20.579269467402792,39.36329237983981,38.31786154822601,37.77956349599452,35.4780480063044,37.06884435079916,26.27165646177339,26.520563463139247\n114787,131.09560129617446,126.30903450748518,124.62898145549786,123.93969581526771,121.1361184858385,96.88097801782159,76.93040119595555,139.50579876467958,130.07979569304385,133.02376660081174,124.04173349471866,124.26095959417282,123.3509629638075,90.52810661470174,228.63241652933206,209.65271487999797,201.32575339438577,188.24182313202226,202.09990941127666,201.482222188017,155.5718221581902\n114809,163.6665126410283,159.57280231087003,149.65241293484561,148.84765550483905,155.6031628537099,102.61145808522136,96.66409666312971,285.16410970944736,254.56029434413847,251.16582144846845,226.13262374031257,240.843762623963,187.3965183194342,195.0059772510358,478.2205437866512,472.861233266692,430.0999864902671,410.40901354929866,457.91052181443905,302.17492228313614,302.4685230941453\n114810,16.609464007519474,15.697973256243069,15.360760074842393,15.386154361891515,14.92876925668237,11.840794783173086,5.218576404278971,16.43678853550342,15.213856860806375,15.427636816483167,14.0484958701255,13.924430092565512,13.417026243373881,5.01998033980696,27.49688527941214,23.579997792741658,22.4845264348705,21.352009498990025,22.33363211173235,22.120272850949622,8.571363201867666\n114821,7.035289442432087,6.8053432639024,6.746467541210602,6.74766791227687,6.63870922450166,2.562165478667683,1.8048508340869487,10.773051536940239,9.440719890768655,9.596975985654014,8.651976978023834,9.08903266875013,8.867502770464744,8.314676869863982,18.5849685365293,17.373806181746904,16.112059240088836,15.127361287604424,16.69277697441015,17.380627937831214,18.02530977580979\n114824,36.51041918751042,36.05373119210113,34.03135226047592,33.97717184966526,34.9276258948196,26.33149867028694,24.662473595846514,60.35066712877093,51.593194326635974,54.15546187853133,46.50577995143093,48.94638717754544,41.25739565427514,31.982216510685756,100.34494713989507,96.11033915206943,91.05052084391781,84.71795369796781,94.02903177697844,77.40484841163288,60.89477570601607\n114832,9.936905572142992,9.223556287130265,9.054981060922191,9.039555426160083,8.666332975936932,7.2304823905257605,2.3483373394253286,16.124171007036107,14.999063487799026,15.448845666670488,13.947093931124959,14.686939158471645,15.851030589014874,14.318938226147122,33.813594113092094,33.01778288288874,31.595956824067347,29.12478085005582,32.075793625574285,30.669960324566855,30.722953509020737\n114842,15.481669272756092,14.757979864911881,14.653855103126512,14.684646134543021,14.382361623373106,11.933574749049463,10.24889328145208,23.954119952692867,21.293129311301183,21.036509524827576,19.634957404616696,21.374645189953615,23.34610250046841,13.842189910419544,40.30487475435714,36.80220119628785,36.505610718194454,33.948511066999025,36.33705273901906,35.82153431734891,24.03901686215368\n114848,6.50331524153336,6.388696421301298,6.262709228491684,6.25644813126225,6.217237748887571,2.702753721595006,1.4905814944650175,7.3538365882614745,6.798886613898868,6.770475464645245,6.172829952039645,5.878718000671732,3.5429180923726484,2.343955819355133,12.23948398074197,11.641327515716538,10.741673664708452,9.693231508006878,10.071762093681567,8.312301524810298,5.9833246707325705\n114859,10.633854736253223,10.113939721596154,9.887238310537755,9.849380770433385,9.697050943522472,7.826657519017046,4.612021955695619,14.307109791958236,12.806148182592194,13.256389508881593,12.18115245723786,12.388213157224284,12.670566567321213,12.168439796751018,26.56355882429514,26.328750087262904,23.669952480373045,22.552068255471767,25.019438888037953,23.491034786024347,25.613290894182562\n114875,47.26425960571474,43.91099370033614,43.80967650409142,43.70150281734215,42.28949988903375,32.497795255506695,16.097659722282398,49.704382195584095,46.63981254706856,45.70747731393176,44.186683372094485,44.402653842303735,42.603759541087015,20.997967897300864,86.31164789444136,77.46308488282685,72.8245005304442,70.18707425162319,77.89030097544962,72.48202544909893,53.52423469696245\n114883,15.851190184536634,15.683786232712166,15.270589324530166,15.22925019593939,15.139449175701476,7.791346995639991,5.3464734458093615,17.949361318579733,15.902848611363273,16.35400729315487,14.366813128769106,14.392868808895109,11.339025128363405,7.2076890372455855,31.418672820408446,28.951746031883015,25.268383056064668,23.591484841301828,25.47358347012052,24.744880217074552,15.52716770705725\n114887,782.841098540006,749.5181400902912,745.5481849514706,741.5519701244701,715.768311667943,468.12529152432114,337.7439469074415,507.9264898290062,522.2818016386356,534.2816206468715,507.74225012592774,473.19777706072756,290.3376259317592,201.15959813935893,538.3017523617931,519.7482052485715,504.3551717308376,493.62011111478745,505.2640612488705,362.2425280057396,238.12918747805176\n114926,15.031933109308161,15.097801013330791,15.037639830099954,15.028840139309574,14.887023599744953,5.350987004064247,4.228179482142206,17.18961274958103,15.859387515390395,15.744311980708334,15.07442242314975,14.952627160945903,9.59273714315512,9.217098498749804,24.72202835280211,22.17322986185835,21.355625641867796,20.51754695255174,21.632370514451694,18.885665602358845,20.436311178952753\n114927,4.408391344058885,4.160922389757902,4.104488594462221,4.084931963589109,3.9920888923158877,2.498040035738945,1.9071121408294063,7.2285611744967,6.113464064461042,6.022064155967276,5.440304927070114,5.705536396754151,5.620905915024955,3.5649645355610926,13.38851456430183,11.648447606457058,11.004937786460863,10.312003447313275,11.259301934628768,10.08038039863507,8.01513368036909\n114947,727.6093766840928,710.7932554364459,673.3704992304846,664.4643742830227,648.4763090181187,316.0548267462325,339.3936326736519,1243.8176284011647,1136.1387853439894,1212.5299092190446,1164.5294298973306,1245.3113955384974,414.34621563738,434.135735412396,2318.259504680046,2184.190643287961,2136.4158214415797,2006.3068384184803,2134.153455569381,826.1183268792566,866.5535138690807\n114967,86.52662112913528,86.09807551644077,84.51260513631995,82.36942114229568,80.09465263425875,45.811629387017646,46.55171136517447,126.49350247675075,118.37299688238038,125.7884496142734,122.45431364656628,127.08768856221256,50.037188145444546,51.698706567716485,214.74786482745677,200.02931377599634,194.75293782084174,184.24016348051805,194.7651198962622,93.50263974284901,94.18623596199959\n114988,74.71578328239188,73.474268815041,71.9208396902756,71.95439033536114,72.8218791631195,69.47867507488566,50.537622574993485,70.93778102471593,68.7335020043511,68.3431025403818,64.05203756310718,62.61837983405874,62.31582317306336,56.609408431401725,104.35712617452113,106.18100653578396,91.57215694823347,85.38951330182968,94.87316911783097,90.79640200736289,89.12475694861831\n114996,17.676858500861016,17.85508592418294,17.64447784858937,17.669803112379835,17.806507279336856,8.81806634942643,6.9721884313814035,17.570879096086816,16.072615886923693,15.863783424515445,15.067553155383884,14.590918478634737,7.517557043656656,5.132911403478966,24.27329950810581,22.352873861887034,19.64493091694399,18.18742899038172,19.37976414105435,13.60501016494773,8.27244016074545\n114998,5.41126471607638,5.115407517896594,5.0041352178669,4.923681912956663,4.861528855630099,2.4358208669020622,1.5580740921064677,7.487780943690977,5.848225968646212,5.91506510491629,5.162719081037881,5.4902490909917665,3.9643679106303424,2.51128488216325,12.890608666249626,11.121099157042153,9.609461188350974,9.420389280534016,9.965206717596587,7.3676327856544646,5.656308801717112\n115030,26.086681733465465,23.61075659893056,22.734745433539306,22.689824637374716,21.890891671462423,13.987023566546206,11.924301137838809,33.53385005231111,30.025631230007683,31.338914561289464,27.638647734547483,29.087360998351237,24.271359715512617,17.75419136788366,66.3120263061765,61.2781268265915,59.728310275706015,53.75258347886449,59.021807000295034,46.234107852719404,38.92733218759929\n115045,21.14913425998635,20.563335403374563,20.232553947359136,20.26459955261867,19.957754394452675,15.312201330948263,9.748135492996937,41.98744447008688,37.24790431506179,36.467441095516136,32.453597738659475,35.951638955945825,44.83116332217582,25.99081527391793,95.94998756079245,90.33079038523009,82.62845172987798,78.01437028873833,89.84146706956702,92.28061827784164,74.73152183584631\n115048,7.199997691487513,6.907969205707459,6.6533363857884025,6.651971311749378,6.562256491988792,3.1360259733205726,2.6257405163597842,7.7717956552135785,6.955254172350539,7.046529181408915,6.454699980121194,6.297843811563042,3.781008304508458,3.096519065556407,13.270734516485074,11.879876575945815,11.025633214299523,10.400743144768299,11.13731483544575,7.5222273142075995,6.808958975338958\n115055,2.031046637382221,1.896766978502241,1.8675520215552404,1.8738716044618124,1.8127097761540398,1.4917142847375902,1.2224189196305693,3.4781147586579126,3.3157974225359017,3.3004680716027646,3.048678372378577,3.1891909646487817,3.5450143251540385,3.5144469028831438,6.508960360700577,6.398223239178055,6.355199686473514,5.991764821200678,6.358714382840701,6.229088236416145,6.282814636479934\n115067,17.45854293420509,16.160851500111345,16.02456516608976,16.09906992099171,15.574153683837732,8.532630986622003,6.360145333105452,30.281493400383596,24.395221261403577,24.175635068502803,22.2969330345875,24.84828919280151,19.14325679714058,9.661225475810632,53.88921394395499,45.421220089500295,43.9851159061841,41.171526180716974,44.220281168410345,30.49342199155621,20.47952574994766\n115079,303.8482562262726,289.82210591806825,269.850653342609,271.0369855054539,257.86862447151987,216.35463790016172,227.6894930699968,221.3269531741762,206.7881453805244,225.10027699908662,201.19441553177222,200.90166844335442,182.24971475213223,158.61266069153285,321.23839371001674,299.00788201703904,290.909221806407,275.6270071940593,286.83612856262476,269.7026600656828,225.20172221963114\n115083,32.294622164360455,31.00812143834112,30.474530002141016,30.350414893611074,29.585294009752147,26.868314877150013,22.00311350625755,50.5778418686446,44.002785992474955,44.57881569134794,40.81190836390232,43.165724061324696,49.10847458573818,27.57892675708021,86.35911363149158,78.42709401352896,75.37645040905693,70.30782850437116,75.60958993155461,77.60886226972957,48.45109662177012\n115107,6.772350267943631,6.7046684031382355,6.693981101790059,6.697979766628157,6.582643373747906,3.116874531440544,2.955799357320389,8.768284037378303,7.634474929416167,7.6946052599484975,6.972433824033541,6.8863480610912005,4.708099984540769,3.7735008453473147,14.032146867749754,11.99465798222472,11.527603353887091,10.64350645599929,11.678406106449117,9.335637510495028,8.247488556489184\n115120,97.46067329139987,89.7573605847153,89.41692708426108,89.17151953175674,86.1881757054058,62.03902074919066,37.19806968043828,81.05742414924913,78.27068074709307,80.87954544980676,75.15529373226047,71.40929398840657,57.791032518760964,33.74794507328371,134.61296734378314,120.9038000884248,114.24329116931969,106.9407978455284,115.11998812920153,103.33287912368935,69.5110093531096\n115125,186.14819644341225,184.23929956076753,180.82770853154065,182.88685062114482,187.6274048028848,182.02243207903444,158.74348946652484,211.9611450571352,203.2686623141057,190.24440234273803,176.1138951874041,180.9968962491808,237.11206542697295,111.84493421620911,301.6238898666908,284.0280588661005,275.4512896918707,268.3969291745874,281.42072985112014,333.05776037748933,182.46216556021653\n115143,293.7161953208224,292.2496708403934,282.9352442198178,279.715005833877,270.82146357953746,129.04712052712523,133.35634001378955,381.18708828940066,351.0532195727955,363.6165862472549,325.0400073095215,331.69566349811845,230.62875513363974,232.3755047666206,655.6584416926997,596.8899522832112,568.29750403947,530.2313443553286,566.1860173288061,379.84918102207195,400.9274048189563\n115156,169.46926089905273,166.64234113827385,163.4944349101853,162.9130241732202,168.32229582525108,100.47082765751519,98.0151567491382,215.5832682731212,193.11788318528073,189.70861674276344,178.3806297975256,176.1046219516014,121.22279372790257,129.80893924401897,323.2790523065104,305.10965181467697,271.60061146465665,254.0719186993118,269.94858325932347,197.14141657535924,199.94546415361188\n115163,5.210486429328027,5.065043811956749,4.966166388096406,5.0056833640539455,4.874185109573984,3.9908699455807244,4.098872693470544,9.426220592503027,8.517867117954864,8.662326991738219,8.000511255074995,8.609548594296902,10.225325919064694,6.845772891584685,16.513642274099873,15.600888900022166,15.503944826803641,14.398515069850982,15.926698458710447,15.735733146623312,12.606731595172372\n115164,116.54843206377475,116.86130844442637,110.78605815778896,111.73261860487342,108.28871548230329,94.64879093354867,92.51598666067969,151.06986836023316,140.7954697644103,145.25890657131737,136.09526345510346,138.2215547013336,140.23350036161878,137.45071056235705,280.71458264026205,274.4910687877519,256.00095108534094,242.68270567393577,262.69622808179724,251.65438252431437,258.49850960274705\n115180,5.217879697969114,5.221267835990422,5.061169587454098,5.007671171786048,4.898156937176087,2.2831475218217516,2.0370733887116255,8.837928738724818,7.270647869009308,7.691063005757292,6.499540637488634,6.610518006530211,5.454480172084052,5.979110365134984,18.352424109436658,18.081357549363425,16.073394689400146,14.815498693511321,15.577117586437398,14.18531084414554,15.214970662438757\n115185,5.955017222397613,5.733151652598431,5.677729461111993,5.6564287262054425,5.536273286334669,3.3253309280709935,2.5165318495512357,7.570042748952333,7.072217354611073,7.224515526620366,6.368610729640143,6.53233030018872,6.382653511698909,4.04239342372344,15.283010045929782,14.085851521751287,13.114459054103833,12.082507822684343,13.325235709126154,13.11525546901512,9.294744710099824\n115188,6.644938227442861,6.384851046239669,6.210859080171243,6.201878179862076,6.275153536051992,3.729052126812861,2.3545582120761304,7.215452675811192,6.682920067196648,6.610157516470671,6.09831277101897,6.000778547289805,4.7125216884672225,4.889766545135667,10.962752633930956,11.535954570968338,10.70890261373452,9.753572781849194,10.322513334830361,9.821599049663536,9.271935006079783\n115221,36.89920188466526,36.7839299703892,36.406473776544,36.29835512773613,35.509092000159654,25.28270876654063,22.343506159470934,36.07137324447506,33.54198484645447,34.30303744681102,32.61251145843063,31.390100841995302,22.691709632371257,17.112434852030177,62.21542628783284,57.173931073356314,50.68948493889163,48.54010582921337,50.530217670132444,44.478507583227774,34.35003689181118\n115240,6.138358647754234,6.0614145750335275,5.952889615772734,5.9030238676806785,5.861382813384456,3.2232932468024975,2.88799493312516,6.503052295622734,5.805435788578479,5.937274724280057,5.563480584096662,5.405773822469909,3.7707945806755685,3.7837186560515534,8.55198460379585,8.150944204154978,7.39718646875232,7.056187225733673,7.108325668456023,6.348937179298855,6.491647299288252\n115265,1.503202090364913,1.4938458050925214,1.4553853130350438,1.4892335770127239,1.4561968787781767,1.157613089163284,0.6876973970478765,4.226585842496824,3.50751913999478,3.661449833794311,3.1762660346999523,3.2281513109245172,3.304789425223818,2.1137395177004628,8.652078682294933,8.083236582180614,7.347318572574692,6.923053841971432,7.653712628993553,7.363190197100993,4.731694244134719\n115309,13.226697742633005,12.121643802810995,12.096260398928044,12.073009058638343,11.5527810222697,8.267184286465328,5.1266724893709865,19.461297417937057,18.24957261140565,18.911756293107036,17.014226447327392,17.96758634398694,18.532399002431234,16.798659895111836,39.71616697591561,38.436492149696186,37.64256676172428,34.76008889862875,38.844632498578484,35.91845958899029,36.73757214291694\n115311,152.38657785797642,146.45659690277,143.97116957242142,144.49268701952613,146.28820451679758,134.5617434539766,134.08541874326636,204.21551706356718,195.6483777673043,187.26379526633772,180.61111214962037,189.8558380170488,194.28749531950183,200.71532382185714,289.2548965345107,282.74035046577217,277.14376475888173,272.06780816307827,278.1660383251321,271.9159392849115,280.79690530138146\n115335,1.6809100111468978,1.5899712604066798,1.5520677039642756,1.5601526248189195,1.5154052609559623,1.1076042191372342,0.6978369692220313,3.9697812133219688,3.6260754872586354,3.527814479750918,3.261918152300914,3.735418267476506,4.079468784091494,3.808836587651126,8.410305923448101,8.288274187082363,7.8073756266385566,7.360678267054428,8.220871866153134,8.097611973051274,8.383037901564169\n115336,31.037478996568606,29.380106273473267,28.998594777905787,28.901098522947212,28.145765247331013,22.212214587579805,14.656983984776605,37.232008476534176,34.33355481957175,34.7951395844906,32.463783689472066,32.79680574620871,34.77461409115557,12.960885667884535,68.77405011910508,62.625828674781715,58.841488554072036,55.26937148254705,59.040836979854284,64.09251594265521,28.15766635320162\n115343,168.14646820800917,163.45006232138613,159.51991079652436,156.27865635899528,155.07966077143223,72.44069680261065,71.05814354175897,257.26165238941525,237.5535887797045,234.79747706473714,220.27403043998595,223.91144151602984,145.19845012705113,151.63008036003998,463.06444254985564,427.65778365194024,399.9986588635718,381.45571389695544,396.6720939871682,251.77884979612534,264.047517576196\n115363,26.35417094776255,25.184348191569683,24.77072375238498,24.638823417485987,24.026264116300645,16.017446435876927,12.214856294588007,25.95838710135669,24.136105005317702,24.434025708130577,22.959662820657368,22.129227403884954,17.167715112661398,11.891210078807308,42.038644215238925,37.404431184128356,35.49031463650542,33.63897524049705,36.53774172175866,31.278952500084173,23.260824030738622\n115377,30.870348704030725,30.491508880086396,30.222645303419053,30.047761884304126,29.4403517626437,17.809965870873096,18.32534668543138,87.2085116752414,76.30213809391182,79.58692757493706,75.80697599305786,83.79296136549281,63.00404126195788,60.58047941559189,169.7082391924048,154.92587042832702,150.60693128191298,140.01470113527728,153.2735245768724,108.85725537993544,111.96522279805679\n115392,6.16639396521186,5.9661580721078415,5.758160180640844,5.716302450703669,5.681517897116951,4.367301399563551,3.729353560691813,9.336329450054068,8.007710532462136,8.424659897764823,7.149067610013236,7.308132937169466,7.257116107039376,6.287569959671395,16.524976396899664,15.834846117166478,14.400944151255494,13.24427907474782,14.839256881304411,13.529039455402225,10.980758524835105\n115394,51.72254774390305,51.39563961952858,51.34555895286346,51.47370864265973,50.75182976315088,23.207340447613277,21.104705309695127,50.588555267306305,48.09255888965493,49.198438999356114,46.674213227571336,45.497754966596695,25.08305650708905,20.81738092074703,66.38602904478591,58.91052812746858,57.18066860272379,55.56681882473577,58.490665943057095,40.21953731045603,39.502699299062556\n115409,7.171924049447029,6.942042402006887,6.614770853135765,6.565056738657436,6.467268093046658,4.301982143730972,3.6444636188351414,9.632749019173382,8.680264247187328,8.869280850616347,8.195662297737245,8.364206642495878,8.573638190869744,8.829239389434662,15.875233324708338,16.1982858881707,14.953097477734401,14.04598018785527,14.892133788095741,15.191053685380082,15.38869605347874\n115412,720.6443354361288,732.5525037065879,699.3980083374412,694.0242087036269,681.0826993216974,358.5747477134938,367.13834263096606,903.8923308852651,849.735980670299,849.7872153809925,798.9712893847126,789.9069326329922,516.6598696347402,501.5381405408928,1507.7818388998417,1432.6597819488486,1326.742665299603,1262.4763773403122,1338.1260748063337,858.1653792427109,884.1875242081537\n115424,5.771231124702411,5.595369287745362,5.550077721517359,5.5521785105689245,5.487735882874257,3.69358616575558,2.874738535484521,6.0775275533161786,5.853617943638974,5.918739994392954,5.483051962539323,5.385667971592571,4.811196000360961,4.073614108906568,9.115005953595315,8.41473436188293,8.176965696454774,7.774666189187345,8.201535520602924,7.714341091940341,7.226080336884719\n115432,65.54893536579489,64.89489993908543,61.900784314148744,61.80235090669368,60.13743275009998,46.71622557058271,32.18946135071426,87.43510340698622,78.16193038639328,81.26971712114062,73.02941582791122,74.79090433023947,78.8165781882738,78.87532403492871,156.70701682807896,158.80831618225338,144.57583302213152,136.32840969498366,146.25747024163442,147.8873111899098,151.38077861489492\n115434,15.750776654377976,15.165935773441815,14.632977544054281,14.650313282624335,14.885436887885225,12.038820500360426,9.363366461878085,22.97322644843553,20.589471693350173,20.243665106499048,17.87543078958867,18.224784417410834,17.886553244102153,20.276939318975213,42.998605425411725,44.28744755351556,36.97853189053844,34.05286935101517,40.010388886054116,37.53175863517814,37.92843251462158\n115453,20.384646476566385,19.00166529376712,18.428095598892597,18.390040697821313,18.39731570952149,8.772808535858221,5.8545398994525275,16.387310890735016,15.513678671357932,15.40981270735272,14.79506058844273,14.438523352491186,7.385615815836424,5.739887024233472,19.813600047901883,19.365092432056453,17.50016338999546,16.29302582621096,16.342867812116342,10.090961130205388,7.544353669433926\n115468,8.736946368665816,8.609705873257923,8.346866323642345,8.352476351421316,8.350715289558714,5.316647725397041,3.7254174505856605,9.569188414725158,9.119433761543592,9.411390662241482,8.32475417335489,7.727350215170193,6.2990316882691495,4.910841158339957,16.377655235736146,16.47730567070195,14.769509934511689,13.39157225946765,14.640744919760705,13.868764977820668,10.587846830197796\n115482,14.296271108939399,13.501336142778964,13.182383324059861,13.190039948506369,12.870034462597049,9.39526425425331,8.701854576247438,19.346863414589567,16.836911453937354,17.052283996639993,15.5413894885173,15.827729899308284,15.858567249692259,10.19989457414028,33.386986219797144,29.101958471434173,28.302924967292622,26.10088692638808,28.348301312868635,25.540706703459296,18.328017364394707\n115505,566.7436048094369,557.0097245593638,536.443697850553,532.4245262109644,549.6108016416975,360.8342830092992,350.2181476402324,544.380926981412,520.7695773654006,541.5638643372056,480.10334856649416,448.69518608414785,335.78193093322085,339.6943604655451,752.699154728167,710.3441040288087,668.3043007426402,641.0953811015011,669.1061199559559,418.83206283766026,434.59273072677615\n115512,14.517273441388172,13.661469020070111,13.19593754834411,13.189860452617646,13.235533172293716,5.594001222950448,4.302775645785559,11.672585859404135,11.00863532748891,10.957784580788577,10.330118089205358,9.88658734629035,5.238882023786822,5.388487507937786,18.798488728924116,19.1688144213828,16.077800178563503,14.773009387518796,16.09084131739625,12.689804649981667,11.743877847760213\n115513,93.2424286919865,91.54534254667094,89.56824574460971,88.86783448893394,86.97658372093476,70.91476867997622,60.5806295562672,82.85412419526598,80.28359653750985,82.37002402400704,79.49046793380792,75.71957632356302,67.84109121250408,54.33625244623383,128.0274390180681,118.92190539362421,111.00519014194425,105.54199032423782,110.32502685158592,116.24376290361015,80.86917723359436\n115514,4.181363258539257,3.88921853728237,3.814547609248015,3.816325993522132,3.6731036456149524,2.452751421354413,1.4148361031129821,8.31869191442673,6.692924343867917,6.648021716299503,6.059967535678556,6.459786466616198,6.38336845217115,4.273361910678496,15.009477161617417,13.076934603718746,12.601622985599779,11.596730752748298,12.741562251555303,10.957889620317527,9.237651730387093\n115535,47.02736849257038,45.941350605446935,45.42623494584027,45.39457191792035,44.818541054469726,20.997322666208767,12.8620444064338,54.68802596568865,50.98285824183264,53.584556210515764,49.93953537454755,49.753271183992005,42.76719843752114,38.18983261187545,87.09682136372568,82.51849972849068,77.34557117557354,74.91081375198299,83.08828105511671,73.43326380236905,81.7539211354611\n115541,5.484123928871169,5.064599853810579,5.0679513830255445,5.083975120236153,4.926381463731172,4.257073539470331,2.821554508972599,9.33115633792703,8.686382259506821,8.728674958291279,8.323077942454665,9.302834870135865,9.74363770679312,9.098910949333373,16.499166279628795,16.334466350824247,15.962981639159002,14.896763257672628,15.961441241484872,15.672617601601068,15.615146712420419\n115545,52.33155396990519,50.37980265643141,50.40241384779141,50.19063832191173,49.34395288400316,21.348671499939957,16.39673651533061,43.86133736924034,43.65295894577736,44.5379424882013,44.252407057367265,42.27486598747393,16.278340315479305,12.76918013823776,52.775593952801884,48.99011381780016,46.61935903260161,45.35441003150373,45.68809064538091,25.736622771089753,22.124383121433695\n115548,45.01361618246096,42.80981301081401,42.27447887790886,42.18394687210877,40.86850795303254,34.042755130884714,28.05193239129157,48.687791638604224,47.080499591673295,48.068098291250664,45.33372139648814,45.69316995837331,45.198001553519134,42.52471936954621,83.61820635773958,81.78432150410332,78.28497573249285,74.13422181017424,80.26166762247023,78.60473523323924,80.12260271318355\n115549,201.2643167991794,198.84308326312456,183.2141499434063,185.87488726617696,187.19791165736774,119.22356153148195,117.63798885477746,311.6486625155704,278.65909506388704,273.87721740441043,258.29855367347665,266.7162726723867,202.37113584289895,208.71530201997376,556.2469698529846,541.7949348429676,506.5291150134991,474.5421186215104,499.19251774148097,317.3311311524729,322.72455585979526\n115562,145.930830504764,145.22871272032484,134.72243985346032,133.9349383627091,139.01477151404765,88.16843913536785,85.61463632086732,176.3771033740406,159.41077312809108,159.9651222043187,145.36040801789883,153.8577416089286,116.38534284863844,119.61328854312433,260.8917715250588,254.8920563250523,243.5051311677363,233.808606962318,250.2544512167424,170.40479161558508,173.60283269900546\n115567,10.664317845248497,9.937580334885752,9.938684539847934,9.897423961103847,9.658946276440131,5.507217888904843,3.647933135039525,12.085015846582712,11.360643976250243,11.596635594874213,10.79860543920118,10.524468983002931,9.004865355199726,8.96602682439715,20.565550694985664,19.3190424167733,18.69938078751584,17.50380199823507,18.802404126788964,18.01638407001212,19.54712708339066\n115593,850.9322807090783,862.0630888719315,820.9996713079744,817.7194219102453,826.9222949128392,723.5014761357212,599.2117839200675,834.5108701552039,778.2245538828487,768.5799100665184,725.5869574304379,733.0023500989963,744.8168162244884,446.1786118629258,1192.3318715514797,1118.850591205993,1031.4224235529048,991.276170829919,1024.6484524721573,1051.7742286458647,571.1994491502899\n115603,18.594613580110444,16.182967678972574,15.44876002750648,15.303584534345312,14.924091323345063,10.932784812192027,7.008731420274273,21.243453263902534,17.90346931161074,18.62675942682343,16.829800953272645,16.49810478761881,14.897046345684087,9.637095607410837,39.78523739864448,36.74043742192587,35.00583235456772,32.902860526578166,35.07568523699954,31.606061535952097,24.370093996891946\n115629,35.60866937865226,33.73161214382296,33.05110358116346,33.117643871702505,32.36586810222163,20.219040946869807,16.682609951472884,33.13347992831122,31.304478651259185,32.71257178906339,29.000324014958572,28.138118059216495,19.68275405562168,12.571029115863238,58.15007223998991,51.22437725574065,49.246029692076185,45.850170236840555,50.088469860966995,35.767943365232924,25.41452281673267\n115659,64.14922448279984,64.38740638884978,60.41481504436704,61.695039416379835,63.414201666835055,47.75487875975952,46.86342323105183,97.391331176967,87.38616348164418,86.54909707037751,77.91632295731563,84.90310475651826,71.14249266189977,72.60049007947669,159.63825792092467,158.40194221584272,147.88640020894996,142.48110165310328,154.48163459070747,103.81430494751244,104.38299912893756\n115662,16.373587976414314,15.917340340258498,15.571572852596912,15.382060795309643,14.952363371947554,10.572649025194675,9.829916170574593,24.164831548302278,22.12636372646685,22.448459895705536,20.3152247910994,21.99408963896308,22.095806977960255,17.308813628136825,46.28688473282266,43.60482474975454,41.9367729431872,39.37350678128892,43.57264950956712,41.185332279498155,36.38145466947692\n115675,36.43725239694355,34.703034360198394,34.51497196842726,34.59949596100696,33.974910368958916,13.012191326386178,10.12900714175809,40.12352553975146,36.49361991462061,37.819228556772316,34.164760097682674,34.04725508621735,20.32615213133541,14.921898956190912,63.765217321877586,56.35414748613458,54.12203978101476,51.700485203211954,56.87249708488519,39.21239243245531,37.26107153705538\n115738,3.6936740814082496,3.6174052822843388,3.589490382509944,3.5759403067923237,3.5022408556354017,2.6384859488535986,1.9490853696994905,7.288313765241163,6.634645513897635,6.609397588362989,6.199089876024339,6.437234590733701,6.8830582400256874,7.125149249113979,13.637821514473295,13.36092011466655,12.686194934784949,12.085399170741528,12.820401332848714,12.908858094949828,13.709598913552886\n115739,24.869252263283602,24.03201091574551,23.867763053225744,23.796686161546052,23.628990032734603,12.694039877873438,9.77553178720489,22.663853110674502,21.994028285853894,22.149797510258015,21.628927145466335,21.03105066642767,12.82177875154033,10.904345943808048,29.52298788004234,27.30950862888257,26.461875527905693,25.802275744766227,25.91484322889468,20.28839399296771,18.400700952075226\n115757,7.8245649955255585,7.803922229390755,7.678944964617664,7.541785644381507,7.5451991154755715,6.264252609583612,5.350251823841257,10.411428173284897,9.283935699178857,9.494576270564957,8.378885178934297,8.413010150679254,7.983448706913484,7.200270385364748,18.46187739706545,17.91030700489696,15.888452447705275,14.372951305634652,15.844279474667786,15.598154446931973,13.660370520360278\n115762,16.335958242048314,15.669974934268339,15.428926357019336,15.327266927219261,15.005128738713283,11.869426456026316,5.063797290892196,16.536899021650004,15.322293086170957,15.374847635004407,14.45783599466057,13.908709187004975,13.101799678444788,5.659724873241421,27.639567972211335,24.573969027571614,23.260804446604727,21.965169951426873,23.175831680192985,24.81810015753184,13.176278192065041\n115806,16.57552248021006,16.143847450563587,15.64436878874291,15.397855452288713,15.224095843461662,10.57203668567337,8.087352711012768,18.036882717379825,16.06615133458763,16.61472433331642,14.814921401770901,15.077195823883251,13.361519949907175,9.997257942294926,29.472334731577593,28.72222865155421,25.51160538493169,25.051055693150385,26.412665433398125,23.71713716419942,18.673708050990616\n115827,2.130170302264635,1.9918147643325652,1.9740797680412543,1.9749861159110353,1.926990957576371,1.0721083032642078,0.9327264772565889,3.64878956543969,3.28924686077645,3.2356929864387673,3.0050283890536056,3.137461035038393,3.1633883901170607,3.0652708680307312,6.424972268590993,6.121599952305096,5.928262821928345,5.576541698047117,5.924482175295688,5.753003743820752,5.687976216490031\n115834,119.37059163485154,112.76753166478387,110.54115979408218,110.95959888088642,110.19751410596983,66.42878367089503,66.72412377422938,203.0385165709757,190.5772621663635,184.09247869987084,167.0814773309917,180.74518835597215,113.13670180285288,113.01118291271987,383.9264452023171,367.49281143244417,356.5917810790689,337.09376082794523,354.3361093337754,192.1393567440809,192.16737437931158\n115863,35.54777700001132,33.633954864034926,33.39396055022178,33.480566689148,32.65797860886393,13.591133747625426,9.859086594465706,37.58167899048801,36.353031291657125,36.67699826529056,33.88718756951905,34.795260864624225,28.71500681042504,26.144882851267166,61.01502546327728,59.107660655052555,57.700265550855185,54.36926000611098,58.308859596486336,52.22820592461819,53.11953337898075\n115888,241.6497390847632,245.1691689908857,219.77438312571283,219.55056368506993,227.33702097713805,190.86582333660135,200.9614390156168,185.64392197818475,174.66979251867926,180.42227425698363,156.94413330581486,153.4250631734886,150.77741495599324,142.42261002436823,243.92408784055743,240.52829757328212,224.83429562489155,210.39441742374822,229.789237732886,205.83988489436334,185.25842294230847\n115895,1362.917373794928,1422.5824288670065,1369.4201839859697,1374.398669467684,1356.433890107117,936.9696563376332,898.7446885060183,865.0646629630644,872.243985332854,888.4535424109409,836.9360524936875,748.3828326187155,578.0120392016569,403.8656645710009,979.2020447238966,920.4803997921367,844.7851127110893,811.200087582656,822.4353379674395,755.0211786117874,479.18496043745836\n115931,28.211755551047844,27.19347580304888,26.616895730627565,26.55228898186267,25.9287099975055,13.937311787778267,9.391604870922063,34.97274457911161,32.34927820871589,33.266229068889416,30.139971587319238,30.44582269138518,28.266787424044892,27.613006342331623,66.25186931256101,65.24269438936939,60.03315121421025,56.61715266058063,63.28119020608099,62.661407844299596,68.14319323554669\n115934,149.35306090820524,146.21129440188713,143.61076784057246,143.05509081965704,140.72412619050507,106.89212943504147,89.16821550568383,137.75287737757802,132.3671883819385,135.4073997691781,128.79040762303146,126.4988636864067,113.48769290850423,87.84847471371225,192.69109733013426,178.39599963020507,169.93634342241788,163.52051738218097,166.941890739999,171.89438714347142,120.3302362443417\n115952,13.83457794117633,13.486065753373083,13.14649567512292,13.170049474262068,12.824823608962559,6.176764905650348,5.38602944757144,21.831114257243048,19.354085104632812,19.413640989116924,17.50326312494091,18.99277897592292,16.9659322292401,13.666613352549001,39.84476477059423,36.55354248609572,34.47787803858441,32.57414941856645,36.32505899924652,29.36563308630869,28.917848698081315\n115958,6.555331570360131,6.237682582090683,5.981759016600669,5.968390023195512,5.831162740185587,4.812271654522988,3.792503525577512,11.717343123072574,9.897809728556334,10.293120423900747,9.164401703679776,9.619154784380475,10.180136484604185,7.69204327397912,20.467349623145054,19.08170158285118,18.254266610115707,17.167651274080686,20.1962720599169,16.211866694486886,15.249040547634857\n115965,12.030111613145522,11.560688818392642,11.273649187486798,11.275038377245409,10.8846778582416,10.40649079597232,9.613206294844554,15.49604577852186,14.275019407180636,14.77801540462802,13.472279112645966,13.646033419187773,15.005117762526238,12.575781472760477,29.087020531711484,27.085180271986882,25.461743303912073,24.26954248111889,27.539428755163424,25.122832095311388,24.422236368600544\n115976,1.5795141323047452,1.5342988269158158,1.4907386708355985,1.4867607396676374,1.4975705482381954,1.1332448479586799,0.7589391397277773,2.326614763591362,2.0860887990331047,1.960242661142026,1.7427772774128356,1.6787736892764418,1.5721519391389098,1.7454039506901595,4.349166596087983,4.5087403257471195,3.951867412956676,3.5073726437187185,3.8957304719609938,3.8725897237083196,3.7240845867704664\n115979,178.23531990871408,173.9012396303498,172.25446979454992,170.3674011158995,164.6978984168991,103.1580098680581,102.98796109507956,251.14032077980826,233.50314377347502,241.80892437623262,231.66966423005772,235.9233704159056,154.39779018048714,159.36670523632478,437.53196069226703,407.26438842792254,387.1945881513524,367.7485803083749,388.2009030282068,282.3336348036543,285.98102545828544\n115983,6.001764154106364,5.851541424350261,5.774464353621153,5.7705230352141,5.67531953787539,2.7750777360077565,2.857308514492305,6.962648845127147,6.430069594233078,6.41706856413932,5.908399079073719,5.794280663802411,3.60530293086219,4.122340577002965,11.577774870991819,10.227687128486409,9.804403745057897,9.163050576981354,9.65831582718097,6.665958708589976,7.759847147236201\n115984,5.502635950347038,5.14142420131885,5.218259714963687,5.211637815532313,5.100605622008877,3.3401213580494686,2.387857813301855,11.471796286054726,9.803447377595669,9.649325604209842,8.876597299196561,10.438005577505939,12.135274895243533,7.06348506233108,21.48608477390655,19.739263088102412,19.470298347573518,18.029932249343457,19.868374549814938,19.292086857193144,15.61819951713124\n115986,12.972645275651958,12.554201111615107,12.40007700042254,12.378179256674704,12.082712252027775,6.054538862278685,4.15560325405376,26.9469391607789,21.738831136503883,21.918597911643094,19.676920846369452,20.544927449132228,16.873342101799125,12.692884496396067,51.8952976958641,47.91368551904778,43.28835735982398,40.35843934688445,46.20132110246732,37.24423928889301,33.068435344271094\n115995,12.035346082769376,11.423103551436759,11.168248005884363,11.055657832010063,10.819673092154964,7.581221458452436,3.8131007589307333,18.625910096936813,16.144143188192604,16.73455476266212,14.5145212098466,14.777428572844125,16.8746957153451,9.997974159118728,41.816026887884405,37.84722359437768,33.696056136599935,31.792577038640257,36.02611631734974,36.19635564163573,27.144778445788752\n116023,10.59983538072357,10.50894374727374,10.366655180165733,10.35174019724815,10.115573629989626,7.381551941870528,6.002096162148631,14.623063538878203,12.801288679260082,12.597125629137512,11.527191892168874,12.032565696217715,10.811698658106343,5.988912024436585,25.96154066125193,22.461458337231857,20.973071801788485,19.941034740969414,21.604640770070333,18.408836153789398,11.68428751480762\n116024,162.3650923018516,160.88027041843938,153.730107560474,152.7651640403629,156.30005012331662,112.45417574489245,102.35590439990766,381.4079485761567,330.0929833177069,332.88392672627884,304.77026184997527,320.6867755477062,308.74962309391026,296.84909603381595,701.3006077601236,653.1131214260539,616.453236775259,576.3596846310229,633.9375287455596,502.2299590140113,506.0760058732191\n116029,10.72124550171045,10.408162485097362,10.285055339827991,10.25942481042021,10.083864746433758,6.642187685790755,4.47205831499047,16.22794659388185,13.907979665956725,13.628165550349552,12.500757344044516,13.120160461998982,14.610338983872525,5.850843235873529,29.389227456272113,24.9411736111188,22.461815885084615,21.299121306673847,23.117627846205757,24.316441125471698,12.222168899075355\n116039,1712.0921548403767,1638.568894943228,1611.4457483390668,1624.0394258686295,1623.5499200566062,1475.9748228330454,1250.6788689426846,1465.4723564529718,1442.4470100092944,1395.6304286326465,1304.368158453303,1294.670706482141,1291.4254013626091,1023.3828833626314,1904.21579014055,1867.6870610785368,1841.7576796545704,1729.9616803895958,1796.430156080867,1684.2757974862172,1395.3828175359042\n116045,30.692310528178247,31.078246667470346,29.275098910640846,29.178485742643897,28.800637068983196,22.230113736002128,21.989844066542382,45.68682432918599,36.45721475194158,38.06590325874231,32.75916543181097,34.20390299922009,33.46176919827168,23.034539195357933,81.4229472304474,76.69167597221795,68.13970006803393,64.00256978893356,69.1587524927855,62.94368084375093,43.24380347480095\n116050,11.147302630665456,10.840426019339503,10.642416612576762,10.589271571207414,10.684596240379053,7.787214336047707,5.623412267688532,11.32401027564965,10.383303010633547,10.157756940462741,9.423165933959005,9.074727662769027,7.567591321933014,4.811467172952914,19.504910716664014,18.81527938981981,15.694885670757492,13.976049713547129,15.250594145368362,15.367600292510515,8.385835887959393\n116053,1128.8058948885769,1103.4606379633424,1090.665097544402,1085.5924442917787,1070.6185421454,946.5386112357025,828.632760393979,1040.739104238,1032.653676722758,1059.7681695489741,1006.6990153466876,1001.0955631063547,948.2320489058199,933.5954686302233,1350.9918380673128,1316.8000946025807,1296.5390380724507,1247.9680775245056,1314.2348128220567,1229.2682088989525,1292.4032611875946\n116060,19.687511276225237,18.874100923689397,18.36835704527888,18.325875684399904,18.66888989896255,13.215894607791782,7.4905283364113195,18.620370705885296,17.244390727562966,16.81789540532599,15.193697732384669,14.271706265214956,11.269736937022616,8.197759511458623,33.282366798402265,33.71453112338605,26.787852506751925,23.62926530596523,27.784623629982157,26.01853729322729,19.337855524918265\n116071,3.9625241262368767,3.7421788061742105,3.6303657230666477,3.619955632974217,3.5153011411520376,2.1330164740872535,0.7566779527405404,7.0365392629701295,6.504051415152152,6.465040406766683,5.96584114305104,6.27089362680931,6.493487416928233,6.062289820455779,13.420898591837108,13.261698413289901,12.52464040987893,11.8711102050484,12.498740167311107,12.236917448178694,11.923719192680498\n116082,10.22712166942877,9.929177437627287,9.746300369929719,9.689532588699281,9.790374431339039,9.04796491980721,7.947996517117832,15.463266020822973,14.223042596715937,14.160960587047455,13.699037660192554,14.751080912352696,16.29667096445387,12.181376801045598,20.97048008237896,20.48522833608899,20.70125708945872,19.67618437339866,20.68918923240085,20.74120519015858,16.744601621219854\n116095,7.664889743319245,7.511390902685375,7.248998772914356,7.23227090738981,7.123237215827128,4.979002368438683,4.470983477110521,10.346037417775696,8.753862759113229,8.82578455243025,7.835551788836897,8.281456139546174,7.130510086942746,5.216053609626853,17.66724276326537,15.658196658106428,13.799609397168435,13.125674217324345,14.467397896063554,11.862179966609732,8.007421814778185\n116133,6.489721706482537,6.314990772500532,6.319839032926902,6.338165770279851,6.198445997519421,3.400640384083099,3.1101337162673954,9.321615810228487,8.140959752875071,8.255403630073623,7.503234312055094,7.618088068759579,6.256244074339193,5.3166957794929095,16.178810333624714,14.42639629521948,13.952910925249151,12.839202454443063,14.269998511785493,12.341851949593906,11.872714994476462\n116138,19.64670329200619,19.288518826487564,18.900368899690328,18.837885602822855,19.09396580021321,12.828603915847236,9.81690561165268,19.161529481799676,17.93276148416622,17.286551141868177,16.336857790404427,15.806307021867873,12.74765801671453,9.082702977050563,26.186659768642283,25.150014808999448,23.444756840348187,21.7920233129569,22.798805249646836,20.954946786414375,13.738294072362846\n116142,183.98703846380317,180.04017009597348,176.39156527673256,175.54540087248515,171.66448074708418,140.04163914653697,130.15029199105564,164.8528125690223,162.93255114207403,166.99654200686763,157.0407233135765,155.47548344996676,142.33947801354182,140.49699992637704,230.55775614453853,226.62947106142377,220.6507281358989,209.58242833396295,220.79279866731375,206.1335634940098,209.9469733176141\n116145,20.0194058549782,19.750264611005303,19.446681868939677,19.4412300780294,19.000544077348856,14.150864630929723,14.497170611660007,20.793310982093036,19.976755995563536,20.231121208836324,18.521195567009652,18.26092497895753,15.787544896895856,14.989941239460853,36.12100114094232,33.55440021818296,31.81914570066036,30.061825999452257,32.212187642100055,28.492005275157894,27.61885591487114\n116155,1136.8868256179426,1144.6537305840188,1133.5540937301407,1132.3247221537267,1133.711176157972,911.8380799532154,824.6044105988171,1148.1112262030188,1110.4223483234398,1121.5551891123973,1094.3738550825462,1099.1122068575298,983.0201143328422,849.4482470103449,1323.6126206936344,1303.8855685392443,1256.100857601686,1242.0146541060215,1264.5137130204519,1185.0208680366777,1000.751597786861\n116159,2.274225992719462,2.1793845474509714,2.151332886782036,2.1568506182689537,2.0917996749896095,1.801537979210096,1.8861446057937532,4.5419685286199085,3.898897417707411,3.8308982138125023,3.447570805408262,3.918603075809916,3.729406648237028,2.9975112120965677,8.454308025397847,7.65258115675078,7.338445006020311,6.919200316671058,7.672693714191082,6.361916266048721,5.834048169916641\n116183,2.9096717132344625,2.7882264896457776,2.7359022338562053,2.759862175777304,2.722250770213053,1.8453291588095386,1.760030288637519,5.077060056490179,4.313671133186664,4.292943472317903,3.9589795153074996,4.416304589022926,4.496224226606674,2.7031092332569573,8.5145723853147,7.7654832608865,7.593964621799107,7.0942162162954565,7.616053501689713,7.107705317014848,5.220704558667715\n116213,45.053392645797324,43.592402627409854,43.70624207295459,43.55627905140431,42.4766267590331,23.734170559007932,21.085661885413337,34.39053579754405,35.09962331078329,36.348442531526466,34.28004744481902,31.716393278019588,17.83447323059967,13.648136331103029,52.939362694809844,48.71340256669666,46.36845362521536,43.71365223859379,46.75358935614983,33.77451676090784,27.724063433450812\n116215,9.190774484667473,8.72195115733716,8.46865930588928,8.467104787880652,8.581585899176567,6.193090498443087,5.415065373097682,10.210199920464994,9.450644195520235,9.256869310897637,8.320290194410333,8.102428340566778,7.365068198176705,8.44380534532317,16.60722578640063,17.074934802249675,14.598125407833933,13.319166836845788,15.14857536828075,14.69800547102202,14.889439681419121\n116221,19.702002056003725,19.14271415929491,18.995717719005313,19.025846149868922,18.500529734524754,14.8924710398656,14.910165031729875,21.888236132676024,20.489015794886885,21.030656873411843,18.73741018094885,18.63478214450792,15.554897040169212,13.061873219009803,39.54485961677694,35.19457413636928,34.20069234096999,31.456681196775737,35.35541375325345,26.518812305274253,23.74060493252633\n116225,143.22776891856367,138.86797243907276,132.8959359805301,132.13480382599948,137.95163165486045,75.38989740197034,73.63443193359325,239.0058809969976,207.51568643549402,203.84078218809555,182.21207693548396,193.06757343512845,147.4474852343658,153.67304964629514,407.55603726052334,396.25419255346947,360.8543638347331,347.5181939096978,369.2128313547493,215.3597742492816,223.60900294054167\n116246,102.39812424340877,100.24598143308832,92.48605907374353,94.8584537189723,91.80543931752099,62.82619517321649,62.08980135919896,159.92767679683692,135.4558920534683,135.44026426250437,123.02803368423467,130.322958735324,111.24503278798119,110.20262789538441,292.7641369826923,279.3702963543809,254.787743722345,243.60643878821801,265.61142593881306,195.62652247887155,190.46448372130556\n116254,9.792588916877332,9.641499997653042,9.180481791626834,9.166375632825519,9.101134200933673,6.5725911258772305,1.9427143574006787,8.836679542711485,8.334405898984945,8.379528810976728,7.935587284996026,7.871982652848081,6.297568679041394,3.3243740434553204,13.121910931227228,13.078769046570365,12.188483138070925,11.172253619803447,11.709823321562329,10.496539034583567,6.053939049518909\n116260,30.91695463608756,29.906528474740643,28.618592411934696,28.234164319285057,27.82781149138288,19.648143220207565,13.88532816953263,38.89788280042208,32.81913282336657,33.12920594924621,28.2484083202559,29.812016060736777,25.01825133105396,18.33291548494232,75.85217102259934,71.91801773686227,62.46490086570388,57.29131533415084,64.66335461304091,55.392242071359384,41.92064968346651\n116279,2.3052426774800745,2.1275639423715016,2.0762310807389546,2.1043971707757225,2.0585212034443963,1.0373102309124134,0.7450251659961474,5.308403099022274,4.586169536740208,4.847859898831546,4.2985739325612995,4.716126553935079,4.619400016020674,3.6328475099337525,10.689248724964866,10.03462737740136,9.848906491045287,9.029793442572027,10.360001717055352,8.992111290041253,8.663013245638368\n116295,7.053535125588534,6.597481332510812,6.50791725615145,6.4798041703425735,6.3142352144725225,4.878098206483889,3.435121590875764,7.272272060972023,7.006625179497581,6.999974776652992,6.239350935735998,6.107764315337997,5.739741854526693,4.3570083881568635,13.466356661618834,12.206311176622979,11.885383798458312,10.96803686748971,11.932302737656546,10.662985986444937,8.952222822521264\n116296,14.03074559108691,13.535709379079508,13.330892962106365,13.360431650869296,13.012592815497873,11.655883708111904,11.423576591412237,17.045660893458543,15.852778444594035,16.21662135654567,14.412896792429228,14.696454609719058,15.27784862049269,11.345022052825705,30.231444235244673,27.68409670533746,26.87353533314499,24.816331265821088,27.259569624610744,24.993152431484038,19.984158606323746\n116314,253.91274529468114,256.496560925195,250.68779281127965,249.5920691101981,246.35625823176397,175.11164498768474,175.9537411197383,226.68585993408087,221.5293603719645,227.32005827070358,218.7664711240015,212.41382370584407,162.94864243527658,153.1294388841968,284.6617381707082,266.88596091228624,253.4880811382177,247.72271147931292,257.63379539922585,213.53468460970953,202.4643611370757\n116315,18.494953814770906,18.203912380125995,17.67071877379836,17.32677358786954,16.664069806927337,12.859835165554921,12.088044621198465,22.910310296163168,20.273105740639963,20.452679596474788,18.826806507970893,18.58961234783627,19.16128936035465,12.22195164505744,48.84426640416904,45.356601356827746,41.11780813500805,39.04759782228298,42.73603309982716,41.390249161753474,28.817972320684138\n116317,12.865934257097507,12.681095220706124,12.257516360659801,12.195322937272243,11.96415070823903,6.859666397718003,3.4310945009956595,20.251948321898027,16.318558524149157,16.983745018577796,15.204736648952778,15.279049962616638,12.664741807897732,5.393694442518787,41.862181156967694,37.93257836515077,33.27347659614876,31.150049138597595,33.56928647725586,30.385271775831153,16.181929396882282\n116322,374.41814236185576,367.11691128641775,362.9316909984873,363.1560980260721,358.66723915617007,256.55141804364314,245.33018899568066,345.46297162388805,343.8250408024332,348.17894052474065,324.0529974595239,317.7781449669554,276.1619515509645,259.073843738566,515.3447904903642,500.93343151038835,487.3967145249961,457.9319552542675,491.61660506364376,433.1205352095155,432.7148152195813\n116323,1.9767186500435985,1.9667534226712775,1.9343238952764537,1.9118717470923248,1.8820713942248612,1.409392198582576,1.181915775752156,3.214866263952451,2.723324809547538,2.883711858938094,2.4727174407079637,2.4174275094599054,2.7329597285531517,1.5360609069062043,6.656865046258556,6.519704022402692,5.789011635489528,5.322133093405821,5.9591001814611015,6.135255422917119,3.454754028653753\n116328,1.7556306566586564,1.6993557054950637,1.6640145915011915,1.6574898731370429,1.626675335032456,1.2527053977057536,0.9757099637394678,3.522480183058485,3.1810029332572474,3.2662032758107937,3.0205076802310793,3.206954857790123,3.6132105902708904,3.549710472041283,6.692313680666646,6.527359902952911,6.080649312316949,5.764098591685616,6.301337017462199,6.160648462348031,6.548439757089095\n116339,11.313574580276475,10.696025290292601,10.653809847493603,10.639579143362408,10.356656722639284,6.897058142364675,4.848626496164918,14.28460664680387,12.937501240077278,13.133585082144371,11.822464558214918,12.066330405955163,11.457694062756772,7.484281522789441,25.330070899241807,22.571331264983364,21.918373971635475,20.28893001732363,22.426634997178468,20.288740342094084,15.837188377707202\n116346,4.484197160400245,4.244184587590181,4.1040083680031465,4.065333175447444,3.951816030722709,3.2752973944345802,2.7767659997770573,8.54849226544119,7.5829947319002775,7.540385245972295,6.945395211870551,7.6187697292068925,7.95887177741339,7.151376734297323,16.553862454266493,15.77535424342448,14.918753759181323,13.970597371743985,15.116552910387028,14.180389545659338,13.512001456526205\n116396,8.839718414495318,8.08358015987737,7.424917371925129,7.414125964231139,7.182210441026153,6.749946720387631,3.966430632229756,21.73464913673446,17.901632079761033,19.47556216867129,16.685951137970008,18.21458947941235,23.97322558441696,16.876699092361182,45.880564223258254,42.09698385807109,39.02695344471981,38.01375065266653,45.140426056480784,39.601584704615306,40.52441530953416\n116400,7.913833276434377,7.459662602425198,7.336604576909586,7.313080771189906,7.116393326552846,6.062737654820363,5.687373768200188,14.475627047159334,13.431779259694313,13.777695645058854,12.402527784015174,14.068059530038608,16.032937458594372,14.12470248712424,27.90883838610406,26.918840659594718,26.280443422256567,24.94774431027067,28.22703797593734,26.238119654358602,27.55813514533042\n116415,4.43774267234832,4.296402485010154,4.293845992701101,4.309416037346203,4.218130483224109,1.134800429408365,1.1306166587661315,9.087767003113274,7.709554323757171,7.546895036378624,7.090302212622427,7.887581226667228,5.715350454594539,6.3930424696805686,15.57174172084172,14.188251242011178,13.62854449307364,12.531426496105249,13.494307413402296,10.08658369161407,11.561287877138284\n116420,11.4544923558686,10.830741901140378,10.679199848053853,10.679415439421565,10.368214736199134,7.298452997102097,5.601556895986162,16.350781477349596,15.463790190976656,15.738122116099165,14.470868965355455,15.11255810710908,15.374895351102484,14.368038735613542,30.721008366267153,30.036834586179868,28.93665248224114,27.057139423742974,29.16009717542701,27.891784623172164,27.419859337239505\n116425,6.561456771475214,6.0574140023644185,5.930665644866057,5.94345099503202,5.758329871999117,4.454598705869372,4.086522250273673,9.46947668119133,8.99958611649466,9.221095921180735,8.464262354536194,8.825130990980924,9.669049671636767,9.4149228642632,17.235677684675863,16.87500536183468,16.72368777187602,15.700416790688724,16.942116977797657,16.316554912495842,16.80015459303164\n116445,303.0485024294233,295.487843066891,288.2670323178065,287.9960962271243,293.5260381571891,194.55573219052718,186.56353497475442,509.6551909858744,461.0722547329337,454.9218986892087,433.2304273197461,443.22370528639425,345.4332379114914,354.2449328096706,796.829928513656,758.474276504018,722.1374108326826,686.5326560676208,720.6277134722249,460.597576352054,471.6046154251715\n116458,25.748244704777207,24.038767965604233,23.94390481022443,24.095275822248333,23.605736766500193,13.585793206080709,10.577954097918028,26.456424176889573,25.13832452887055,26.416283573671876,23.405166117226734,23.4588645134429,19.208882823740698,12.297027604129205,45.96829913617613,41.42243446642316,40.51953500718887,37.63688589022124,42.92763063851211,33.707552268753936,30.533881910829507\n116460,42.330640286293665,41.572882470804586,40.77013544654361,40.901886434662764,41.18475576957562,29.997556030043686,24.009426543529546,51.26270973564609,47.61990577290489,46.87263153880336,43.90792347253785,43.936072181511804,38.067828963797886,39.91146051598449,82.60021479314456,84.85975945833313,73.77808235717154,68.17715933532169,76.10662421897952,70.73342590262752,68.59372795088066\n116469,7.110286702084506,6.865758085735509,6.709452834728299,6.711444553738782,6.590542214203259,4.267854865391114,3.8170345066573437,9.149676467620141,8.001709068742219,7.8398797551472725,7.167920097558532,7.363328398742178,6.780825556521541,4.04926395796026,15.660211926572975,13.536633941310164,12.68723460437246,12.0459845533302,12.921061083532372,11.750028189539835,7.447555290603472\n116471,19.944951832135946,19.101799069526784,18.742380922760848,19.154986645842175,19.419301450759892,15.685086552444968,16.10243504712947,32.56439037211242,29.829720595693875,29.251871303624792,27.545275753113813,30.452993603174487,28.617631192860227,27.647860980456382,52.10270321879857,49.3948621603339,49.041382637634015,46.11372800371702,49.15058253803889,41.174413521501585,41.53841325783242\n116478,1156.2104770030544,1127.878759457901,1117.4128630406092,1116.8755567913768,1103.991601644728,988.1993236096425,808.0089286718782,1078.5532613221435,1067.6390625260883,1079.3516051525148,1070.19936827606,1061.1560691857358,974.3408879122395,747.132717892588,1125.6150938916066,1111.9588478081425,1109.3765236288946,1094.1079535718743,1107.5546543751952,1035.4600026012695,760.700756596181\n116485,16.05404406635381,15.88941031518343,15.5446975898732,15.465514598652081,15.106402546649226,12.05107612457713,12.14366088471865,17.1163649315319,16.04990081957142,16.653736321805102,15.087099752682004,14.505948427786006,14.542355165233257,10.53499450828998,32.03504502307731,28.82058012639645,26.554437515320387,25.068170127810856,27.8023136991677,27.819680608363967,18.92891371233627\n116496,15.579169818201892,14.395333723595401,14.419879193020455,14.404198109152986,14.028759681392703,7.538690221630058,6.026301107604313,16.719764122726524,16.321316116286777,16.826201147189394,15.546754088471467,15.64306486700503,15.203319687507449,14.100102459760542,29.613694152276672,28.54954056464082,28.050891386462542,26.131027871473908,28.903349871146176,29.444914620039697,30.952320281752183\n116497,143.71814035385393,140.87443210824105,138.4715787762033,138.3615709370811,142.59257966853747,74.55231099122874,71.30156898785675,202.98397662418543,181.11196908075738,178.44063821959747,167.8910244087519,166.86948842708628,108.83401219365932,117.49189591447166,313.15044866459,306.2061366472498,278.40365800913196,262.34917595771986,271.22387902392,198.41748325766372,196.72593152004907\n116503,24.934990720824945,23.008128690952805,22.72966471485276,22.563965397476764,21.89244018529724,15.726083171657129,6.716401681486985,29.568869137414868,27.2060696857726,27.467501690627373,24.2431352607357,24.316293816940636,24.983253441226466,12.56318234870276,64.66277612144812,57.767779700739126,53.108700519227575,49.45192882262079,53.98665238279207,51.59176441335498,36.18710461897001\n116504,23.29022071998282,22.2816423691033,21.477708112476844,21.230793993674315,20.633362428794246,6.263806014511395,7.154923223152628,37.723089475144995,30.735718433505472,31.139116575589398,28.300854644678637,28.6703918333846,13.409984210044945,15.568448343421874,72.82975261909009,61.44400643742172,56.213019883121035,52.87183193218424,58.62604770029772,30.276118814018066,38.268490812431196\n116515,32.19713267409608,30.736647312884774,30.2528288914844,30.346947468440007,29.5087861529715,20.815934548936593,16.668680721636036,39.75452400071913,36.78275265127643,37.6750545598601,34.24869071704231,35.304624177006666,39.58755574637853,23.05062249023343,68.4333118169509,63.637872495278124,62.59369719599599,58.71953435572232,64.78905744820237,66.3924048697317,46.32560674146282\n116539,27.680943427069643,27.962890064854857,27.35211510182327,27.42508985683047,27.56270482213938,16.653363203976348,13.634837654408294,29.991894618702162,29.84124136618108,30.17610514026907,27.94483607584037,27.891617058115635,20.594455638261678,19.11599718139754,42.85985496663577,43.16172275270524,40.57198826139613,39.18128739594254,40.65329856239815,35.81246632793118,34.87498481568891\n116540,24.638968219030154,23.406255555848464,23.047466016828984,22.877332816463635,22.22982204956113,12.585717711294016,10.487539449614701,28.15401091842189,25.248435000465204,25.82903469465437,23.224320442265817,23.46882260876247,15.984391403384159,10.234548043529653,51.537808006120855,44.35637474082017,42.46494834404039,39.39155713998542,43.02546860807139,30.46287787349824,21.240896590546548\n116543,14.530237088077282,13.870054662879756,13.99932667232892,13.99270422657138,13.709629723462845,7.185740632049237,5.786577733628542,15.61728315195266,14.981810201817712,14.842572819718725,13.77490273739481,14.059171551849973,12.327222457653944,10.27375763470011,26.46797879588964,24.614786217920255,23.91068501397474,22.5875235069906,24.332459507644163,22.64482176866598,22.20950546136608\n116544,120.11861717462713,109.48514404053793,108.20112494905851,106.80484979647939,103.08374328304168,99.69690271336131,57.57616301862743,137.41310211062722,131.77575509525408,137.8516818204958,135.18530187834128,135.84981008502697,134.93263783830358,97.6635098538688,250.68591395110002,245.97347584683735,238.69757310178437,222.1380385379756,242.83507777598555,227.73463265681926,199.15428211215004\n116561,156.84501603543674,155.5964455535966,151.4685339520333,151.2995328994348,157.02405770535717,90.2889194356705,85.81806700928992,199.69712400185213,176.03127667709882,175.19998565616885,163.60714735935196,161.46598968082668,106.29646671390252,121.29005243520638,323.13772719262107,323.3259336139617,283.8310575064848,258.71341181136097,281.2405785282225,217.2683363731059,213.70463450818087\n116583,13.917717518866864,13.773601282723225,13.700114837983776,13.713930025732154,13.85446470212036,8.711350679950103,7.760868516215099,13.666888706361775,13.09306593486227,13.07123177312293,12.325515680115801,12.148899271696415,9.418146227230874,10.867222210313955,20.016007386932813,21.38582986298125,19.542899960423785,17.42929083911212,18.16645901094356,19.243667168127867,19.31791722550305\n116605,9.050712617690532,8.368474075045748,8.12596563830235,8.134063602503794,7.942318749754351,5.097476340480675,3.2122767479934,21.779823470575735,18.0604022126328,18.711675490306977,16.728506452709098,18.587819956859803,18.304039025481103,10.231633037538234,42.08793006423764,38.081054188785465,36.4605299316151,33.6030136371808,37.90837399600611,32.4752996480757,24.326118319266982\n116615,12.898261518510587,11.997184546184146,11.96239023581004,11.929208865041934,11.465186561217841,7.746208159257175,3.701591522039752,14.888147513955326,13.674402369778388,14.036214632301839,12.243982286730882,12.367600972827237,12.497761559430325,5.450100905319599,30.079231701861108,26.888750784767318,25.894070086127066,23.782977609766384,26.610857440503285,25.149487850452473,15.91781520377899\n116629,15.308959208926153,14.925811505891682,14.681005768028715,14.546448706693921,14.162752220781591,7.23756356423254,6.102020833731985,15.635272926472503,14.64232258515406,14.797000148455153,13.779891090506942,13.338592206711473,9.886095693413537,9.73387130965911,29.548298302254118,26.69396075047477,24.07710113194643,22.733956806430147,24.42253675904584,22.580076263009886,21.458582694778745\n116649,257.0555151039783,247.96452150623097,239.1547559102203,235.8142825306561,231.6166210709995,152.98156429615932,118.32602174766556,220.55749960907434,211.15043112295552,214.58502658115387,201.56276317023563,195.3701089775361,160.10938986971797,123.370941594399,313.70240179347707,293.3337137114282,277.83389472692795,265.86367627253856,278.5653717005662,249.2322144788318,206.87498215789068\n116655,10.227683034586859,9.572092827257272,9.516965480672894,9.50363748155378,9.286434034564365,5.283599251151546,3.660566339947178,12.696596342118609,10.79997280981942,10.656549573819017,9.780288467261146,9.671380128051293,7.807443163767538,3.7361472664171513,22.78624986806803,18.228625416690345,17.322682527573832,16.324152762986994,17.260912708979124,15.05927450870117,7.988848759514326\n116658,21.87045715792743,22.25726748749615,21.515079126814634,21.38768999036377,21.14837689734232,12.793281901769804,12.586895234708622,21.336502525608832,20.16675828198161,20.68558352936632,19.03586342616413,18.452685236947474,14.003315217422667,14.125673535077272,33.05563557006196,32.523537037588454,29.213010705724194,27.593092507264142,28.78349599442143,27.599921490178176,26.014868094828568\n116663,144.85888286290765,140.24299329829788,133.67957419868375,131.62417906729715,129.326108278522,75.73032030186245,76.55661150777802,211.2701134878314,183.20266417079563,183.65258289161113,168.78231472260535,168.02644480633435,127.00403115282532,122.23220430932471,397.6729799258942,362.0038292674634,336.48864131999153,317.5229828728093,339.57013693748434,232.56471056608854,236.43789577989332\n116679,4.711826144985905,4.467089526885143,4.409320952232604,4.3649572014650335,4.2378384652521985,3.3558694665906224,1.9256920271649678,9.993911907071963,8.423249672407023,8.341954191909359,7.50640692006083,8.01396466926017,8.758906944968398,5.593351562001711,22.137063847590078,20.09137772096056,18.177308951747683,17.27492196637526,19.299674590381013,17.848213176063087,14.59430088349791\n116683,40.045439874456605,39.46856916997549,36.90489544558701,37.13263903993263,37.48121293506882,22.947508408618514,23.574439310164514,67.36573629780295,58.748081179879236,58.7566431876105,53.818518436957596,56.86604278254637,43.30650520335072,45.36747261842325,116.84486435971125,115.6623685278977,106.38221325755175,101.23689531879495,107.55368256749567,69.90568117392561,69.3324950134717\n116687,556.9371955236338,544.3137117145981,532.800414124239,536.6477198371869,547.689056356347,503.6338365620478,439.96546897008596,687.5314484383666,650.5444313170808,631.6025137759694,591.4716866882329,638.5569802331989,664.63338533242,543.1257499957463,1037.0367410087454,1014.5113797813245,963.4049093903877,920.5840430904965,967.1524673007198,966.2766767596884,824.703110957827\n116694,105.4430627027375,103.43684377103385,102.09527001651335,100.92602114278317,97.2364400078565,39.64094724175322,38.21272535668386,145.32771220408216,140.72184985831993,143.6928334329483,129.3121389242097,134.24500053340333,77.69879986249364,79.14179071561094,273.8724783135158,268.2026893381808,260.60801937598626,245.19584195985217,257.4273773564042,172.72718809661507,173.22990133074245\n116724,13.565132830069972,12.926566652439151,12.712500446403665,12.727299181022897,12.36621761210766,5.516105524594724,4.360855977063373,15.495330902915114,14.720479519065945,15.32593616201862,13.909899296644314,14.059841724473312,11.624283105315822,11.340034560550967,27.106797319239707,26.140719192517636,24.605247681236452,22.988156838776103,24.90226114404828,22.423783235639604,23.150172408632795\n116748,6.636569856249037,6.319382051825935,6.267839442602662,6.274189374393954,6.117190488228685,3.850764960254625,2.755850199749374,6.73572243350614,6.469731240502406,6.692507447570268,5.882299089087251,5.753122812991561,4.527959252020065,3.1897310052330496,12.303898674056942,10.933839431309565,10.498308582871799,9.827400743466034,10.90810763886832,8.916301345962355,7.217191366977659\n116750,7.737086037026428,7.27610261679288,7.273018148304125,7.240756960039225,7.069524137043613,4.44283644706692,2.322146677970312,18.737027740674808,16.289674684046688,16.345227205978254,14.4025239152656,15.833524587130062,17.034201157160123,14.826355520058996,42.901887083625084,40.97858090274655,37.64125530044959,35.15557564977532,39.02594010213715,38.038868556532194,37.39898865079016\n116772,4.6239872122815235,4.293721770293981,4.308764572273609,4.310108749445245,4.191137147875586,2.020964975389744,2.3627323157176514,7.2403028736923,6.678849845889747,6.900917981955688,6.031341249367414,6.25469876139433,5.7928824563220696,5.691275116845952,15.719941260373867,14.557618557968143,14.130917904517123,12.939919222893224,14.790049904423881,13.029754137420722,14.076057834526992\n116777,5.882024704496285,5.747270889719254,5.525876152894669,5.4459103971821525,5.344915402979134,2.5266011460421933,1.5727980995631947,9.671317811053417,8.354266452451254,8.686872214123621,7.340993878055061,8.265293154454634,8.918936592335646,8.551412965512027,19.696625370906432,20.392260907194196,18.048385503574156,17.357824250733106,18.586099127050616,19.340109343749578,18.7611176595214\n116779,80.37725509990942,80.16541255917308,76.94091331171273,79.1845098595337,79.72940406161007,42.50724675844898,43.46626259795049,101.72328530646354,93.82698813407532,91.24871682369323,82.30306585526971,80.64486451007309,44.52939970119073,51.816391672280034,177.93198502030853,187.43987141225588,165.8539363591231,146.73555471957565,163.9394428602657,113.50020946349133,107.84341518531424\n116787,8.477193056714919,7.986320887154347,7.841635482300668,7.814843213520615,7.590706338397018,6.250188622074917,5.907643625795977,11.622696412029754,10.59732789845722,10.65464106048973,9.403074128947825,10.10252288973208,9.739311220900795,7.9790925769710395,23.032812225483656,20.844690954430252,19.86515170900974,18.66869582875108,20.968931382429727,17.497194298907452,16.439978043743803\n116791,27.8520525401162,26.739001686982064,26.637407203800706,26.446979888769924,25.51747693200245,16.549430820118445,12.533636369164881,27.91907452095372,26.594172804520067,26.616333410085584,24.24194804795556,23.330144340417938,15.695445289186836,11.0873076729389,54.58447060459672,49.48059374179714,46.86298091034432,43.429983795759085,46.05645694589721,34.82308677649702,28.25177064255109\n116795,16.973061247993,16.36527987474415,16.1626134230355,16.252291428642856,15.8207564691248,9.820622910154711,9.350565593025228,22.77753020097609,20.27259074624987,20.589468037746702,18.158704918048155,18.701610357102965,14.987417363661496,9.14277015480882,44.814150840450026,39.44801125390012,36.743500622080965,33.80355295819911,37.25885589325646,29.900050976752308,21.024204610974238\n116818,87.02775819651029,82.65488063381298,79.9369521432315,78.92263279329747,78.52612086594524,73.52560060035457,45.77996481892724,101.64540957430684,95.55884422374605,94.87643337928256,91.23327558376728,90.84027585261263,97.37863325052133,83.16737415885856,175.7008646642108,169.7132689624772,162.86088839869305,156.05760823638428,163.65185873297577,169.57446190155102,146.20654357730564\n116819,20.845920091326768,19.781124167640648,19.42809476918033,19.318228357721186,18.94116925630377,13.937178831396045,12.06851901107306,43.84904966378278,37.502309875225635,36.66316149158589,33.56838804159854,37.040091022102004,40.48814522279096,25.37546692958458,87.61650645395139,80.45821012488271,75.0034856916011,70.39993006571983,76.11316890080609,73.52251952366102,54.29570086147416\n116822,10.739437377830866,10.704135823402977,10.365157419480372,10.559137733526526,10.543392587746588,7.045221013574228,5.923493106587745,12.200252244962439,11.396193845550382,10.842520316017643,9.603648921118356,8.964059254655973,6.165173477834108,5.510334668150335,23.11841269277056,22.45136192203769,19.723785596631327,17.04178843212654,18.85921023801988,14.694420957744043,12.152811986489732\n116829,2.641203095177124,2.436525479686211,2.2832212118248014,2.3156211890247804,2.3400881113334076,1.4974556446419716,0.9789905047211709,3.753812289589845,3.2391800434064795,3.0771852215321047,2.706234849903243,2.762130258762701,2.7399691674756874,3.0799822140056996,6.805165942660315,7.14223185342363,6.200819175435272,5.554382890710546,6.153048304826428,6.200216244445883,6.124814548808019\n116833,310.8237596305495,294.51013884220066,295.4437406093402,296.3092675671998,297.9046392549166,198.46848819819402,140.91062687243794,334.43796794568044,313.332612089217,300.82553293760384,287.34256581510334,296.4545556906974,233.6117750420582,91.62221581485595,474.41590374007495,419.4013913292193,413.04708477046876,398.1309800660056,410.36088145441255,344.34932603760086,149.11039310448461\n116839,66.31613431776324,65.1627180905015,63.21091380704267,64.171471112711,67.5352288119165,74.30413349394672,52.3507017643819,99.46486414272533,93.52366582056214,90.40824551550385,85.48753960531324,93.13966089762043,115.06891071989494,67.18882950553602,142.85276415022096,138.48490559781027,136.07985788678153,129.60616509746615,139.72977842493606,148.34781638003014,108.68184140801999\n116852,16.748464394049567,15.901879938735538,15.848968369235397,15.817506499036188,15.520047650056174,9.167586753684061,8.009283185474905,16.97851568387529,15.89539372352282,16.060219661075344,15.233816022466172,14.958074002886152,9.487508886735704,7.879046940520486,26.558688905061672,23.60336083652397,21.786569940605435,20.59803679279681,21.675994157640147,17.1068388133563,15.325234655081603\n116859,18.911269342773156,17.752462589151076,17.440458762682564,17.422908729578587,16.922743280287367,11.521038969085346,10.184884165579474,18.915789797993344,17.76805402690764,17.969046410074288,16.281301154418017,16.081970902772845,12.184029784674765,11.140558430184878,31.523180139302383,28.52850279049003,26.584519905398544,24.75240524405659,26.348096064725713,21.776536039079105,18.745210592049435\n116889,4.183513921859136,3.969296959214647,3.9227767638653286,3.945449296154495,3.8418974300575734,3.365277301807484,2.990131174154185,6.34052621543808,5.9853535929762325,6.257160115673395,5.64202449717987,6.074815995563068,6.7429965416855,6.223603400546403,11.544172771654223,11.228065618807609,11.25437197536287,10.408536027755222,11.699696691220431,10.73081988582167,11.311288660514393\n116897,31.43036723127858,30.673617059463663,30.257250389792635,30.221475243587648,29.924892468256896,18.865351306676597,14.52953287027569,30.209523727766374,29.52462568168682,30.018266575795547,28.20272276499558,27.909107591912502,23.24264580328574,18.339915556648165,42.283726630643926,39.73308410903371,38.24209608867106,37.321988460421714,39.70802596839483,35.97228910586671,33.4728503247537\n116934,14.057717272196886,14.382743051297494,13.909452119916219,13.908695605059346,13.738198563078472,7.149764885390438,6.519079693320611,16.898040345445423,14.489422634387243,14.57555023672929,12.813008625552595,12.715710820786397,7.221481268545667,4.917159668038611,30.042687325889677,26.539877599341455,23.174993342786262,21.599235760296644,22.822180088278238,13.975862502049933,8.386129350158066\n116954,6.139098760073844,5.796583602626541,5.772709217765253,5.770602375610509,5.627379099413033,3.1237008401529978,2.379505068129677,6.205065444140398,6.0356480648193545,6.116922805223236,5.474199364151467,5.328500543965826,4.433503326062233,3.9035132893752453,11.69826255538598,10.74168557430432,10.435953046574582,9.711212630665225,10.539401258296264,9.856721609714874,9.42298069908493\n116956,1251.969000267726,1191.861673021185,1136.9482552859051,1112.7613550454935,1070.0678494375009,1107.7263359925173,578.5501214827701,1110.8563929312695,1022.3577896457032,1072.4271256430889,944.5773701116249,957.598550238369,1160.262403787131,582.2205328662433,1682.515218668615,1600.0784642768576,1580.6375879673922,1463.457538560834,1568.4917500486463,1656.6705773051765,1056.7419707663385\n116964,9.455369805285562,9.400007232831745,9.335298997574258,9.276086531370352,9.063922096134492,6.121283393413437,5.7855880101771,12.480978993295427,11.596475082396775,11.68386792053127,11.037397694218729,11.013038782402472,10.707556835527358,11.0045074943422,23.8020469819238,23.61099147182901,21.267946829496427,20.002588663053842,21.420048864144647,22.056088727932824,22.520390601869202\n116971,24.94829960917909,23.861670895417394,23.26150846076117,23.17700575086087,23.58210032011701,17.700229107240844,14.958535594448378,27.315598446698985,24.23831676511036,23.62561999790154,21.262192677291097,20.698006963642058,17.93178217232422,16.72507319311195,44.66559367940378,43.515613992796574,36.05723196205053,33.96533564902649,40.30591079362665,35.18897422803421,29.159861459438513\n116979,121.44820959445667,120.89630112724443,118.2816101319619,117.51767254058359,115.43079052928948,96.95614856043277,84.77935816692035,114.0544134542263,112.4176841748968,114.84158723557363,111.5166314611317,109.25058223702268,94.80432231642291,86.65652054212156,180.57574202836363,179.22934797660864,166.82870259288006,160.08233215402475,164.58867995899206,159.82759977446702,147.02299287007907\n116981,1037.510897302514,1002.1372893469207,970.7294479414516,982.8299991689751,970.3925066441464,611.7844302216149,624.5799709098475,1222.94078330771,1155.1272120461012,1173.0515595368108,1076.7586173407765,1088.759495250839,759.677273565074,760.2247759906664,1835.975341325192,1769.0059544199944,1692.512075793688,1569.7960023737037,1655.7990038247972,976.5442363986598,1000.5814042670545\n116984,6.35921027785773,5.982079751084782,5.982071912231988,5.983549984784169,5.816386776266182,4.21274053159368,2.9693393959548926,10.608305024706212,9.928412966790495,10.33721794085981,9.261195678207422,9.91066469951757,10.744108413973308,10.407340757443647,20.29556425804527,19.550216574728676,19.53681463183088,18.011713295466944,20.399983361937952,18.594104527703553,21.014383778306097\n116986,33.46822339160873,31.777430961002146,31.173664082802777,31.00986195281679,29.799993821244307,19.33854616296367,16.145475895574414,31.026009926308394,29.329539282551846,30.35845215185847,27.834657688613433,27.708470413680832,22.0870437204093,16.00824887201121,61.72795733356542,56.34483008436958,52.157653182242626,49.42706495666963,54.54611184528596,45.290079832750614,35.939906171325354\n116992,647.9012966318026,615.0907019674814,592.509522675965,603.4808492799144,586.174524652336,360.96299204737795,362.4202421870604,827.9700377600667,766.5819261418466,776.9338291021485,705.6276620082339,714.554501335514,549.9829274872515,558.4861958427996,1382.1613071723182,1326.6368622078473,1270.4847230413911,1178.1649962588158,1241.9904755064968,836.7386811086359,852.0934176638268\n117001,157.55742399768474,155.9247312669936,153.81125697999735,150.05575943736653,146.01804426989818,81.93306801513403,83.7382835955025,266.7009226429677,246.94297867748062,255.42446983645547,247.16969438375088,252.75850161230744,118.47224153363972,124.03444155353723,521.3578244491413,486.45617011836845,458.53298651219353,434.02145984666856,464.8914108320269,252.794406840946,255.52825665574446\n117002,181.93670582536578,179.51739057496064,172.8172021625329,177.01038028423147,182.21981715608038,103.67948784423018,104.3619635496819,316.6791049203736,273.29574435853203,278.460491448891,256.2487620253078,267.48905770422255,180.51643124407113,217.17636386866073,607.2564755497665,655.4206230159369,596.28133455136,526.8789655349245,561.8484032193842,471.521496743018,446.8699472522306\n117015,14.271201010847605,13.483789042349999,13.32825459625862,13.28481267623361,12.868438002055605,6.219732989541144,3.318527129378735,14.345782852360788,12.892546116795222,12.94572204869356,11.756450565035188,11.549160868332338,8.10317578714562,4.787575583440167,26.208990908419633,22.27960762587394,20.275471257680145,18.75757015195843,19.96959698452402,15.516843923301249,12.020605869977656\n117032,46.92952549318477,46.68128188796656,45.31361008081086,44.97921559351248,44.94758427237668,26.58739224736021,20.16996527711481,43.81272128318713,42.20901679233956,42.438598848896454,39.91256430067896,39.12429825655204,29.942105618029974,30.16762982932529,61.930721431730326,62.120930957726344,56.89763564521717,53.234976638052615,56.60827003674026,54.53866991173709,55.03424976665564\n117038,67.93987794455525,67.1975811294223,66.69527001425942,66.42985460158235,65.22824042529322,55.61208209496784,54.54074547684478,81.40901982445237,77.00245890461757,78.24402123107139,75.24084536763476,76.11382433990744,74.41696055382472,64.95401807391954,136.85181497146846,131.19398213823655,123.45475008230193,115.98799747601241,121.98019980694586,125.30249251191385,109.18445105471355\n117049,9.32302977939065,8.84747023397528,8.570768356891163,8.46399709680797,8.265938895392928,5.892812769252663,3.9872721611687165,19.01953003475269,17.425348453015506,17.662021676348985,16.418220330413845,18.317400337523857,19.448438054327504,18.077898509125173,38.61165471122534,38.165671982978445,35.819437345272284,33.883136542472855,36.618918149752155,35.6900334472852,35.57731226799924\n117082,30.49801475169055,29.49394044348115,28.324724833262405,28.313429423354528,28.73935474069329,14.97926486090018,7.184698911861462,42.530130592131414,37.214001029991,36.60995405603424,31.670361946480302,31.8358630169017,24.737197578315577,25.08579582314218,88.79386078968233,89.68822425388836,71.90763250654346,64.47788973483244,76.04840256285284,69.28979935303654,62.88013445160294\n117106,9.320820599333345,8.944912212435359,8.587740913866401,8.5090442130737,8.3353223883715,6.800312352772717,5.147521790937818,13.152703331799184,12.099606598121087,12.431943150435975,11.581947101756677,11.804976434609266,11.724205472810109,12.250906888596264,25.236282200050447,25.501083068196667,24.135423580582152,22.693152599959934,23.834498216448218,24.030061330285275,24.02387022088511\n117107,271.3287516575165,273.05350937249034,254.9634356243509,258.6497410067604,259.4114640969136,171.65637921147606,162.75615470965397,501.7715971893745,456.92267873367723,451.98062757500986,431.6531916028841,440.1464119608613,342.006395463167,347.99963838447303,844.3528531359574,816.0807741316878,780.404470502769,723.0690328130231,765.3110090207969,494.66588473242695,498.57766202283057\n117120,8.622748947792925,7.88339957251843,7.67528364758593,7.707024435263307,7.399714437706608,6.213565330885381,4.375030478110414,16.23189605201601,14.760299803152643,15.443552678461648,13.563416084483746,14.720901667428784,16.614182592033593,12.696193960627697,36.257762257974974,34.25992167870195,33.40135275369073,30.65033736766486,34.97143123478174,31.204908825245223,28.538684668530703\n117121,2.0685611749331962,1.9833310535886324,1.9727328921390168,1.9880464920150194,1.937534523481085,1.398259374641125,1.1686972295394495,5.383335080902564,4.941555706773831,4.745225527871713,4.478860469795673,5.2428655563979945,5.810976148035294,5.4035900720825,10.443600484651865,10.197590346789433,9.855245148171687,9.326740492637898,10.409868651579394,10.45958448997857,11.026832372049043\n117133,9.103833048376888,8.777187893374657,8.640367240899167,8.560481823264311,8.438569089732782,5.735077275883799,4.675605937491683,8.433783372245019,7.952780583626651,8.04204154769524,7.587260717098049,7.197038604361378,5.721965520654453,4.407877581961955,12.755350288414776,12.591833211540647,11.43616183426976,10.603832831712603,11.453083631897307,10.245178323821644,7.9118433942070014\n117139,1742.361742537063,1718.6626329021728,1703.5834444115126,1708.9081140593387,1699.6856459682256,1534.2292553604318,1509.118136151847,1719.1597115438776,1690.2324133705524,1674.1226199028315,1603.2819961015311,1616.248926739487,1597.107148131032,1351.0572143399115,2192.5042440861857,2122.0946009953686,2055.8039980974545,1988.917766872237,2021.1054562561208,2024.5380915105518,1587.2051683110087\n117152,914.4069304481358,895.8670178231309,883.2611817023721,883.4904501222921,877.4536222639719,768.5052476184964,694.6395775775978,786.8249679887625,788.6926945397921,789.5874876923369,759.0397614731506,744.9093835792414,676.8192822494279,599.9083084647781,962.0663470665618,942.5375485351141,911.6315463493552,881.9667688380523,902.3401879624798,821.9438420773315,764.18583003262\n117160,5.810760913509952,5.801668878890815,5.740365693187531,5.717672162962663,5.622153505105352,4.026753079833581,3.381949772892891,8.950852809763813,7.65166342982368,7.804864121228777,7.290573336843882,7.3784409077704,8.234699441628308,5.011190401277604,16.020467133196103,15.148143275007087,13.85904548273649,13.084969348046751,13.94657194094487,15.40942609557202,10.503777842854912\n117162,40.93725559500013,38.58803173276933,37.736568328950035,37.93382773997854,37.0800941419662,29.56973552969452,23.386929636562233,51.469098491019516,49.08727811698517,52.23893007152077,46.7548011291797,48.607830043914895,51.01903506095808,46.19659477564171,89.74820175638887,86.33832048160441,86.72754373094573,80.061932188232,90.86381786585696,80.8192158846001,87.35262186865464\n117165,135.00205850181592,123.24203927899555,118.22078406582715,118.42115771509887,113.36159518950318,68.49839012888604,71.8340524124932,339.9374632487164,288.7658529033216,300.36487797203046,270.6442281229648,295.6022503202134,292.898644136345,283.54567914870205,744.6765084902072,662.4289712070444,634.7546234724492,582.1489308075891,642.1123159187176,523.814146381828,546.5363514235307\n117169,13.371201859907895,13.145312728149415,13.095128944534432,13.08197144685005,12.915506630493338,8.374594163262197,5.532598234533653,13.29208377265339,13.120627273457373,13.31552265582154,12.718219478115966,12.64071723680763,9.679294014691154,9.526293269295914,18.51931632084741,17.87130426190925,16.992332138511014,16.39454913030545,16.809140380487342,15.062221726180164,17.148231389483907\n117173,14.773957116283553,13.621291543166901,13.498362079519412,13.517509118729885,13.175308368028668,10.169431303320769,8.525572127471545,24.457827659382406,20.352092552211875,20.120941586933572,18.720021289912395,21.17372831134125,18.87595453042233,13.16934775904176,39.8962474502292,34.41812334626264,34.09082112131881,31.86459708084202,34.50555111248614,26.436445360114227,23.03868087838387\n117177,142.35281335914843,143.8977841910323,136.1828165686483,135.39047330409778,133.96587919962735,124.22790840413393,123.73452657938397,93.28936812721312,91.45675116243672,85.55603352900873,77.4113699277437,69.52925037935582,82.03309952579045,54.367030809538555,119.11132050133334,109.6336190309956,106.00606084672548,104.2418274326249,113.08773863624774,118.27678752863974,81.84483783588706\n117180,14.40210566797993,13.94025385900431,13.618188056481237,13.614526410779817,13.48636522534582,9.134939591972934,9.066604589155357,14.318242115135005,13.679349700866927,13.851362949552588,12.216663510885557,12.265713623143924,9.563728172695667,7.537977267395957,26.192059738838942,23.449392928351216,22.59843241288115,20.953708986357327,23.15600031899776,18.30005220834756,14.630362938478406\n117218,24.00424348693848,22.70856775793246,22.54543078105667,22.645311037978537,22.10858118268015,16.94037111965514,15.052810928545457,36.26142958674239,32.34973558979176,31.872607144834276,30.408792383062547,32.3442547243007,34.88337913936604,25.23793376013306,60.6538138712152,55.90614104432816,54.40082159128892,51.157870399094485,53.51007804332374,55.12978879721309,43.249077556994266\n117229,132.77749216398806,129.37278353106288,127.11949201425705,125.66173657549112,123.06569118150419,90.90558933707695,68.17900916114009,160.5388552123435,141.19359714217532,143.08993414059637,133.85377649048618,136.54232318708898,143.3852487749924,73.67872657178513,241.1958410991062,213.11596762917466,203.26959571418848,191.86916652354162,200.27144145252362,214.91111138281963,110.06447827240424\n117230,35.68308682529819,34.27700004208332,33.36646393720138,33.270089100509644,32.37276772061204,22.061464257542475,19.065984897313765,48.69520778862607,40.99145571022027,40.78425354968986,38.48492372603102,38.30544797545948,32.037361369910556,19.21522502616527,87.90981151566838,74.85935127732165,69.2196820546087,65.2677286535581,70.0904052234558,61.53013165335264,35.41541740515531\n117256,133.2353360902534,131.07058685292344,127.02798008128644,126.11835532105441,123.63022760116728,120.86056833519226,82.16391186123286,110.08033407286563,107.87853413537658,111.01931576501968,106.47335837659514,101.25629418646378,99.59564928619699,62.27792693551841,162.41004427243672,148.66579014699545,138.10706102414926,133.0194833350742,138.57764131509572,146.82828207570137,81.39302016025268\n117266,6.692115846699403,6.456239015376301,6.190153159113597,6.199350297373122,6.047352293506526,5.406399261364796,4.45441505438218,12.284385946680814,10.167451474500199,10.343349637513791,9.193490583779303,9.67967321800625,10.856442679081477,5.895984835243215,23.692326296173345,20.97291424662321,19.86054001123879,18.777540440638827,21.675076363717142,17.84417306908592,13.168841586646385\n117273,111.01689071901454,110.80213039952622,107.29419896731004,108.45346755713852,108.49375776062979,96.35574004362273,93.18805355847479,123.54589090411997,120.61490416887916,117.23140998094478,113.41744928718806,112.71456772971642,121.04868152969189,98.42842208177669,160.99801416025662,155.4023515335687,153.5013572808996,149.509112921294,153.0000683020771,165.8093525579102,127.32529474472045\n117274,6.961657791363315,6.730268072064309,6.407593713065509,6.3012041507208,6.3248600089296705,4.772489146239236,2.668542134732724,9.933437821655234,8.281457154553436,8.530786250833996,6.906368218725048,7.347298574137094,8.035815703723046,4.1371930346217605,20.83841917294033,20.738585984831822,17.476496029279044,15.65021274279392,18.175785539393033,19.655947471099015,10.260219139270555\n117280,97.89263547915763,89.5985217347859,88.49770775173573,88.16479001522578,87.7404331935715,50.13335609305176,53.26281062431379,216.76741622955942,190.3671417388675,186.20478326779,176.28450898160705,197.19783401021618,145.43814649046482,150.09653869175395,384.4508274927915,353.4572249834966,346.0492363038502,326.3024185981291,339.7855055377004,217.19371660520548,226.25487994114113\n117287,8.019754649393667,7.721880127454244,7.5903280849866706,7.592364717424849,7.424586199763086,5.077992632945736,4.949249733766077,9.51934761810934,8.613720507185448,8.719129434648194,7.894563856618164,7.883069048147127,6.251219242714966,5.363696897398878,16.320466684726583,14.533098552048937,13.798766651851249,12.71372125430126,13.753749404031565,10.91483414861673,9.376739078074003\n117310,4.6935639831876035,4.245652756701848,4.223416577541987,4.218184469300533,4.000967384370492,2.9621107627300622,1.785175356512246,10.0913840571161,9.348305614333535,9.409386037741273,8.412703659180043,9.16531957188757,10.143013618460863,9.617466181443522,22.41864255307078,21.82470245085186,21.110784888422607,19.617232423256823,21.720810037362916,20.62729125354789,20.70742686572632\n117332,5.835477548506791,5.730797349508981,5.706252608624246,5.6675240382457055,5.530339242960274,3.3105894446720603,2.446755175487279,5.7741872795948135,5.505208612566572,5.602041967353024,5.287827486522151,5.085871158181208,3.772269656063965,2.886367696895653,9.571517189227494,8.938109744017563,7.8745694202477905,7.430305286550729,7.776399800184121,7.8304944689154246,5.993964178910404\n117357,28.933321946863593,29.36257956821902,29.29618862298681,29.278093111720885,29.127871111486826,11.189128133546346,10.351341898153017,27.9274250999789,27.3014940233995,27.4130456221771,26.538518739407408,26.15110507666037,11.544669916576956,12.86258393092995,38.66364206315234,34.67417825815113,31.964407050358165,31.15323458354535,32.12679436080257,22.46975302055638,28.652802963150897\n117373,5.8839388940377315,5.455476403670401,5.374454286788115,5.283795186760617,5.3431482696963135,3.5827803288864355,2.3730298835801853,6.672394844153781,5.873294406286709,5.600506510756715,5.129746782789787,4.919525551166454,3.7292962925227005,3.6543423291377963,11.337976765597377,10.971286596848653,9.28899565384777,8.218251374339072,9.30339832887699,8.069401438878792,7.598755626346079\n117379,61.12912193603349,62.946409189379445,61.435152217927865,61.521898997901985,62.50060309157884,38.219314316379595,33.67547485394212,78.86587365520717,72.48552790384473,73.63386073097722,65.99904991215577,66.01235681424467,32.107150928840674,35.961580269089644,134.44670067797762,147.59980329316048,132.15341890912686,118.82817538936338,127.41999064601971,75.35605641399813,72.91890477792104\n117384,14.474019187547809,14.613060867100735,14.49649798199756,14.565722393755463,14.581148215975727,7.466222645138442,4.328895107313068,15.154203208913831,15.095732144947108,14.972232564615569,13.894397289236561,13.296235055805628,9.226463111003186,9.647723797446073,22.189243170872043,21.96452058793114,20.161353175217425,18.487646217240847,20.554935223266245,19.09552991315952,20.275072299662906\n117413,33.536080761760616,33.466000129939374,32.451832494288816,32.287633080465945,31.77728371960853,23.890276362672314,23.45498796305625,35.025069122838744,32.76625124212989,33.331558681410236,31.472134985081478,30.860702061091857,26.122446638380058,23.857686456146617,54.352213488183224,51.22210621087738,46.97828988465056,44.35126551395394,47.204381613977226,42.098018402402744,37.9066837806396\n117418,36.21359533027671,34.37835467342696,33.58095549858143,33.56424106016585,32.82107842147265,23.770524856609658,22.443398422291786,35.50296634795952,34.41893849801396,35.721073382910184,31.773186093953832,31.62265427659817,28.714656462425527,26.284942779682456,60.85359684415443,56.980614294045175,54.82827506134183,51.4371914007221,56.00181840718681,47.49612257501991,47.073538567565976\n117429,14.806208252332988,13.519209970557185,13.527859922775573,13.546974851838375,13.181162423006837,7.982482608935978,5.467128103541888,20.203356877635745,18.950699741155653,20.128221149613474,18.265164162275294,18.999133980859924,19.88779532043723,18.29859053621748,35.71034309170155,34.57521326701679,33.9073429814076,31.567109757873734,35.28689518996339,34.93312322525689,37.73949060252393\n117431,295.14664846595974,299.2711315865751,292.939427495878,293.5059273583287,295.6243159244924,190.55785774626682,152.75377923100316,324.29785579880996,287.33504249604636,287.18352498513207,272.0378402742655,271.01636010614493,214.03201258311523,124.21301431480511,447.3411219544247,411.4133965687405,376.72786050398275,359.91912284705785,365.49056543193234,325.2513419448519,185.52158378334966\n117436,38.82979929457598,38.70306723127855,37.90964912456194,37.85369259154336,38.70811493357044,30.65827348162945,30.236977793919827,56.10897310973141,50.96183809047515,49.726045602963666,46.55810032466433,47.88110793184139,52.049187025143794,45.097914558885584,91.24106967333576,91.48274053637225,80.98750004598605,76.4438399736101,84.13267299075083,88.02042011113461,72.46160774190732\n117466,21.76225282814727,21.780394118156696,22.14785365934313,21.405696099097995,24.815583771987686,21.5799416913035,8.503249002092227,60.472678030547655,56.024418369595914,56.448253629269544,56.456238866498616,63.03409685984511,61.969123840493424,45.148476338984054,88.85436796626831,87.39409098400789,90.53025213839052,84.4499505840672,91.98369338423372,81.20703292490464,67.45441668071815\n117467,81.11278972769,82.31721264749838,78.79131124711913,78.26637727446644,76.99149826302717,31.236578998366667,30.579938772962734,104.51685269543424,101.53019591134623,101.37098376716004,93.70908732045687,93.1747149803411,47.66274421504218,51.06053464464804,182.24866739083785,179.63211111025868,167.55166129586505,162.86023137336477,169.7752143984669,106.82880580625638,107.06443034155957\n117473,64.36741159849333,60.62676768273671,60.58668214644832,60.15911274329129,58.43444327873009,44.02133135270635,44.212728010202895,50.743243476250186,49.91074850358169,51.579128071773255,48.292468362721536,46.29240639605072,36.6044489332412,35.50946278371144,74.13865378213416,69.00548812230315,65.18122017821625,61.857521439403456,65.71637628755397,53.467265716467026,50.73436842858158\n117477,49.13336917299614,48.82724227074418,46.92792480964486,46.91979752699476,48.787324095940505,28.363172898525143,26.647497637211004,94.53803135485971,87.84911639624343,86.08057635480671,82.74345621716394,87.50257854547067,64.1895974781065,64.65416362487841,147.78469906807211,145.49767988739137,140.54087981294563,133.85744748456324,140.9899759817187,91.19209338229912,92.32262129873062\n117490,151.62952189433938,150.3422672818954,140.6934761199797,139.99275074931901,143.73848678537695,75.16653828348336,76.1488247426304,186.7777696959936,174.93285055977273,172.3634170827369,157.82722373439495,155.5417587943357,119.19106190760321,125.06026751796293,295.6831934447572,295.58540979135677,273.73554209678167,257.94076154902683,261.3488502558632,195.90186181234407,195.80599891491482\n117504,113.4999236289306,109.41633781867796,108.51131167950484,106.04630725900978,102.1694157016839,52.972760575092074,49.66880101962657,145.96434275185635,133.10573838838735,135.70104761778947,121.30084246319548,122.5003144200125,81.24212440903207,81.85277668500868,235.95068960230208,207.28035021713862,200.05558530358914,189.25593799689494,198.39840966269986,149.14330739198442,155.61109261037743\n117516,175.73090902235145,172.7307849724002,168.50084067667254,167.26099671205264,169.52663900081006,86.16248724699138,89.16522669928706,271.2410233495387,257.0391482415358,255.45468889923538,243.26444165656827,249.65540379079775,155.46345839224745,160.30363285261356,406.3147660855712,389.1220608813594,374.4654044094095,358.24639111945163,372.65096082420126,221.414090111477,226.96034839090066\n117521,28.88783054987176,27.496006946741907,26.983395997065028,26.915687069015167,26.291282644182647,19.411195201515266,17.001356192310762,48.638756309889956,43.11124726560047,43.252093790719215,38.94038226452837,43.2842318512551,43.9787592959535,37.008026846604615,93.67364446871684,86.98832193023748,82.74579599771272,77.99027950678317,86.89736586492444,77.01718415924276,74.39035738726358\n117546,92.30343724095842,92.72450960477495,90.93780006805503,91.19924914374111,92.65649484693527,49.64164026258927,49.21841171511565,147.4484111303846,136.40221116751556,134.49839338921706,126.1286340827661,132.53269806100977,76.16077037302992,79.8835135237742,211.72093079096382,204.85839995244083,191.9733694710251,184.5590012390622,192.90379155863104,106.03997540126429,109.44171655585917\n117554,11.384898923256092,11.307501757428417,11.072674154155642,10.995395265473105,10.882229505622682,5.759416593060097,5.6276480873715915,16.408275559658485,13.690493360094491,14.16200567428264,12.190058515104054,12.971324469748964,10.49305943877992,11.401104895178747,30.140778685383427,28.937408646947265,25.14006157307973,24.068145347205558,25.35938537087943,23.778166070842744,24.951537594329185\n117573,853.627787071826,867.4456574372085,804.8127593181505,808.7421016058383,815.4776665216339,708.3941652694822,734.5515444793592,497.5560476343819,496.6420997967899,481.9633492822951,436.8612026529235,429.11613554679906,401.21502505852413,371.86755252963604,501.0627459058614,476.2020103093657,435.0936141017917,424.1657723331611,442.73256369989133,404.2953206668294,330.4117397204373\n117577,12.202429604947323,11.949630438030658,11.815957231013726,11.832778748269053,11.552301866096245,6.69766339654667,5.9237808262031235,12.422780299639069,12.231184437496207,12.419962190235697,11.542185496269939,11.289301611406657,9.050639247389968,8.81659054770565,20.71075229732521,20.217081860455927,19.345118594409005,18.34060122992851,19.03676115479455,18.204160542078796,17.281037349812234\n117578,238.5310415008598,234.3454926551977,225.86805028828428,224.37918457762652,218.57697860182526,117.24645364637216,115.22918883643423,433.61388658738167,411.64427351908193,407.62183979858634,385.59744570668255,415.58598943402444,303.2624404959961,284.55764423594087,883.7826240562408,866.5346522834382,816.516543123631,778.5316051526527,851.3705259987702,612.8343562156099,633.6033694582231\n117583,130.45301564759632,130.37616782371475,126.34263243090747,126.3596457478751,124.62103399679125,104.50591931769341,108.28408918752892,112.11688605497251,108.97054350164659,109.13597676799108,104.1297503336519,100.08908958591587,92.45711725978066,81.21108165020402,159.80370129880976,153.37020817313848,142.86377664463123,136.68969600747909,141.11860294834102,141.470157585255,114.59574472787772\n117586,1274.1602190165072,1288.020214520566,1269.789063413199,1271.5232430031242,1276.547881128815,1016.5976771613024,974.7655012438781,1276.5707370603263,1244.0571395146046,1252.057652111874,1231.0171417359236,1234.020307253069,1043.0835654079226,956.0364452555202,1447.0978507139723,1405.6247497081322,1373.4176628693992,1374.509402407417,1391.850938936094,1227.1297730206338,1109.9371419027987\n117589,2.6296724851695696,2.51869306830671,2.5321388494145043,2.544916940952338,2.4825397505046474,1.7049785560951425,1.4316227376594413,4.192801292093941,3.6993714661900956,3.744670213762226,3.3864999158191287,3.57977110917494,3.685922732308136,2.826205674146488,7.526002352819704,6.953067383044818,6.801372609819453,6.257403005153728,6.99026166933243,6.6261309772115435,6.003321580459367\n117592,31.86278266722241,30.59735492146288,29.854588425019497,29.91643192850831,29.334592425879713,15.90682501542362,13.647024104871816,30.279225542638997,28.876572210514,29.20870434406607,26.948327442748308,26.767982835685817,18.302249597365023,15.641742364650757,51.70755356551157,48.44135686112021,45.46036649507386,42.70352375757103,45.61657978100562,35.62519738889166,31.961417071178886\n117597,136.64146999302778,128.06964291132223,123.87167026263383,123.25516257326348,125.43702082889928,82.2407889214953,87.32978029281146,257.4033567895318,236.41294761612772,225.53003203840794,208.3990051997788,223.7511070163598,173.65880948564285,181.92518488738838,430.5134273892838,411.8907751039767,392.72377681445676,374.37073869599305,390.84512362391564,247.88323110886557,253.217314264899\n117605,4.1658965560544114,4.025108002191791,3.9134968741381893,3.8278964914297786,3.769534702042885,2.9479310572056185,1.264236270954106,6.4522396476606945,5.237883556651038,5.338975165732328,4.579024959091651,4.821339111291291,6.034989494263721,1.9789944605678953,12.849180249121968,11.808659316010006,10.1822298355932,9.798015795001485,10.238350465852722,12.464375167013591,4.465421055683311\n117623,477.3838383388651,449.60793171323814,437.4014103341522,439.7464969600707,436.56904186936123,280.9099202314589,288.1966277313961,750.4134917554038,693.1783052418982,683.5766279732458,624.951941166778,648.5831815364991,623.6557422358565,657.7602873428807,1105.8030332462076,1057.7108861141246,1019.8517115828613,968.8043618843777,999.6135429661422,863.4256972758615,892.2877642121153\n117640,2.4949435243368225,2.36007543120528,2.297724418959269,2.3127191746009133,2.2643680459650986,1.4310700032619168,1.2041679588768044,3.608354812808575,3.3250758507999136,3.3666432697388133,3.055753069250476,3.131433586271253,3.059496369716385,2.612864755272715,6.953585238030491,6.541031927549651,6.35719530395513,5.961008436331702,6.391150192049081,5.759609461317276,5.4647857862553835\n117657,70.47919114163703,70.09749382527765,69.04900462283251,66.71195603189638,65.5061634323886,35.621154483530624,36.24886702642856,105.48127482215165,99.65671416947562,100.98866012825835,96.38387234830738,99.64177222141714,61.45154863215685,65.65348043427406,186.20185837767568,174.6014239984872,165.2015615900235,157.4856343101144,165.91630233131278,121.275709511323,123.97696847339238\n117661,10.253128128961722,9.918529961764634,9.588403502958052,9.557830848533932,9.311591785522252,4.223068240407269,4.586384860531056,12.27753403187493,10.819032845054064,10.914658449645565,9.72343694552475,9.643148482732409,4.65608826746521,5.10085247727139,22.327816806614422,19.22830439205275,17.841405267688987,16.731825664320052,18.138828517134343,8.892686967729746,11.046859059662054\n117678,13.664929134411846,13.811996668408774,13.158019388629318,13.060791728955332,13.133911181752282,8.227220955718225,7.856737534052111,16.921797494617135,15.324776449678758,15.65630562365487,13.926391754535167,14.361270469093746,12.647447718882361,13.148960595691797,29.05316362965704,29.153306753494675,26.923192044273147,25.02772909895386,27.549483119582046,24.972210825768013,23.66544192449344\n117679,245.3821363652991,241.32506734813265,223.8028526886992,224.4305532356158,224.5894412973247,150.1884563347538,151.37520338142144,333.5493929087778,310.7457407261827,302.00569691978416,279.22489321792887,283.59472603455913,235.1198617156789,241.38309841003615,579.3334115555631,561.4418754574815,526.774553497291,500.1516344694107,519.2254772017792,367.8264571833919,375.002004708345\n117723,43.071341617773555,42.11259306846659,40.83392470231464,40.51448445687511,39.76960773468245,22.02030339381053,18.70088049867591,52.74548405093855,47.72136816123173,48.5171760053069,43.11934468486334,44.41704006661576,35.41026894570362,28.52108569089228,96.55291580401445,87.59329786552497,79.59499211712027,75.17301820272729,83.75365569001345,64.6577852422653,58.89635249534356\n117742,90.70890599388407,87.66962688698396,85.88675695133738,85.5138982210916,84.37229016606041,62.02173793150137,48.594255702259034,120.66564397640705,109.5448182556384,109.11193418634515,103.19654862798646,107.90516159567304,111.42499685440399,76.60566918920418,215.52618159943407,203.8806815172688,190.4476985783634,178.46636445347795,187.9762716922791,196.59078966706372,142.21599845240186\n117749,68.53689300846271,68.5981154782947,66.07312076312738,64.42430690320671,66.58997317104614,30.620613753459224,29.51161360921515,114.77092955148649,99.08480326155151,99.53492955624317,91.39605483915769,96.78031300209982,61.53387734644161,63.97175482712297,211.6393552868333,205.51317163056655,185.65125681491733,174.9126189706219,187.50315688722384,112.58667167017308,111.7259462145314\n117750,826.9428562841326,820.410999302556,808.3115771405725,806.7197240182064,800.4267810891271,500.7894046383712,468.1685027249144,656.228615725508,661.6884421894887,663.6090488565443,643.0256179233861,616.6638033324915,394.2360529500477,379.10432566873277,776.5848036199841,762.6874858262264,724.1410810302142,694.0743506445351,695.6044957956434,541.0870673115821,502.00612488553986\n117751,213.39367721983564,207.29091607716578,202.21695255777627,203.63458828304474,201.91941227053263,131.68979999013715,135.3162854183073,342.4281168201807,315.9582266989175,317.86713496512334,292.1886020577071,311.4397794551567,218.34787424050495,215.7236990798077,582.8171778975441,562.9368693667648,549.0763256869817,506.6418484738668,539.7963697960066,329.043119654212,333.0059044019492\n117775,151.7460568368562,146.43521201165046,139.1727444301559,139.46437414926254,144.83822535579307,71.27598273964847,66.68033694440261,349.42537240058874,305.5640328855915,295.88324025302995,280.78617671334575,303.58654429624244,222.39443474908867,221.38441986337756,680.9850316180531,625.328633230539,585.3528256917344,554.9774033776199,590.7339677898843,349.71985977838324,364.4396233135113\n117777,8.43444078714885,7.959504696239444,7.7489616658356395,7.742322586809687,7.542498599150697,4.194302273497804,3.961627402382663,6.965142656048937,6.568040953851285,6.657458973368631,6.2199270614304965,5.93079664759457,3.8933535492083164,3.2910822096865724,12.258932063631091,10.848642569419084,9.988540647672945,9.45054567910188,9.921974113242653,7.798894488975841,6.9823518719091675\n117783,299.347679414942,293.22866268684396,290.61244930440773,288.70356211905363,281.015485965139,236.1437443327812,241.47788436153655,226.64053383523873,227.5061746104832,235.59913044600066,223.9773756445958,214.46612540843924,178.84814475086597,171.13287291792744,262.5249034201723,247.96346233493685,237.41998964713133,233.2406599808976,238.4295688611805,208.4328571889554,186.30313649572014\n117793,168.53797490414493,162.23636774821125,159.30281447646038,158.97475469250173,157.77276424971694,105.36498545997375,51.423593481029805,175.09548503328702,172.02395593872637,175.44346994763532,170.64849219082055,174.51638868115373,135.78357332847537,85.22772214078559,263.80402453640437,260.13267892753737,246.79579264182917,238.1843964904363,251.40756854138107,215.97824491516843,175.5315475472022\n117796,25.93712930816189,23.442789645852496,22.548967440939595,22.65169267094192,22.524778370438845,20.000518512276695,19.731179008349088,60.637571480806585,54.3834647617941,60.108615613045124,48.26165856286719,54.593840327512304,71.44745838996647,31.10154017758945,125.4913644316207,114.26632357291409,120.80469339865111,111.04725045732073,136.10758140380187,109.45905084543651,88.68157363744638\n117803,32.73556657637831,31.60785912682015,31.11470592593591,30.42830332932188,29.732090405657452,17.630088347612194,18.215152418096288,34.24437561967903,33.37062950025423,34.362135610929236,32.758323030642366,32.19480854722679,18.62907899227776,18.861205531246053,50.482734687185356,47.736108752123215,46.74440737253277,44.875205417241425,46.75661096741422,27.25615062950358,27.259177446305852\n117815,196.3819745297149,193.44288883895246,185.04588491615527,184.49356713649428,178.01648104555244,77.48798822357831,69.49351589716896,326.06404492852425,311.769936368441,313.8520546238878,290.7439249477195,310.0325400754983,143.5387345792871,142.60257605574208,686.5987660420213,675.6833263758483,638.3707758318175,606.6259768638295,640.7731390875764,324.45641296714086,329.37122971308173\n117837,129.30674909021772,123.7590829017622,120.81052003564471,118.78485884713344,116.30201048530718,113.72215230568754,52.52727741429476,141.49574966057722,133.59726775306265,138.29129556352785,130.55850971938324,131.8238176557048,140.77479888895706,99.15472067955724,278.79746680249076,271.91161723066597,253.5679659941094,239.28337592113542,256.19310124983804,266.5521945389363,221.33616978921094\n117858,150.02466351497551,142.08408397359472,139.4944789510157,138.56179559328154,138.44420221789244,95.63767335543649,96.87322178267003,249.96497717257006,227.34680536132853,226.2882036196253,209.4271473536493,223.3889041989769,151.89395940918936,155.3140632510413,434.7990778667855,404.4968586641426,391.2587261643409,367.15687970302537,387.98951133108966,223.2521386930374,234.3417120438661\n117870,185.99516318840602,179.25297955708348,174.48436103591587,173.2149774132113,169.11191620169953,88.1500352304302,64.61780297237655,149.4070046659253,143.50738951979162,147.57022890986616,136.17704041146087,132.54254712152454,94.29740352503296,59.77641202942127,226.56974294558322,207.66006569060187,191.0286404577518,180.30311157002706,188.3557663346065,173.57677313433163,116.26362584963982\n117881,334.170607170126,325.8014973593961,321.7167873834588,323.33746517148876,326.0618561315944,285.1156433667773,305.47552592528297,380.7714213651164,368.6788109251912,357.79415599410953,344.35046093086646,366.16403451797896,364.3398095532895,377.7038328574182,502.0896283955446,491.96916572433616,485.71818754627776,473.23624488880915,488.3402201147921,467.9452663017169,496.4690379849468\n117890,6.141836010541186,6.0404374399221785,5.8600226284244785,5.872298604884214,6.0244695568736795,4.1107095623661865,3.7027492413739815,9.581366001210045,7.648304307686429,7.5095150151875645,6.706214733705382,6.524129074613584,5.162312841289545,3.822552874428552,17.775202473523773,16.516257074763754,13.644968390893778,12.455268776085253,14.656631878317405,11.78939249555905,7.874853239506123\n117922,6.808104130290736,6.494855325695016,6.4214515313262925,6.424301578342101,6.265448608479466,4.618767948613766,3.9326638038316357,9.60002691078516,8.516380374445776,8.611387799023944,7.999210469207794,8.377855185736767,8.761832631777645,6.248272724316987,15.416423921291287,14.002354971324007,13.647603886474197,12.685652963155164,13.78407027135368,13.187888623047717,10.470429197259605\n117923,10.502524903899333,10.358087751450675,10.082077084919792,10.054875449702202,9.907136150605547,4.760046250861809,4.961589628681405,16.615026854544244,13.769457265326762,13.495885803001508,12.529549914951943,12.488608208599473,6.616241266765668,8.057208583968686,29.39585052816166,24.70784474639623,22.8475219064001,21.47699096448607,22.79676926459698,13.104071231653492,16.782456099133015\n117927,7.224997678630441,7.1547122696565975,7.01164084991148,6.986233356417635,7.078150073121031,6.11147083063354,6.196112531734398,7.367438034464377,6.644044219802408,6.476879446156331,5.896670368792932,5.552253938966431,5.2442522761856205,4.974694608639279,11.908374211709342,11.639863777225575,10.283717058257759,9.233736372639347,10.099759365424362,10.132955281367344,7.510038413917275\n117940,30.193912507491127,29.60289014797308,28.91783633360975,28.807641930125122,27.847474436851115,18.294655493558178,16.307425536580762,36.017777273748145,33.62678288383371,34.65451301904657,32.44558536683559,32.321173546332176,29.103705962089577,25.461407150625767,64.46614328991006,63.81680168516807,60.51405607937738,56.84498097374511,63.18074542532606,56.144349434748335,54.58738404989212\n117950,104.04913123276458,100.91434537715922,96.95155119622333,94.89439046992075,93.11310791459626,49.39604484251899,50.49071479557137,139.51529976567164,131.90268580921287,128.19023996274592,121.25475899725723,119.60755154707903,76.77538707639172,80.37880727870021,276.9807012501654,260.2758434263174,242.37152154069818,231.15397346347606,246.8461052887681,154.05631120236637,158.77635264600482\n117967,595.014360663094,573.6290014459663,552.0663384642787,557.9985185930905,568.8944448117002,370.6064028420128,389.14251855779264,715.040839017142,648.6455712011438,642.4706687132843,577.3377459828039,568.4361117714542,414.53905121933667,461.58898963246077,1056.6514847236417,1045.6208033135454,942.975004905887,862.0911010181898,907.1976972532754,698.1750210067443,696.9185326962693\n117982,684.9938544709589,675.9811559077622,634.0772907006234,642.15591278063,670.8827434561128,423.6146997296947,388.9134555610478,1561.417172750331,1347.7977385726447,1334.2772110823755,1173.847247221126,1283.749346237932,979.1414209262139,1024.5389907200934,2601.379212534145,2532.447151582991,2338.357312477503,2221.6948866083544,2544.653103029506,1706.6096820268417,1702.7081142636703\n117992,40.79194037685258,39.24795730011318,38.96057386213916,38.88474590006123,38.55053301646197,21.189615302872433,15.099055525512878,36.74529261283728,36.115325055861405,36.36846987296446,35.465050882091425,35.03372713388118,22.691990455686238,18.362212821500705,43.06553166756584,41.33023108422146,39.0973961250811,38.387887470374096,38.9122579495655,30.74823848418012,29.623619676019192\n117993,7.974663022105724,7.525955861617031,7.485008893427632,7.532244362331124,7.335236553126929,5.189685561469814,4.487754133438356,17.034903143090393,14.5835046334874,14.248904991280998,13.468296660623482,15.5250129936638,16.26284019948251,10.499917049072982,29.60533587929521,27.33570106221639,26.909413975585952,24.85133909192835,26.774970591133926,24.608287325262786,19.67865300414085\n117998,75.73142983937842,73.7478601821163,71.09571267331968,70.61498060338585,67.84880507379634,60.88870916472665,59.712252406173576,74.15797019183061,70.16329136649408,71.84114026083232,62.951870261272404,63.177603863202656,66.74596473592128,53.5611744279947,138.0434832975039,129.1972201238385,124.10938544182862,114.34169541893999,123.24703184035221,124.33879553342089,98.15118048538066\n118008,58.40457285072013,57.86950628134707,56.20178122887654,56.811786575168846,57.56886172854828,41.74174530409018,36.80551109886361,64.05789459336752,56.62380707253402,56.08039478093789,51.90099831005845,50.487296805747384,48.990170341660395,33.60065107374482,106.6512750496079,108.36261669102711,96.31659048333944,85.87697440789363,88.29943338115383,97.05733215007683,60.59366687918394\n118010,14.249992194562923,13.7836647086077,13.600278679598361,13.616236242476063,13.262643817071465,10.862128985318474,11.25939539651146,17.558085571983217,16.915652811605334,17.423682674568777,15.985670450328682,16.457093162569294,17.252443439720217,17.01167484544393,31.542403447091303,30.838288854104448,30.235584081883903,27.982784941939304,30.660294444096426,29.477300496283146,30.365995651885036\n118023,7.043904618825118,6.919881736235784,6.505504086647143,6.416916221603872,6.284452108655598,4.869411795203839,4.085734527913708,9.289258744244599,7.601559302985134,8.065730716921435,6.88502360991011,7.129863114043962,6.6246962950606445,5.2225727463660006,18.090880161285835,17.235897116118213,14.901816947271495,14.333561089177595,15.44383852199423,13.201225200562444,11.676389394416265\n118028,11.11728533482456,11.2520402903148,11.173793587406704,11.18239844744433,11.102414355435178,7.710647468781368,6.811471526182564,10.725804241780494,10.392741775650778,10.33444658135256,9.829546989655253,9.481206699352942,7.188132294521408,5.1812000306032395,15.676679394707612,14.875585470751306,13.4820274849854,12.553473061285489,12.918012578298411,11.700510991756982,6.6996467323046724\n118033,19.518331159754574,18.21016305044038,18.024462408691633,17.888738772418684,17.40895981421909,9.649549215086829,5.346457996287292,16.651377365699503,16.11249028917094,16.32908588481613,15.353649602141605,14.842831254833238,9.192477725514461,6.297665797855146,26.92369450377095,24.061970503471645,21.647322632599504,20.535876313640234,20.987409497954193,16.00350780755874,15.320422074491912\n118034,20.80838815974083,19.523388992637393,18.711556338229297,18.65686444105176,18.31658891676774,11.99549099472449,9.593674678135695,56.44557200643399,47.458294726435305,47.30159591002396,41.45389566505037,47.5526939644203,47.54678241562638,30.13251254506085,117.00727312736112,104.49146756065169,99.56973659760212,94.0504116832515,107.32705673750178,84.15517819640299,75.8923486560948\n118039,28.414694094698426,28.10121675076541,28.096701331033014,28.081425378079484,27.613790306553508,18.25643937134209,15.375916763509876,27.092128104317197,26.534675198868886,26.77358515361008,25.62803477767497,25.04261379616354,19.74298382677609,15.21193790021655,39.20812518007097,36.75833565194979,35.045257146198956,33.42046969843949,34.18576503494992,32.568455155794695,26.552831477138852\n118040,22.915825124927743,22.09040127128062,21.458348216427652,21.45436448895192,21.79103004299268,16.578224224792397,14.436804283171877,21.45163399587986,19.64880330978353,19.544811104771398,18.096884767166305,17.245390345942752,14.02549524951351,11.7377756298365,33.74313874247988,32.66757362349858,26.733103024916335,24.105376345369592,27.58013843983359,22.287695449749123,15.821500138482554\n118046,10.352781657192411,9.573099589452346,9.61865315257704,9.650018911057716,9.36466198984996,6.234065936086852,2.918765575982237,20.79163942912348,17.136606261802545,17.65787819496159,15.605945319262968,16.801488270300638,17.763519823953242,9.444965370595138,39.86487882846473,35.59469555980506,34.48588483386939,31.62630468593792,36.118449435067134,32.49507694168596,24.822428686278055\n118067,114.25937845979792,112.28373006915052,108.43414579922661,106.71801242220036,107.62898114627772,70.15675011837953,66.79933818187295,206.77454793300598,177.93914980519952,177.22264610668657,167.57555340010833,172.5003921061862,138.152479720955,135.1666822960946,351.5263077440747,304.75652875640753,282.02124511886734,269.7231092799934,287.9710563184767,208.562316441922,218.61173055155348\n118085,9.963436114865084,9.630851264125114,9.318478238663493,9.21988484781574,8.916091626357904,7.399342232509038,6.776033245909743,12.896998399584717,11.774043475223552,11.949538406001087,10.93839320375039,11.339173707701143,12.02889488415945,11.955141212202204,22.725747424471848,22.555674579774887,20.809582526826347,20.141098244204905,21.773280119916663,22.23035449865042,21.384868977553406\n118104,60.69990275269649,56.116433146299684,56.89624797883125,56.93791304977311,55.06220051688974,29.306937105315395,19.163512872718144,63.02790106352654,59.750516846623945,62.40688258464201,56.77967536185985,56.495365676871714,43.96390751836837,28.40524386459188,114.51887720273417,105.26125121623235,101.161814117434,94.04467541273338,107.07865454737609,84.0833130078356,76.55906515416962\n118125,18.65845802597259,18.668607389924613,18.269771375011068,18.156744025349976,17.889540850801726,7.074608779403012,5.4091014711552985,18.60931621237926,16.944211642128266,17.936349058015246,16.43777600134728,16.065725080871445,9.554257750837808,7.850194513539333,28.59804801699335,27.619534083749546,24.190995082075393,23.489181317126025,24.211049635584914,20.651179727068776,18.83984374032471\n118127,227.7668147267645,210.38334089755193,203.34791133595587,199.8739508924848,192.92063416729164,180.26181379199673,186.61366781294498,372.9959254349305,332.7221077817539,341.2208881622209,306.30289676619077,324.68867944105705,321.62164688559625,313.2932002624845,685.3702967623319,633.6563217047229,613.3289925099805,568.964607224686,610.8183141231307,507.5675023643187,519.486458084767\n118129,32.85990281980819,32.393187750682955,31.64478977226787,31.552768699231166,30.983234635787646,14.362370786960351,11.32686452960172,43.302291466000085,39.99221345718474,41.124719315677005,38.637769431021376,38.94241895287499,38.37680775563535,38.175190776256635,72.60920478732437,69.79874817778605,65.9649064402483,63.02145522498531,68.06762383453453,66.02196567806213,69.92070811638683\n118146,10.890142097769548,10.427493827764458,10.069100140379481,10.000838393788044,9.801797259009136,3.49107874982988,3.5190746100391803,9.136644738018033,9.102753973977393,9.433691884189853,8.80303492720881,8.585454782494164,5.079740630562518,5.306314176620258,14.557040673007007,14.101755520411606,12.851880552491048,12.48291360639746,12.82695971737578,10.418731528868843,11.046535379128551\n118156,26.316059550390335,25.460867231319032,25.452940892684115,25.49730204349163,24.969547660342275,17.585121236988872,15.796491492422353,32.036356297261634,29.04429635915943,29.13657457705933,26.193130995136286,26.04854453078893,20.682808219334056,13.057245799067738,63.13637206009106,54.34253637588417,50.485887418949346,46.617623808016496,50.45414828525818,41.599022517369306,30.054287984554666\n118158,147.1529484649872,138.17269061390556,129.8647479848893,132.015583979053,134.10659216023853,95.66555079436817,103.20478405609781,220.53380513000818,191.1786717742272,190.87313031757026,171.7172364123492,178.29174545402284,154.3754669990936,175.77811079927545,367.4807241980958,379.39244178236055,347.95480990639805,316.2556595989467,326.0521304639142,278.2718721949689,275.76669409751463\n118163,4.38109188994912,4.174958893688225,4.141903383132524,4.125952142479311,3.9651395735295356,2.7988723113500273,2.6406712827084737,9.591235648717392,8.091005840599069,7.746501687602434,6.944598233833492,7.795202986896206,6.698266727344245,3.68515521285526,19.15924470585135,16.983978241177564,15.759748379840778,14.806809882594367,15.940590850463588,13.571073829468176,8.368943464534603\n118175,4.511188592241858,4.461299431407431,4.373458681908351,4.368937328450154,4.25849263650034,3.0612708682298186,2.500921244962324,8.022346243811555,7.284685342088613,7.31744630351165,6.900024909621578,7.1690272620372015,6.967067294444811,6.58243820873666,14.049154987788063,13.993733950977083,13.150243311846403,12.388491786374814,13.364092662770085,12.387156422883786,12.162646147012365\n118220,37.889994265874286,34.37480749299814,33.671385478247004,33.27508921247608,32.37104262682529,14.528335665382924,10.467791784615128,36.535682371869555,35.9149749002061,37.3534077582372,34.09201191297798,34.301108226479066,26.879137347913705,24.352235986914284,62.31563615655256,60.493570857066715,58.86024385336981,54.92712046855193,59.762584748481885,51.2405051140937,50.770744867953546\n118247,10.513362670181674,10.0736962267021,9.833298892116975,9.840809435361047,9.631981036818342,5.401454146423571,5.7747362762047665,17.160981643593104,15.887544038515943,16.417729808714256,15.218632377143715,15.948869567070723,17.332710615320728,17.54299291986974,30.590817214724794,29.928500365979165,29.032747240099543,27.042860949464124,29.140145228484666,29.15945505713776,29.99927220434015\n118256,631.2661174706816,607.8266529547909,608.0583612833669,612.1634341036628,593.8502535951346,435.33868174610006,440.8190835491468,557.6831468189083,547.4412186336115,548.3393571482873,520.6485733858475,512.8954459039813,485.99457058166087,377.0135352031426,806.1556223806099,761.9210590139347,742.2882866220656,701.5577293699712,737.3370302119952,745.7387868262578,542.214177846644\n118261,44.01792686505339,42.58250923155445,41.963065819225505,42.02749496566091,41.2205839616191,20.654481119639346,19.35955764216729,42.74692375778701,40.30296980214069,41.18408025333605,37.59002521005525,37.49617911463442,25.86939071031043,22.513431546533724,62.686522589319104,56.000468731171836,53.31741181113451,50.71503896424733,54.31779794387632,42.467552124694585,38.579620523150545\n118274,4.50918664358616,4.1078446983451276,4.035701403462152,4.0619424502653345,3.8910432562984116,2.7962671351012456,1.9132330812465685,8.858816312062064,7.6515035801459,7.791159874687323,6.934817408322085,7.627508268805984,7.8961969987949745,5.9069538197534825,17.98281471011589,16.369447894195602,16.056447086876176,14.765339230468623,16.558841598372926,14.184262413731508,13.209845308114007\n118282,7.387601688514288,7.039585714176724,6.732227058670673,6.710697332366962,6.490924909171317,3.5572543469102027,2.6527878446032442,12.667175147623967,10.706783970913229,11.248750262973308,9.962224284112802,11.309594751795649,12.387494168646656,11.329115125629727,24.85233758518281,24.706397361026543,22.346155049678256,21.260286984828944,24.09858379143214,24.78547889548514,23.47505494673379\n118293,7.178993113492434,6.736033448859715,6.5268101593740955,6.540324004325904,6.351938432293334,4.367638050792605,3.323014641843063,10.7665881833313,9.727588613577483,9.968239538999926,9.411186259389405,9.809262844021283,10.650285636292455,10.269261279187319,18.476390310864883,17.87158706186436,16.95701123520392,16.16609987617511,17.705533473628662,17.3068307979212,17.96362534490982\n118307,3.1276209530443024,3.021478899953227,2.965475946548878,2.9451957772719353,2.8898147621918815,0.9648847741374804,0.9054407768506617,6.826947536314969,5.362160535781384,5.39948355966316,4.760371741572975,4.928043868906454,3.854120420593408,4.432063996795857,13.493067267232885,11.855337548375473,10.423739127503381,9.74944394828304,10.893766466944488,8.738421843059346,10.383073090694273\n118329,3.729129187798621,3.231201530100202,3.0402183829619527,3.0524031973556003,2.945358778904039,2.5659219812777643,1.3299479286419418,11.472206803362273,9.765023084961902,9.752772134418436,8.77183655760944,10.511405828214256,11.336897097502565,8.924608823825128,27.348935528721427,26.02515440803637,23.885320456736043,22.534736965463495,24.941345850624742,22.41373885905114,20.29047753039427\n118337,730.7551331013209,695.1742340006149,678.8379555886523,683.7561479558848,696.826053577081,431.02317357996105,343.91700919663595,937.4789754697065,827.7804310725033,776.2127000705535,731.276158997893,756.7627212621848,551.3399231873453,309.455593633406,1375.6296609898095,1179.3800798560026,1134.6248153851764,1098.1990809526624,1122.8400502674685,886.0510187892056,630.5448728992259\n118340,120.73833413622944,116.73878123788069,111.51313959502863,110.73266923811926,111.05365804034172,59.426505719209054,55.88142722597882,210.89405913655003,188.98324909376132,184.37077299625946,172.39526767698788,175.1126981161899,124.55145668746441,126.68041079443762,378.8147823635085,358.02651341711373,336.9215201044828,316.6102534559126,331.7707402429876,188.8695615486816,193.19668655120338\n118345,167.79689831532522,168.33117802139313,158.87794921381976,157.61677866911884,157.74856020393307,106.42258838447742,103.35561944134885,235.04802386017562,204.69272528219355,206.25272571615642,183.05887866690193,190.0385707090168,167.5684544723909,167.15228534207196,395.61809004873345,387.6110835488356,354.7924428926079,343.40987375917274,361.75778650295706,280.72102137202666,278.98206732574226\n118346,8.922882533663179,8.209594650779149,7.914800967749018,7.919596815074126,7.703133133140442,5.489266691057723,3.1607781186492354,14.159464804606582,12.686251796537771,12.453392429477182,11.474116948828472,12.050022222208328,13.006117434431504,6.6725605357029885,24.381341944738445,22.546172903144136,21.80787093099621,20.508542556705347,21.5792351797173,20.84158883862759,14.182609082806161\n118353,23.405737805734407,22.482839760986977,22.281680286366782,22.268220957540468,21.976628268686504,10.703880925460384,11.735697443385984,23.210741779086067,22.352250543909193,22.882735922717025,21.206837119617305,20.6104919334639,13.32659238133013,14.164630463670822,35.97779759229625,32.77118408903713,31.64614201040026,30.389771754870036,32.72346377384945,24.769849289750272,28.373334630809442\n118354,136.15048427784353,138.35808446263502,131.3315925121511,130.6258684140269,133.10786506837402,78.34843304611697,70.79718063295111,202.93269757949764,186.54627746546896,184.35837404991997,169.77848875656193,178.0038931411436,112.9171787469966,115.83213925645438,343.04962820898095,334.746899598659,307.5053807538123,290.2092133309798,307.3693522745297,167.1583826443982,172.5770493514181\n118356,14.437636599366991,13.85741574248791,13.358409070678235,13.332252534619052,13.066564468116626,8.168150929431455,5.155875005313282,27.502101920375598,25.55358592479343,25.988446990458893,23.972471358480234,26.444443001877644,29.67569580017392,28.222029576295625,57.13534207195361,56.16668968509754,53.018604146288936,50.34366068809365,54.6998036882615,55.76059345547411,58.12030333100724\n118379,101.45575369557082,100.32179993333564,98.36321351040763,98.48085699250551,99.41535040364035,84.26386639607796,85.47596773049519,147.7027244313428,139.62845394498763,138.9446068999688,127.16800074016066,135.35110318584097,147.19559687243856,142.76702190771863,236.32380553434086,230.53312324526192,225.72732137604964,212.13259941758156,236.64461042739168,223.9015444043959,228.7101348607533\n118380,9.434748201248098,9.270264399027145,9.309487774304756,9.309766390908276,9.692469356678284,9.73264536141561,8.441422109786163,18.741713069746382,17.443985914575244,16.68503984433011,16.16309026331613,17.62002311067732,19.876601391637926,16.53788175020073,28.269014373732208,27.69654831497921,27.50471857319084,26.810830496736497,28.130634438730002,27.99113237443176,26.058133401487144\n118381,14.28101039678488,14.15457104703203,13.773579149705974,13.757837115849643,13.52020709642658,6.4249315399873455,4.046133228274282,20.675641926551222,16.806765995579255,17.53684381455486,14.663259534188793,14.931112422622805,11.74110822300004,7.423518348569562,36.460613424933754,32.05812202586623,27.96848577983069,26.458838284981955,28.867671401028964,23.955835386876622,17.826642283206862\n118421,1043.8360782917732,1019.8993304308253,1014.038149844654,1012.2320214812551,1010.3033867390942,997.3867298179973,822.4244969945569,1161.245897715474,1134.5481718786418,1131.083984796966,1084.3297002968213,1096.6328848059989,1172.2519284856637,912.3334566855285,1674.332010553145,1595.327245487125,1597.6977816840217,1531.7458471467273,1615.9989409513075,1585.5084245441913,1305.7017747871855\n118434,7.727791117011117,7.259284940122926,7.126093714455899,7.100459837358614,6.912497468534943,4.565043985429376,3.962724191356461,9.989662481040344,9.559417041610937,9.686082637404228,8.949201493005855,9.356910937126724,10.15484029788952,9.87154449423003,17.926992672465513,17.56901245068602,17.471877797238136,16.21073491318246,17.76337716791876,17.31699807171189,17.81310124239761\n118449,44.63687201366337,42.46697658493448,41.17569891118352,41.17547549634111,40.09467299084199,35.82478934578106,31.08715717253087,54.955198663589506,50.01701131400829,50.874459181912655,46.25633336125432,47.5163168617329,54.85358717818339,31.949396566572883,89.1598913606197,82.26072543168351,81.17176374727568,75.7113103915742,81.82873851987566,81.06521877804123,56.5470181605474\n118454,1.3400803110500523,1.3151793259425368,1.243550754329949,1.1995832099677801,1.1887937672430016,1.006693290908686,0.609199088320993,2.6811704914751777,2.12756029614288,2.2451054582417136,1.8878176779311961,2.000760933060139,2.2213177762370004,1.4554266711225798,5.712888611278384,5.569089534404126,4.822827007137493,4.524187470526312,4.887992288730416,4.805195764679057,3.4387299263512277\n118459,32.303745549556524,31.784616934755412,29.993487175849918,29.722138999302548,29.788173190695744,23.65461372866875,18.775045998456992,46.87821861482806,38.538583083054064,40.39466345216756,34.73340431121135,35.45160816401654,31.66512391132602,20.575606459450984,88.52927566918218,83.17860159787828,73.16721429746259,65.97444451226379,75.01570008999217,63.850412415974944,41.65868424257078\n118464,17.894592512466573,17.35130784356051,17.090695838325093,17.092862027752908,16.810202403030424,9.592085080275355,9.06745080966424,21.929151889229182,20.4328163420888,20.960116605224062,19.205748954260585,19.610957439089834,18.46656115380756,16.38718436969743,36.61833304268974,34.239322974870056,33.373363317187625,30.936019563540505,33.96030088536428,30.968063656827155,30.15319622447917\n118472,1.5527404004788994,1.5017457387078428,1.4815775426808975,1.4787172428258124,1.4338301674064222,0.990316939388496,0.7631278836125147,3.3531063535824632,2.9552465397413132,3.0248847931454637,2.7685906615623748,2.977669764636368,3.3130558610269554,3.2023912042251723,6.242432135352482,6.225905518868223,5.803506580188819,5.432192459950693,6.11135370694957,5.917848238675492,6.188930544366087\n118482,54.28461353582747,51.997291583741315,50.65800365294818,50.003563598518184,49.48856665733889,47.411640815657954,44.23452505397583,69.39474108652232,62.40227450312127,62.82391287857834,59.45622113591991,58.70307312049303,67.61511143445333,43.69222820205557,132.80767555654896,126.72441355608707,116.81109950488054,110.09164612838107,118.15950351386121,124.11552322562245,92.40417449170023\n118484,10.209373869588125,9.946368556028238,9.6254155574899,9.611849343369938,9.382344911238022,5.244894001844115,3.1403959072352414,12.433721682502824,10.678469160499706,10.905101493953019,10.085136681097474,9.852194511476858,7.6633620923046735,3.6878693492799615,21.26124450369984,19.797511389450964,17.552315908063935,16.57985544056533,17.75885200999626,16.17475484321962,8.988979556693835\n118485,6.3220463142018914,6.0856989694398385,5.90473511271449,5.903381877862935,6.007971495320642,4.437398550655537,3.679754112768271,9.932096341950876,8.097177367294874,7.605555833249432,6.918648833051127,7.042190113261864,6.152361158572752,5.01022116566833,17.788336321383262,16.931753490694536,14.091321857054595,12.869650303911806,14.839288098976098,13.144740222684785,9.27571131812077\n118514,21.56928080474703,21.504960163938517,21.23562227145729,21.2292423595955,20.831948222187453,11.59312151855166,11.298671917514364,25.077105126552592,23.943982927002374,24.3027769787252,22.979035589716304,23.269091276741353,21.694438009103205,22.029737827584395,39.65651603793006,38.697157652524744,36.86562732420789,34.957558879820304,37.21314435390843,37.33077765766986,38.37053852110434\n118517,12.058062031534792,11.198277739443101,11.13992464798192,11.149718238808354,10.832844775974147,7.910919139025051,5.805628495529553,18.691912719023545,16.577214902586988,16.181627442846192,15.252467360886781,15.547992125282544,15.91648460786656,7.832194420454318,35.8687442304282,32.1782409695212,31.018300594415543,29.232543201986573,30.232557866991637,29.76570821321032,15.664305412631927\n118525,4.685336566766031,4.494046026120473,4.378656584853783,4.31207101695973,4.241229138754576,2.5095277920060823,1.6610156756568568,4.432625444497838,4.258073119570945,4.38279884612328,4.085818962173856,3.9237478374155983,2.926851857883294,2.9288260588148396,6.731712292791162,6.779960910728058,6.104147255767718,5.672779391563639,5.709596677808725,5.6323783921948465,5.858849944081676\n118526,4.511406906852011,4.208043894362183,4.090858803882818,4.084651675697239,3.9746564665330597,4.261468869969461,1.9392262527152753,7.456434573558018,7.121373688834421,7.150155157647574,6.48893044820497,6.845640419877825,7.642279810627258,6.797306020041192,14.1723830596945,13.869481710488243,13.697822697691088,12.921646037456252,13.763096320108444,13.287087333204155,12.840553724953601\n118533,671.6627557957743,677.8574265711567,647.1341672831335,637.2851546798797,637.6107614033056,426.29017890049556,421.81990237685415,926.684801549218,870.2470854787498,858.3635229163186,812.0418691983216,822.8382613672686,654.7317944053548,635.6837963577071,1509.914420126418,1419.393931313819,1354.7408317371785,1298.928259596292,1363.133693346595,1026.385039531619,1045.5040356801928\n118554,4.670076488279481,4.3795543041689,4.227058517261208,4.196112639689579,4.0717407866430495,3.231650223677933,1.6002345856476412,7.6420731953177725,6.697776066013241,6.533273251035326,5.901472021376749,6.2430492473830075,6.527397194770961,3.6307138672531316,15.205675797987482,13.834023421227569,12.889579537728116,12.146367960394654,12.910060733800242,12.113237699758047,8.489475316477218\n118576,14.95786219101244,14.563906093635996,14.339926799836503,14.283244848858455,14.04020425382087,9.824208880491975,8.942700817863466,17.793177408484066,16.38665032879129,16.314368229041698,15.313654270943722,15.32783696111668,14.469055471777839,10.505343978471902,29.178462556443318,26.576994581758726,25.224090698723437,23.701300119606294,24.575423603302927,23.996425452353723,17.630640705981342\n118579,149.69950386231602,146.2486994846226,143.64814801428702,141.4807272257783,144.91424719124282,89.44797404207704,92.69329494227155,226.79917138737704,211.12603804666475,211.15159744818868,188.89791299176323,188.38369528909843,151.34646504322845,161.99502950762712,408.2130487930019,413.90606936312855,379.16513123346414,345.9380590451027,355.80786204332543,312.5534371552359,304.11404010513496\n118591,5.213939947061694,4.8711991327467405,4.791514907027966,4.783192280271394,4.572979495285174,3.7968890591801308,2.8378665476520997,9.527482987629057,8.846547590240906,8.952225324860597,8.071111057394877,9.02667026866546,9.60501136486858,8.744417772395582,19.690486657455246,19.261224527596802,18.196198900888607,17.244649010222673,18.818762951443166,18.390351184014012,18.18987224923141\n118594,142.12672653959498,134.88286018016157,131.4940457927948,130.06365273743822,125.76676569710662,101.23931423855214,64.09655565231324,115.82493133276596,113.3250138479362,115.57090718504949,109.58177278795662,106.09483690370557,90.2939363196062,54.65954542186857,198.7581621474097,186.32073841571443,163.41568794790066,158.40341166916977,163.74917752667673,160.2269234397889,94.31058799175592\n118596,6.424216563555462,6.347586887021973,6.251764993329302,6.22332501506965,6.126670257962927,3.5653580327868997,2.816839126105131,8.405120938020364,7.172664188625804,7.620303140873353,6.851695769926364,6.818274873987631,5.38260416536205,4.532689071311397,15.737169961572786,14.91733055262965,13.210560143463871,12.46486753115953,13.401917819207874,12.593473611545985,10.546725981923787\n118604,4.494465230310039,4.276335029175627,4.226299559673768,4.205987430373311,4.126915600916364,2.926244756076324,2.0278229947277664,6.883722872067687,6.126196054978415,5.977802140632507,5.545904285657136,5.661711986251392,5.947659915768896,3.934153697374986,12.09075484562201,10.932581209079888,10.4687436086555,9.824377608098796,10.311503776471506,10.37435575766376,7.652855206508513\n118610,21.992029218150112,20.7933226084534,20.22932274381828,20.08112433302976,19.607320090302398,14.653669816986108,11.603005670825086,41.29707568884914,34.766149777505085,33.91726657556022,30.099627534863473,32.71428348798648,31.898553359266945,17.449118433753235,84.93544833727337,74.73278712054945,68.17293053967109,63.71717932391046,69.79222532165802,59.35058284377328,39.358659422628754\n118619,132.9819701111523,125.37625042747163,123.34931575413344,122.58745515480638,121.8873440418214,71.14824254204059,72.7676586331757,211.9387999483783,190.47389146732954,187.16377279797447,169.88125556417177,179.44608310992342,129.55079843886776,133.97033651770585,353.44647354834154,330.39314649632206,316.07068090591684,294.5023467939278,308.418496698651,180.7876659367501,188.49327580723488\n118636,13.4981966689836,12.937546613697656,12.577208551564338,12.558916113248502,12.327091487371833,6.833333672665924,5.709441548783867,13.373228470171334,12.433083473723942,12.61991597294323,11.574525161432167,11.343024033311703,7.312891751485564,5.546467103897224,23.233311533259492,20.745489626706735,19.145661895602824,18.25575026458174,19.120731217703124,13.501976873640162,10.950532501711091\n118642,41.89551788071032,39.44856016141837,38.49887136333762,38.25533571179344,37.214713326208894,27.26862857616842,24.94956918411013,46.34122033659699,42.37843696263384,43.13383935184832,39.40962136632124,39.778742536566305,38.09423188460018,31.10503128612404,77.08454074961226,72.01215781534816,68.3438150494673,63.391553130317995,68.139662663273,64.8454850031547,53.52134414909348\n118648,182.86914380980988,178.54473625859453,173.8907112185457,172.5018103076603,170.573401072324,158.19345990509217,30.733633022497976,215.84131003572125,201.2316312117744,204.92150713899613,195.1707571813204,193.7447539169296,197.78896706995087,94.91968036402821,358.7506349453833,362.8165598044385,343.9182708767966,320.1560504103948,342.063914705733,348.23497024480355,212.41626067388157\n118653,10.022828690421411,9.412796901415954,9.53928384323134,9.557454890003276,9.292222926690133,5.791389307969191,4.3763621068964556,16.393252691502475,14.589556243545609,14.30003549458497,13.348762836917789,15.065236971426156,15.928154863543432,12.376734244993317,29.30903540404405,27.29581123053041,26.99892468210167,25.09395813954861,27.47119332673294,26.348004240756605,24.584552453922765\n118657,2.134228079138135,2.055870663368442,2.0173999712999455,1.991618941280663,1.9835766555038459,0.9457018743564022,0.550695146097477,3.2769304497220606,2.6028748055220317,2.567136392794111,2.3492631936753465,2.3607028521343496,1.645451017710685,1.1872513670658622,4.904670386351477,4.285988384132043,3.8554595263778766,3.610942687446095,3.965738038256549,3.2708036913497787,2.492594714840685\n118661,14.267707507877214,13.547811868122666,12.93098999800892,12.82089517967927,12.558494214477147,5.738143235860178,4.630099371577631,17.194265895470632,15.597777879054947,16.079240211996282,14.405601923929725,14.945461968242943,13.49665159421234,13.531874946736684,32.54647795723649,32.120057652529155,28.863383325618432,27.5309988636195,29.993845257701512,29.66595774816371,28.19982276507471\n118666,11.10788966467656,10.889644861744012,10.518801121737042,10.479200681804292,10.624773674260052,7.355946702438534,7.472498246953175,11.953699909804321,10.27113751585795,10.561111942589767,9.62656041978652,9.081552497883248,6.32345721442395,7.027117444257393,19.549129666229124,18.834005654798943,15.378566247618473,13.83851822827252,16.088720439624197,10.794478381708549,11.865668750384028\n118676,242.53914533981705,223.71066575509144,217.53567292813096,216.85220111967735,215.51149243585658,140.87979067607836,146.74091124462555,513.6589154364084,452.59619447457624,464.95115985049716,420.6291012336324,459.649599596706,418.93299491115414,416.66442749657904,943.6463260622802,896.2138810305183,875.1000250242951,791.6431585060284,856.7331280346356,672.0968091567381,686.8461576925289\n118698,11.530709451638439,11.248658796875796,10.995966103846909,10.871014547779874,10.73511542735723,6.837305208635435,5.651298201640798,20.123234574111795,16.97642784798492,18.427152269729586,15.171078594002276,15.076108079682156,14.730478515884357,12.599432659962451,42.44314552460887,41.4381552578664,36.73583398636746,33.72229110041996,38.22304944892272,35.1984138792441,28.857959216574418\n118709,184.21546230553488,182.19416094881163,176.5859752441572,180.2899329937365,184.6906666030419,73.0662318905947,69.4947055666097,225.07953063314108,198.07015618109824,194.6038504857518,177.96988761044486,172.30620018647022,96.91482290441876,107.44004256232003,348.8188756438419,309.87142593662315,270.2803719506717,245.24820414253563,262.6663327565217,186.96352397039612,189.76807423985665\n118718,23.31159177968575,22.12147833169218,21.80107614067231,21.76643874347619,21.29688520673945,15.824091223189889,14.662377542494474,20.48051807776767,20.52432534485126,21.09476707952269,19.538560323859308,19.06312031466214,17.548551301374122,17.329978190023727,30.500690748209973,29.652616613038848,28.728608853697775,26.93530735120854,28.872102388861343,27.93260977967015,28.85154510178108\n118727,114.75235164919832,114.99163011848222,110.60330051722867,110.46549636262121,111.84520212778737,80.56901080306196,81.88303546705578,213.18273910612857,194.9968546456092,190.36896817201836,181.1311190201787,188.79732066645877,155.6575446788431,159.59391402704844,329.70569984243514,315.7796500907831,313.6425897384006,295.33978944149686,308.38454223322566,222.65584366137307,230.21399800559672\n118733,13.180206232632283,12.73824706590499,12.614487307828671,12.60849485616981,12.416389693689384,6.249764233301858,4.563263318342647,18.690152761562597,15.730246423805182,15.534318959053506,14.22841444416608,14.510346838854765,12.534101660119246,7.311122910033716,31.831198718561037,26.526608454012532,25.102895366472048,23.9073148444492,25.67195367300141,22.81510411865223,16.334032293971237\n118736,7.519766142362444,7.180104169298686,6.973403979937679,6.910104879581686,6.693970329544533,5.038422251237391,2.5510544644004036,8.333348577510094,7.529394263884446,7.707142695116575,7.081097713134121,6.877072277919783,6.736260502773609,4.349052727811104,16.311721974829215,15.433731251885343,13.823758594274967,12.91746979039619,14.206769992530754,14.21664013177765,10.658902180597362\n118739,14.862511674973648,14.523447675607372,14.082559327990914,14.060508725440705,13.713308146353082,9.10739493632954,7.615881861604232,23.077325774968866,20.314857933919367,20.697453878370226,19.21814704996773,19.782800526066477,20.128829217162725,17.64118729110495,41.780493394318775,38.89990038538842,36.42921516246893,34.67987264460772,38.33762299326074,35.04620058673411,34.493164459028925\n118740,3.7869438801198276,3.508181195608973,3.403426011053373,3.442005258117818,3.344455978208845,2.9429185653240375,3.005385664024558,4.556087864254555,4.120712165919234,4.3767075847512595,3.897058154688694,4.053024019323342,4.876036821372776,2.3546494667606255,7.437751789331304,6.811144301850625,6.6793101169789475,6.317882591165782,6.96435996862861,7.076476658286747,4.250310649246074\n118770,142.8931884474889,142.16126715453476,142.92165847238576,143.37825131077076,147.42072090340642,78.27976509355676,77.85312443737583,198.1526176147586,174.98916558442707,169.99966794058838,160.23310935635303,152.78683707641005,82.17566412035724,95.37489035585016,342.5017704967046,345.8788565911808,317.09380644045496,294.7307451910454,311.69068618315794,159.80061029845282,152.66166530753068\n118818,272.84409555558045,270.7117938558776,267.01237206237244,267.0621642959375,269.19568690482714,243.47499946271066,206.68200790682874,295.34255615560426,283.06777300959095,276.502883581425,270.31213787367227,270.48249595518615,263.2027936402405,227.31428853127872,363.6420086921694,351.0010795351877,342.0498138472002,338.329823766935,345.93995728864707,340.7270413931435,285.8393075775248\n118820,6.699253009422521,6.202245372544617,6.075681680510764,6.031723321704288,6.070324334210938,4.684628495294264,2.0818418599669397,8.396115808039328,7.083930452489004,6.658877694830803,6.098954811955299,5.942892940301026,5.438748385059797,4.05909906727081,14.056735645299057,13.327902630325772,11.497922345498074,10.568007234552143,11.975763567336699,11.529733219716563,8.85464887928436\n118821,4.411763456642281,4.074872826054587,4.056708385511508,4.089101350087979,3.8974592873728633,3.225986038746516,3.3814192431253707,7.757853578963293,6.740914679607834,6.788876896105838,6.067697690967631,6.53575655002749,7.241972164584436,4.875060320150447,15.750122560821803,14.36094761896233,13.92851268581825,12.7972183071627,14.268734991625575,13.003887213414968,10.414916089354298\n118829,26.070905304390237,24.65834776962681,24.89909982153823,24.977985558496115,24.291305866612877,18.000997605838464,16.121281462178995,32.288323756781914,31.04769393495736,32.58088803061357,29.744329109499645,30.035807731734952,30.53317578076499,27.497989823652624,64.03648087766936,61.658655566781775,60.22881519007737,55.64166365119511,62.49542112897773,59.66975012881245,62.71983672775958\n118839,45.76898604161872,42.6197391293264,40.88626672080846,40.77951452544037,41.68591062807178,23.313348611059617,17.77898419264116,47.51648806923507,43.52282012059614,43.12711990520137,37.861208914301535,37.07111491438985,30.913861532511046,35.728689565305444,84.98984211348223,88.65266390787625,74.73228616024542,66.53382519398181,77.19072838489782,74.47245296537959,74.57492177516934\n118845,36.152745519037154,34.78915718048084,34.35850448512603,34.540456038069145,34.84351620866487,23.46057239355535,16.63545575219537,29.18598082294339,28.549291522966968,28.472177890786792,26.866422941736616,26.05856478225702,16.406491631112967,12.15639391010643,37.749067402757134,37.41430324241319,35.13064025508535,32.50712272596798,35.2291290823758,27.64058467044232,21.580586385187388\n118846,68.199712384649,67.9949634179514,67.04573657931358,66.8752836423233,68.92209500567394,42.48266591776069,40.41223589127181,96.4477766769876,83.87833076986551,84.938685604397,76.48520402611759,73.54323766540068,60.41472980871198,68.82757646373234,158.23862069208707,157.5962833867643,142.8896770821492,126.17287329133899,130.16281510203336,119.00596997336517,118.46640854370744\n118850,122.8684552217296,116.0534556838694,115.5848522205914,115.0850588803925,113.08964878510088,62.53328804420304,28.20914855334509,103.73773447846646,102.46461006908594,104.39374220886874,102.62249793494576,100.84623753805154,59.210417815940644,29.27039058042166,125.3955950486889,119.59646032986427,117.13429681989986,114.67043410348037,117.32435021969697,79.31968315906228,43.53003938045287\n118859,12.950191031770844,12.533713073917552,12.027404276607943,11.984682049466862,11.942091104685936,8.417933289074451,3.690489555211211,22.223646670453903,18.163402874200173,19.177113896164137,16.20483326472816,16.403248161769074,16.544637993558464,8.301667662347402,43.01246792401278,39.305194955345556,35.878262846462675,33.17261035635893,36.6237016380673,33.640550915622875,19.99120800557006\n118865,14.816213676226033,14.740855446327993,14.802044877985617,15.078106955791378,15.646649618861064,17.287620181253324,7.569526539292326,31.540919418939122,26.940620483946102,27.93801258521183,26.250787458719596,28.078268981908725,34.79127854076435,19.608352506212338,45.87280865481762,46.663207583141876,45.659550343788126,43.75671201796098,46.65273936408511,48.04822060266739,35.048305330289864\n118872,5.514008086937533,5.166539047945448,5.091981323678227,5.086527254679076,4.916683084979309,3.594406532351406,2.6891083778895273,9.713327299373796,8.348689766605585,8.364067924018661,7.773760954381739,8.394622619082117,8.165772576042201,6.2333227973439005,16.804134261120865,15.319595806234288,14.969403615836809,13.872112462706047,15.04126528963642,13.494625111604407,11.532641298068468\n118919,24.439038941491916,24.289999935693533,22.816108672818117,22.32026977470998,22.644939217890162,16.502825256841454,16.294703480275725,38.38820172532146,33.26008431140257,32.69285419626766,30.542000647193383,32.2177636928702,26.34555527733315,27.522025399508617,67.21515533914526,64.84967171400324,59.70527367182442,58.098892575713,60.57477563220827,41.885498822425376,42.73463690448652\n118927,27.391848420397768,26.07045740452547,25.54546359735248,25.541078273711616,25.08768716704172,14.341766153527843,9.168778111934365,27.452360157401202,25.385649963349724,26.34974398559851,23.82778703735941,23.423845453322603,16.8593446573735,9.576710751926784,42.85600176452873,37.70090295378336,36.069287636460224,34.21211510518327,37.091045392270324,27.17882423399542,19.104952630200636\n118931,19.72899216164017,19.47650813649745,19.008269235419466,18.94010381244562,18.62395723213141,9.576367338283024,8.489151010737563,21.20195954720039,19.192301981443304,19.368314768378283,18.038952412285393,17.508185355759935,10.504868164561264,8.140214984736131,35.93073488858985,30.751305804245202,27.629468850198922,25.97937349753737,27.410714195051096,19.12799142586824,15.009725524132751\n118940,4.689360373607086,4.5854999070770965,4.548604057904369,4.536783180145751,4.464560140174268,2.0954534608880984,1.3252982477519657,8.719000374602004,6.997874848234374,6.6281864893018785,6.078472376423212,6.612438561303567,5.480823327674172,2.6369696655979395,15.51374875068005,12.96786333881225,11.550859892020963,10.930815999596494,12.248870134535863,10.620301513471242,6.779633332764019\n118942,105.8969728700396,106.33998339992577,101.81495084052797,100.04697594873699,101.3683723764127,67.44203087312987,64.97740849020218,117.54671907319127,108.04021381381317,107.99701268224011,95.22900936350895,93.76429945352291,77.65918769885582,91.55493330466292,198.20663952244675,195.34346348230542,166.59433584896033,156.75533490597513,176.0468265088265,191.47726706501388,193.69651340281655\n118975,32.39421675288609,31.188080911143896,29.964561785590917,30.594035929839094,32.527699456824614,33.61686984260152,24.073696889104273,65.24645878856796,55.301125201366226,57.700636249162216,52.47721946678416,57.27038736409374,70.800546922526,41.69102295456722,99.24567948763588,101.05349738527943,98.42028281749,95.6727475146481,101.56455996718367,103.49613328105931,76.74519763460553\n118976,105.62973249364941,103.7328658336371,93.73230350602728,96.08662441763553,96.32929697032314,68.45735963958244,70.06309046289883,174.12873946660162,155.93532079978863,152.8183747637042,145.12602909619892,151.68405542572947,124.02553481303474,127.59766519082334,282.3055411863438,277.8373037584728,266.9006657821002,250.43437039360904,259.76817651076294,180.18645791584615,182.52033088421058\n118977,8.214536435304465,7.842168522956682,7.525067683012577,7.445152072083596,7.336526717912164,4.997391672215029,2.5147428898199373,13.504256622762606,11.349296090263344,12.077622594958676,10.15270984936419,11.81403769483056,13.413950340623892,11.55852311850766,25.19729156696729,26.189482035574173,23.503413330595567,22.902531197701155,24.874714369641648,24.0527195989404,23.486954399453126\n118981,18.654927519696383,17.767588771879016,17.521259870042773,17.501560246123336,17.17073998348531,12.727808108041904,11.440978818501469,18.879221428050794,17.881904958191235,18.191522689099546,16.459822851284564,16.311921077172165,14.873661892728686,10.908362616312239,30.977788286310254,27.47089162719161,26.44511683830223,24.79085020230403,26.785202978182614,23.57105884668373,17.19816922201697\n119034,121.81950435934323,120.48791691745373,115.03023043865396,113.15729903190439,112.64512809439749,61.47652391125974,58.68903707516687,167.81621862885575,153.63004959862798,149.7725802308624,139.18045922400577,138.8736820094954,96.63615450847786,97.75537276603939,294.22645415533793,278.28400782256085,257.97901622534573,249.3662782379087,259.3952476687544,161.18099920564768,165.3485067446468\n119039,30.356209818088495,28.851602183659917,28.395895903560728,28.298648775397783,27.39580207621883,20.052906967656867,13.313262218284418,30.71501749973839,28.75867137083617,29.07578469872289,26.718699344999465,26.450499974296246,24.119956014744446,12.916644393596131,56.37715398883899,48.86831234548865,45.64251573786984,43.455516477440355,47.27829636825547,44.51262679548325,23.118792454357656\n119048,36.26958681020425,36.16253553170798,35.348316670691396,35.275368272561515,34.76378240734498,23.885984094022692,22.418758698311034,61.09916582864654,53.514028888369246,53.1533312550405,48.76479431967573,53.42735860554831,56.46736002055674,39.47878257942951,102.4516941236147,93.57293606564292,89.50808251708679,85.13390632836872,100.49663420146673,85.72483971124039,82.15771654931926\n119055,21.756756699155005,21.260884252834604,20.564002572239726,20.375259837688773,19.998021113311776,18.26356719472291,18.91624572800105,30.77817064145485,27.503632133708564,27.741993506233978,24.965495593969976,25.115096823542633,29.401353188596566,17.369915763593994,68.59727624936623,62.33840689961827,56.02601634966631,52.58189985252756,57.72829834033359,58.09126011677342,35.29602266035501\n119059,257.8085703611829,255.8453047001199,247.29928169091838,242.46647709441,241.2827485127908,162.12877084550794,166.16297981762403,330.6387754114725,311.5595199517665,319.8628072049964,308.9454490906505,300.42348331456884,153.63229493925266,154.79371056699887,546.5560826455834,538.331882466057,487.42304628258694,483.16106342983744,501.7434546509119,269.0873309782298,266.4642350351765\n119060,29.24278230537002,27.040322483449362,27.278899220365698,27.20895603338018,26.512023487809167,18.30896550485851,11.530046821098178,29.353710209230787,27.54286262675735,27.400170479278565,26.408854852601547,26.15447021623107,21.45603023318848,11.10654822921438,46.64804633098485,41.29622703084112,40.050412201339384,37.9991606743015,40.40401218942913,34.97372331435671,22.302371195326742\n119067,16.536829130159617,16.420331150243378,16.052012557682477,16.071519295189212,16.34956831150656,12.460374715613762,12.724854704891946,18.699450402664425,17.420053657433613,17.270937949496176,15.739391869881253,15.670106716198262,14.174420262725333,16.56274353591293,29.91819836805913,30.64309066797518,26.22059029967756,24.315720097741288,27.641582485947065,26.003533430847433,26.683996878786747\n119068,115.7185835037333,118.1258321080577,113.28970652174178,113.14680508760719,112.49937436169301,66.65995535720597,67.24906667082604,130.39756160463583,128.94504026064504,128.43290104631512,122.038128568685,118.9508000553078,69.07786259496783,66.4861805565396,176.50818319166623,171.6323874517576,159.8076616403935,154.76950160794294,160.8896947392034,87.51655243955739,93.32924391799351\n119078,17.557305230439724,16.625023272102336,16.32418930410006,16.309583605410694,15.906587226100264,10.181717621122665,6.998977448856829,19.5340909274536,18.704904925913553,19.178180891052403,17.81011753427786,18.105692972124253,16.37127589039278,15.22680823415404,31.454738866655006,30.75098994337666,29.48126898912744,27.541701151837216,29.47570995244093,27.72453405553303,27.217047201959755\n119105,15.565303385582691,15.186208719923114,15.13805709590886,15.168145942341662,14.697553160429992,9.515958767584445,8.589427751118139,16.977591002595027,15.112239844847652,14.921123593132059,13.756726368182829,13.322794535043746,9.974291161119693,6.1324583730004205,30.6382915804392,25.91864843981693,23.809235684361717,22.134080231983564,23.126073705532722,19.4824211627914,10.314950324940492\n119121,25.38679248186547,24.258226153885733,23.838643476278637,23.764608656983302,23.20266176653511,17.394395129283826,14.953295291838428,28.917717041838287,26.18551142631097,26.51291906674594,24.24948063895852,24.167430690599545,22.13104327715505,14.961613568666168,50.03691056584667,43.939205823323654,41.314807636420234,38.35433620409076,40.645803290764114,38.992103890905,23.929573276875146\n119135,23.151220747303885,22.51174487109877,21.847211162395652,21.74821775600156,21.30455691036922,13.439651639672602,12.493817848902465,27.56518812317426,24.79333637049926,24.841956466435477,23.532374135284627,23.267771495322663,21.46787707637539,16.76581162010008,47.36077945240335,44.171696865206336,40.27870334749056,37.89392653344697,40.622790205685895,38.85968866152212,30.953669193917623\n119137,126.70612563584352,123.80900489990177,122.32281842891267,120.02992676172413,118.14275426926923,60.11467877297561,61.303705011067,168.55363693508912,160.98798765457727,160.63308435384974,148.05193594622006,151.66598971672104,89.45367015010623,91.6106067284922,271.15588580079253,257.9784950816063,249.7466712888076,238.0485995288872,246.63660566429365,145.02380300737676,148.97338599169788\n119152,54.3622850858755,52.57545611077296,51.079943181614595,51.56087022973551,52.08780875439085,41.40981359509345,40.752933227985736,95.19742401332321,86.42075902969502,85.91667010327052,80.08371224296803,87.19685758680978,71.72759926565841,69.08960771833193,164.66342740980124,155.65694187997775,150.82410749281925,140.53642695747345,150.29710256065675,102.73609072881084,104.24753743956612\n119172,7.776147329697552,7.166097353408957,7.118238504379844,7.1297566863972435,6.900724047750838,4.573269883549572,3.9791054057380677,10.411185573928703,9.929786508803897,10.29429983194815,9.30739400314167,9.843228533296195,10.115242888070489,9.30096613944333,20.047879240540425,19.489712798645364,19.309424156009673,17.762687986359357,19.858470389013316,18.465754583819194,18.837401935035185\n119177,10.183032116562035,9.931747966373138,9.658022040494801,9.60774609177843,9.511994974740833,6.714611102936893,4.234948090455507,14.518202131931632,12.879429373836828,13.010445538532153,11.901759597226066,11.848288178112538,9.60960870494013,5.461919503865544,28.64166092068397,25.735889361620156,23.705953528423823,22.191098076661454,23.351177871298237,20.18838621590351,12.108808625074099\n119187,43.3809242638615,41.666862573571855,40.633509512101824,40.31930877714183,39.71072550617282,37.54428554311918,27.405117641867946,59.3104764992454,56.38948133424643,58.0395892591916,54.55440526593903,56.79493864581079,61.12067893636879,59.97880699425345,107.29268219901665,105.40134389457603,102.47154434956077,95.39847726270608,103.4072238285035,100.43812562138551,103.83114433250776\n119195,5.042891703922744,4.791553164957665,4.786752656430205,4.764748622667181,4.662838369344519,2.4838555353611373,1.7503939773886306,7.911121987983428,6.6999912036126075,6.591787031137192,5.999095484785789,6.207634832023883,5.543441271546026,3.254538183295295,14.458590793764445,12.35974303390776,11.59465499051802,10.684369747388265,11.602312050046576,10.49542421694345,7.317579997799932\n119202,8.967972334478477,8.702793501255083,8.498338731846932,8.48503942416383,8.427856217000652,7.484797979620197,6.067235826990408,22.764892662555425,19.315907021343932,18.645530737791205,17.477361282393854,19.849875093593663,21.657274494588894,13.287034647809316,43.955873870125714,40.48733489017764,37.6446509297552,35.5715074576079,39.906614615330426,36.57757402549068,29.531501982987425\n119204,18.518135227939432,17.374117441006423,17.31549310288217,17.274353030965933,16.94915748721133,11.151938026918046,6.424860495665064,17.934149916274524,17.66379697847867,18.126827754624298,17.073672966748703,17.004518365539997,15.035380594617717,13.851136107212687,26.536071043059838,25.63022298904317,25.02724418102809,23.793859971682163,25.581590272745974,24.08293454032686,26.739392273654065\n119210,356.92871887429254,352.11974743097136,359.32085521737736,353.95347283142866,363.8637618126596,385.84280293364907,211.6613310703439,638.7330168371996,580.2979188657492,587.8731164769653,568.030930216969,597.5365407624191,681.7561163710739,412.076619761468,849.6286083195386,837.2006022198693,820.5717717817577,792.865135512299,897.2240145115048,814.7628419016007,659.9853113686692\n119212,126.87169044969865,127.0523345684608,119.71610651142333,119.67801477634714,122.11190706180597,108.5511585043095,107.76613062990408,152.91259963914024,134.79797868796774,137.64194780701928,118.74022050613694,124.05071494366565,129.19881500190738,107.43947257315757,253.03671603298622,243.5258009377734,220.7121052484196,207.43635047084572,232.68604249710594,220.23016256356638,179.25355756457625\n119214,17.457163540080508,17.47107172744082,17.11306740666967,17.15263968980051,17.33228660606993,10.834514464021183,10.31437782894112,17.44467997131669,16.779374195228243,16.528437780519855,15.235779874738547,14.788851698012081,11.888599853973096,13.661683586298267,26.55393092877091,27.26062368731561,24.213196090408225,22.096903707976235,23.83766931758454,23.751529358432506,24.398544340929067\n119226,228.45374241718454,222.05472356152958,208.4031007044574,210.82647870718924,218.75256996568717,133.7466006112284,134.31264055965553,386.3538196563467,331.8930421141355,330.141090676442,293.06265392027586,322.38119421483236,273.0420603144389,274.79679479619193,647.8355614050856,644.6878256903959,592.3462064474547,581.5641429117061,626.4681507548181,424.11313516567816,425.0813908676937\n119237,1.4273386311663805,1.3951254548976917,1.3656513944592763,1.360778353700142,1.3692469107120717,0.5724465474361252,0.5739343643165613,2.6643608254130644,2.3446355905119756,2.203208675395126,1.9993631092702933,1.9950383015914537,1.511672538224235,1.948940861953703,4.893782018832322,4.7765690745549065,4.101050352163956,3.7603356638263215,4.284383156596774,3.780200668470516,4.007744881605369\n119273,9.993229514755004,9.60094937376229,9.430458152404793,9.49166077895696,9.225840019094745,8.147338594877063,8.019052301570753,17.375325768811734,14.608498012646841,14.733757821837678,13.391201940270948,14.050934437305655,12.80424499142434,9.271300122387245,31.11039372401549,26.87983182618377,26.03511720067751,23.957273565201497,26.17932221089855,20.60936123478053,17.25762720819911\n119275,25.46306352738244,24.639863595255573,23.650749256226227,23.56471225579598,23.500764396572407,15.70163039155724,12.62718675474599,49.79454769667363,40.168982163639214,43.234503425124956,35.80376329705111,36.45694255364676,35.04469648511025,27.841833409880863,89.18817177201674,82.02042874514215,75.05971878486268,70.32140567165237,80.06517676142103,68.84146231386909,52.730059515957294\n119298,24.556820301461194,23.694066319213682,22.98767490438613,22.93593778707656,22.47839066371035,18.429134360104346,14.998354193737667,40.82043553232048,38.33971366366774,38.69562667561016,36.36630124045925,38.6482475032916,42.15670685781953,41.01724417471378,74.90521951993684,73.73945998540097,71.24234764688693,67.8258852106619,72.35697409498137,71.02599481471331,72.99945936419292\n119305,69.17338018804936,69.61185178977235,66.73313179073129,66.0947296186704,65.35636694909108,40.875702150511266,40.96699816098086,104.1259629470633,89.67869323523176,88.55312269104473,80.19881334040664,85.08560204528597,63.015234340361324,66.48468113392836,177.45080140278367,167.85813250260242,149.22900080003316,147.0749525748102,157.3057871273462,109.43705638760913,108.72597688042879\n119310,17.145813113931876,16.497281859409206,16.24962014535482,16.247177837483893,15.957452550760902,10.693373426322937,10.341378230290344,20.952420161924994,18.66607428224211,18.878977625984746,17.26154766828027,17.28542382304131,13.94285934980353,10.259057569834194,35.43438815324916,30.601848843447463,29.536005540999245,27.24447461344291,29.56428875797203,22.9649253621977,17.80377544992274\n119333,15.70201282158557,14.81315236486745,14.796773664700918,14.808483954600282,14.507575653574944,8.87104094126663,7.014971465284756,16.739821709621474,16.32174856560651,16.40184577559831,15.388118659747308,15.71445348484988,15.553101736942837,15.410646931715569,27.441416364439917,26.766459849364875,25.577501856295804,24.22370090187721,25.763133193052905,27.19407814039533,29.442017843333005\n119353,9.42007652761222,9.098762702169322,8.772962603083116,8.73094213280016,8.534755870824872,5.268818937472257,3.9378551619263424,13.140586149927755,11.261995459666522,11.411563329610615,10.444667246305976,10.38877969683912,8.402963036717024,5.781802626281377,23.040369818765896,20.153501140601076,18.592118385086483,17.57890456717825,19.203434833010892,14.905168305793309,11.649180301680655\n119359,5.473563935190732,5.119186295673619,5.109678124494232,5.135021895238581,4.936212554567881,4.211353244338173,3.714329294824807,7.966553989805173,7.456016780888685,7.486955179286572,6.378639378373742,6.712115680879601,7.874531624264456,4.190002196353024,17.457600628022288,15.973513119145254,15.631322195160367,14.347275087272296,16.13202561922959,15.646669787937554,9.504331611776172\n119360,7.394661841485757,7.209384630692957,7.084304135481597,7.080878323029899,6.977737712110684,1.8003311989077975,2.047698559726124,10.551204738724962,9.284157832771442,9.335824790348912,8.31363693071879,8.462147001514236,6.048405382804168,6.397345927712887,19.230561593376507,17.078296147181508,16.02653742214024,15.194638711365355,16.594931543820014,13.167498489057413,15.347346189166158\n119361,25.66248490138995,24.519184109012873,24.412756037260106,24.391972555631998,23.960172622753,12.55546283990714,6.313455630650406,25.640104509434916,24.036378890940462,24.577776529457058,22.891496994005436,22.511513003387822,17.446943303864558,10.644828285959527,38.21350005240065,34.251527548178224,32.95278707771559,31.37392541430918,33.499889963046314,29.74055506195674,24.769852319411488\n119395,8.116434895075676,7.450740198446702,7.480124029871542,7.529694937179925,7.462615919752494,6.364478649807142,3.656946582194746,19.102549220801787,17.025108084285996,16.204913817243465,16.04622946530772,18.106064648167767,19.210054333603125,12.054803000270585,29.617664063717452,28.83065541465615,28.723888502513333,26.883191042952827,27.756472762332503,27.13386424465256,20.57708299501993\n119408,20.916575887092065,20.439378568019936,19.689113055141025,19.5996334883271,19.17685647002567,16.1997763898516,13.383354340182287,36.70004976147689,31.96719813679606,32.29095192339354,30.019223034777706,31.28174220034083,33.91497734081618,25.642608417377414,66.94029470411516,62.84669691006833,59.68902528968835,56.29294220278213,61.352668634051426,57.74922673749929,48.88566648410886\n119418,1822.4902396370956,1819.2783434236367,1785.3955832845556,1780.3281198221196,1762.1634734277452,1598.0621103030992,1421.3189289435925,1882.953370202063,1790.4988117254209,1797.6186770595284,1697.5886507436828,1709.878062766763,1725.3362950368025,1221.145404347003,2192.1293485757205,2039.6065781587865,1993.9028585673739,1926.085472347816,1987.5356513944103,1921.052799383362,1402.2606425197102\n119435,82.92016617955106,73.880999269336,71.20101327314187,71.93234179303055,74.49565366908091,57.57467085365757,61.33545152646469,163.49240396810026,142.3642647300911,139.0628525455547,128.3290920076602,143.60254840806735,117.16335172317116,119.73706135517304,262.6094181374456,249.64793433911166,242.44962101905307,223.9347081528158,239.44800771748916,158.7177246843382,161.83187893473755\n119446,1.516738990844626,1.4133134049725216,1.414033306860118,1.422910303879397,1.381329719603354,0.6638147469581328,0.7335977257205896,3.0740577502066437,2.6765840381327046,2.7687225268823332,2.422050745811774,2.5968356384900724,2.3221599541855955,2.385622016129779,6.168822311408833,5.679950852322442,5.46788978931821,4.995899193000688,5.718787761098409,4.724021030734483,5.251154297007821\n119449,29.260794403629756,28.398665705371528,27.35684783547126,27.228946044987772,26.868926750348994,25.70285889406583,25.57067803048551,38.84192669131999,35.06011851431969,35.16444523467563,32.00000419382217,33.445078190893,41.033573630925986,25.55304368036697,60.19560616629419,56.40531248579984,54.99790641701208,51.7906566197003,55.5327699530069,58.73179384760051,42.26761622751583\n119452,8.374358800487354,8.040242297896684,7.680935828023155,7.430305582121824,7.309427331140661,6.747005403517952,5.948680001998614,10.312595665392191,9.197672917323464,9.26573090204138,8.15584734601306,8.635883128007384,9.045348099585292,8.581456769805843,19.984360666015192,20.095192940895718,17.57479343971944,17.447074373450775,18.24663656968615,17.548303943963425,16.203881040751153\n119473,601.626330890913,584.405159900077,579.5639886671041,586.4330401519475,590.7762810234461,502.6486654638304,504.15751080570567,629.5270941707261,614.4930710565902,603.8080686574723,579.4768907976944,588.949766794808,577.6915148680351,593.6176464500362,868.748588553335,854.244649985308,815.7767739202152,787.621972358499,814.350525428682,788.9522021492437,819.7359875209468\n119474,15.397370948597494,14.844653874339816,14.753691282398536,14.850520483887129,14.51299958795544,9.20499817006565,9.39024383111737,22.74916756272362,20.18136289292294,20.19635137415652,18.744788735175284,20.52249177779121,20.808629889448877,16.09454903102227,38.098808303148466,35.15087969060363,34.59548962466908,32.1190738376714,34.70597539611452,32.32036381628544,28.55059267654279\n119476,25.48080033187151,24.04868105838992,23.796990253162065,23.58538941205974,22.907749197293676,16.957483074163175,10.842403399669847,25.020731013686035,24.822025632939408,25.799771576873937,23.164023332434727,23.16100922638291,21.50377739382375,17.770643611239677,47.33924860914835,45.85890360659977,44.14267750273026,40.95405587970086,44.99086483540627,41.36843588711802,39.18914969402266\n119477,40.52880091159376,39.93106659430948,38.02959381636865,38.4593271771255,39.08377827236813,28.423399897438802,29.154073571057253,55.777545095317336,50.90291976128275,49.1813465712608,44.70704691673109,43.85817534537092,38.60629336842455,40.85111032489878,94.08177072004008,90.67249179925182,84.77908098042188,77.40181149103529,83.72576726263321,74.37177061140159,71.69298651814938\n119491,288.2647734062471,273.98016010763735,268.91663518534654,269.20439317294813,266.3800145400607,152.5274408975902,146.78920876529526,445.436891046434,406.8201260391353,405.6313110271066,374.2384513229167,390.6488902482401,255.36690663598188,257.9136484581108,786.0284674643046,729.0861120239892,700.2581690430626,656.2821056773594,688.5861589311609,393.7340805414681,401.72258605120504\n119498,37.006319813012986,37.28376819342873,36.75073162408599,36.800054951089905,37.20777541543736,23.66299707654051,19.61077093775184,30.4203787432032,29.696702224506048,30.018659453711823,29.00986553816224,27.98941283623883,17.399709097564617,13.354597889685975,40.50337671710562,38.098192863495896,33.73838609984324,31.201112065896314,32.594274549877994,24.596077583262577,16.37343587444043\n119505,4.250602803960776,4.115052191267566,4.019561771093213,4.021742842031314,3.9171982290474925,2.7901745280514145,1.7760951714451663,11.741783566896178,9.9188295478463,10.150970543735966,9.066186121634868,9.84895721197088,12.045516287614024,8.08928371266872,25.265794488564673,24.379701434513773,22.393406050423238,20.949602174119562,24.199800203679366,23.30888333827698,19.215252359486005\n119512,8.845313736162092,8.371305741819219,8.137856398572897,8.12549468987599,8.208388476808997,5.563551165893789,5.030330650923499,8.683952985826025,7.74562961262718,7.522228150813551,7.052604238643779,6.780651600146838,5.05790318038061,4.991392298527907,11.905276794629604,11.51264197631906,9.969113984464302,9.10225685639277,9.847237888086793,8.353782135278513,7.3659353482979215\n119531,62.903923239452766,59.94279638523845,59.03279168111791,59.04870897827097,57.71072493678399,54.33593862328697,38.485084618444496,74.71437272208506,72.65904327711824,73.56578549245366,69.05047068983392,70.14079849518036,73.47551076090046,71.67884172068095,136.05318739615848,132.90368109352076,130.84830107197809,122.57044158447017,131.1781184508884,128.26652129664765,131.11440515345805\n119532,2.148908192831298,1.9756109875536327,1.9955541227643216,1.9840135771470395,1.9270185612493724,1.2507363753167349,0.7316863508175444,4.508400384066722,4.023596534143834,4.069410882566756,3.7150528041345074,4.124594625734498,4.632786648872179,4.417987381330095,8.338502220147099,8.093607319249578,7.80497806230856,7.347660752689521,8.10083788133869,8.260385558828103,8.748730243839548\n119543,96.33752221593384,97.18606834938453,94.20638929446677,93.10528447031534,96.1317195430946,63.21024570785797,62.4819624822362,126.81036161505197,113.50734525332744,117.22392917130664,106.46353369418493,104.4503411251917,83.40546766570515,86.74469268589341,197.48572247269783,197.51659813617644,185.8648116766743,168.1596604174454,175.4659445692462,139.8918482533898,136.04344683515603\n119549,1624.6595928686224,1592.1541400779615,1557.9721272807517,1550.610339614526,1516.242066884202,934.5771669344645,684.8171097288309,1795.941300488668,1727.9654900413022,1736.2945777036762,1601.0754828472157,1617.0210847794317,1489.9139704618844,493.24609263259515,2952.5452097623693,2816.7859327111196,2818.7307783387123,2577.727417650865,2789.229645899062,2635.344869653556,778.9090364987227\n119561,17.23558394816006,16.16593903626113,15.641496151728012,15.582125174460138,15.156644312968401,8.420482162347225,5.772663879573191,20.06352530191487,18.18240562651446,18.545358978937426,17.012983160498877,16.75463710555331,13.065079555472186,10.637890975957232,36.654249398323564,33.74308436158141,31.37677201264871,29.856977988476064,32.61936249790869,26.776256991921386,24.627446337176114\n119569,1.5602791121119082,1.3865215412759637,1.3550601316854352,1.364383996147093,1.2952072933327667,1.2939740271715323,0.8815604671520049,3.855871079314949,3.4782822694624937,3.553818539022348,3.235566273962396,3.688110029484245,3.932187286239416,3.754664820714868,8.027178830379304,7.888663118440534,7.586547496942356,6.996295147919996,7.726307584672709,6.810041494313173,6.94418164464317\n119584,10.481514595478698,9.763593911444213,9.673621306256761,9.655294247829842,9.414607599618696,8.39280140448559,4.618651103243292,9.976210620979849,9.422252041008047,9.481896131351427,8.89206015447322,8.581841489945914,8.631033786150622,4.516587528274072,16.194344614638506,14.416515901238096,13.939999234650976,13.181792926204094,13.6340301561075,14.30233742772825,8.554777101563753\n119586,7.111457550878435,6.863590396207667,6.715651828756267,6.7097456915804825,6.5816342953299065,4.4512277296345575,4.207692176056279,10.23784492081282,8.893829197310152,8.80358382274089,8.0934432932054,8.46534139060491,7.549466101456058,5.893266068564916,18.007499094544986,15.722258308685754,14.877364978869903,14.084972993757544,15.125532171865519,12.697128983079784,11.25664910280554\n119593,50.78252669951308,48.936568202304116,48.31775895621048,48.25456616233163,47.19719206210659,25.385557494283105,18.257712467701563,38.22258000447528,38.799158486618246,40.09097053343308,38.04501748216837,36.347544164470946,23.01486286829492,21.162953508464987,46.17329342874851,44.760887794990985,43.0245926717488,41.95280260875782,43.659498096587825,34.55074836715038,36.42437198404322\n119611,2.493550039138553,2.412383406163738,2.3964078555807578,2.3954601695315545,2.3696968265771283,2.20393586613106,2.154968444482502,5.306035470225032,4.910509923593583,4.748108511424868,4.518137325752997,5.21306459112049,5.787101410964312,5.454971804717712,9.81713232646541,9.666015302115095,9.385413877533844,8.937845552401077,9.90957968269481,9.607793428116233,10.087338817132883\n119614,2.684935390525337,2.5755109696755456,2.5563651363262005,2.5436997105839487,2.4684909522213783,2.043214112352924,1.695050355095946,5.129575038575431,4.72748953261829,4.710509113229496,4.367091467522479,4.8786270868275325,5.2627901878485455,5.060986693510138,9.563925940413643,9.410697334359261,9.03573320927578,8.593813126233734,9.245822511688838,9.119047595869699,9.186135348360114\n119616,9.220905483353798,8.994702797469133,8.636745772155724,8.59642658466279,8.45401948261501,6.0415839622296765,3.877422858849231,17.37698390134123,14.158501398749577,14.823829287380306,12.534456387332003,13.301706626773468,13.89527640617419,9.202634211244863,34.9975414500497,32.26935117295067,29.543688551766696,28.22866546171682,31.09558345880849,29.525152938407157,20.48198661796783\n119619,116.44351234591493,109.772002590264,105.77407208208618,105.69514861912154,107.38283086922068,60.162271945984685,59.256646620266956,217.32309234188884,196.22001607035182,189.2275877867921,175.0473181095456,188.55973325073947,129.30169044108348,131.2304787629039,412.2978449542066,384.6149889743992,363.60755354938186,343.91517208561294,362.39127649544133,201.54789140681004,205.960925947914\n119623,1.1957944434774255,1.1856794969849573,1.1648742571737147,1.1476212521537854,1.135694040542107,0.543906620678586,0.5589930923929362,3.1440530372932756,2.648844168261041,2.639233733779958,2.40262589912341,2.5520070511605017,1.9793808595882072,2.1531738514332432,6.389290128517452,5.911743010011628,5.327844870773071,5.025038946560773,5.525844156658001,4.20541420766398,4.655852519830711\n119627,6.466285411966292,6.100523452439023,5.984208121913439,6.008002599470732,5.895304415377338,1.7740956298534554,2.007009582483176,18.273865061746022,15.098949951007079,15.744237023373897,13.734469582376153,14.720858058795189,9.138659540624385,9.260958023842496,37.22128281466378,33.15341978514333,31.867762853907347,29.470770211589084,33.3929537015076,20.511884057568032,24.25631335536655\n119629,26.7348544518057,25.82731237567549,25.191652523923672,25.22617594789884,25.309531834941577,15.622753456648537,9.465564829910228,51.44498369389835,45.896018456148354,44.84147465747138,42.05582688211379,45.98631877281151,47.73787448178532,35.64050170184342,88.10531722666263,85.98094028244518,82.82887859697948,77.29943676804538,83.5806339451922,78.98657435597896,65.08433616023605\n119630,26.016663418384056,25.20654468253508,24.545720695580076,24.36518230881433,24.363289038553503,13.965464882700307,12.12712488736233,22.980182315612545,21.590756855987596,21.841460589952185,20.620015012501835,20.123291147855372,13.58768811686253,12.964946557655079,30.670592292357338,30.02233445887069,27.351986402263446,25.64572139130835,27.296553311167017,24.110581879956626,22.24272910029035\n119643,1531.52504794123,1444.9286292140728,1432.7694376630106,1448.1177430344385,1414.3216203135207,1346.207029040785,913.6682677902863,1131.8005402252531,1145.4148369196275,1152.0008821327212,1082.0342384239316,1018.3868098862613,1086.3492918764816,520.5471299964848,1654.9181672188015,1584.1252621391511,1506.8238781802454,1436.1149391372599,1465.2804320040932,1607.917390076524,739.7935964851084\n119650,692.702004337653,699.8831771836733,684.3617770683743,685.9742149711367,694.8777967988042,515.0857915447386,452.0556100505718,722.852254336067,679.8410210919604,683.2944412697545,647.9084990458999,650.0339113840773,606.7629001300046,460.4425401974062,962.5783868632205,948.9300435784329,884.1711020526581,859.1781217225648,874.5148057971356,865.1981102787659,677.7082344587362\n119651,3.5271857097209445,3.3703614896425473,3.3436101742537727,3.3403526886898978,3.274035867662327,2.6024987731939184,1.4374864546044608,5.5269126276105665,4.841089617788016,4.657140368859264,4.309814502960523,4.270336962175482,4.931246766892135,2.1181234341055815,9.97422418549034,8.731618191937066,8.481104756644417,8.00644844174537,8.29344279279405,9.202796366843613,4.929795954767854\n119655,145.36358485152655,142.70200082347412,138.996878227031,137.83389165135188,134.57127087471395,115.92282115579859,63.38338198342108,140.94212414445624,136.18613314918596,140.20750891689863,133.0341036874637,130.4042908910916,129.87836682911268,85.70995481779289,254.52149676428525,245.3841793916871,222.71088808503256,211.88182897058311,222.18918870124241,239.69740576270434,175.14006623234894\n119665,3.4026319266690908,3.397805695807748,3.261624732955507,3.290189969391985,3.2659014277052183,0.8533609585747644,0.7506777381976039,6.615233664899798,5.624495414574252,5.584082628485015,5.017326534242709,4.855411987937887,2.3224773006160904,2.957798459206289,11.411240844580249,10.348252447904608,9.426031175882404,8.717049669056554,9.297665494690783,5.353674015897249,5.970885186958903\n119672,7.074688900547051,6.391175294902173,6.121993547969418,6.147725354765139,5.918707040980711,5.20860425923603,2.5947893564962707,17.21189764258613,15.033989142743899,15.13061725757443,13.487934476426217,15.62991744697939,17.814463816635396,13.124977378741857,37.382299558537234,35.27395902564713,33.43347048955983,31.53679174988668,35.30289646324468,32.68273279561715,29.580021392652316\n119673,60.42109835078709,57.92106870439035,57.66782020882098,57.80585671999033,56.45551798839041,50.97786754916032,44.8200237053018,79.74739938949718,74.8180706767787,75.71369986666322,66.3026380077332,68.7765558063395,84.15162455455986,48.97572133798673,144.14772228594268,128.29134327209016,132.79693006855825,124.57136497544332,140.28728477043475,133.10056741787213,94.73338671468808\n119676,26.40774792999089,26.603626115261985,25.771093980354983,25.743098843066406,25.703042071381383,15.553423080322794,13.222256093094758,47.10932381402545,42.22093147221367,46.339863709010196,37.863831013893396,38.189397351152195,40.71213509150259,42.712275824034464,92.43767266969896,94.99128830842302,84.43107084441445,78.1396486103805,88.7717089004047,92.27923755554933,90.27401577927488\n119677,13.221384751556092,13.267814517780021,13.19231132145583,13.16222280883276,12.99709131486013,8.512932562884304,7.954601964428406,12.605248913078812,12.275996122156517,12.353466061371273,11.91697810713587,11.689941480834676,8.434574067243439,7.55633562811337,17.10290126950938,16.204728431447165,14.774550760325926,14.27746971625647,14.364004719116549,12.802860396950745,11.485023701610954\n119678,555.901327712365,541.6668967445448,524.9902081065393,521.1333335086046,509.88728455825753,313.84823474224,326.42375562657946,751.7371214891874,696.9408069384184,709.1545494633839,648.571251717532,668.4994058719335,537.6652086015536,533.9948339941401,1256.4482830284085,1146.3534607663057,1098.982961674533,1044.5281392644201,1102.2966029821655,826.7401978017913,862.1306405143949\n119682,125.52797521925473,123.15786637355679,119.19190435806176,118.17341052563488,116.06250847468115,91.62122928173294,38.31463413942306,121.40845980153645,113.2519253731016,117.25427387747847,109.75682722371761,107.39367352259657,99.59267905751304,48.72750588503129,200.48662592222323,184.7635239794819,171.14109379115982,162.1489914077473,173.12608363823574,170.56630871373054,112.70349422679443\n119697,30.513930071695807,29.386432895863642,29.081440917311216,29.089287226212626,28.684453193013223,12.902882644611985,11.192552875523454,32.57605257269482,31.95890251479401,33.00347391271568,30.62539481088405,30.720233325600166,28.179329781245315,27.526552763677156,50.72062478871553,49.24607599804016,48.47160414558462,46.17188100487447,49.74948168757247,48.00792655300245,51.77003253766196\n119709,15.57857560065113,15.006811317112264,14.621359809915202,14.551206947340093,14.225088002324291,9.522333441258992,7.533603848327868,21.564886446852658,19.151427341079895,18.511371589264073,17.09412010361173,17.8180224836248,16.85854058260311,12.533104810413278,39.23859390040574,34.69433096773902,33.01817482282091,31.09904817540414,34.359655928155526,31.484469041843195,26.62758863542107\n119712,99.69231866775854,98.2619635184476,94.71711804642892,94.1124697649348,92.7275650370009,80.21230506474797,70.8393015152921,111.63622930094161,102.19480607739715,103.02222958548028,98.59578574687886,97.6231751648205,98.91860742411208,67.38977628216384,179.22165271087852,165.7340906456814,154.30449791928172,145.79980733749406,152.2066311151854,163.5927608133827,95.28184325052197\n119729,28.283702403154404,27.73731408360977,27.77657632548502,27.734403133055913,27.281711446811915,18.844973530710607,16.845313822537868,32.267282004292596,30.444238238497064,30.953925632027783,28.61300960011246,28.34259796275664,25.496801995629017,20.322830704888776,59.03714357697978,54.462608185225164,51.16555338322488,47.46833820478661,51.137717369583676,49.045121063306,42.74270918925842\n119731,112.67505574509559,110.81376004563865,106.02406353048886,107.7449799768621,109.28338762469622,65.04583433581293,65.29796720154275,136.18037497988942,126.78509771365114,121.94309690490496,112.0985445201074,107.73026067604967,81.35268703825193,89.63970902657785,226.76092749939698,220.4553935843667,199.1622561672453,183.89982216388296,196.20040764820962,159.75150837375585,157.8853961181751\n119733,5.021768210993916,4.509053051493373,4.482778629788551,4.489495245524911,4.283032201052338,3.3425654507963665,2.161566185676719,8.949927993176251,8.397711595400752,8.702743582810248,7.61993992734905,8.42793973156959,9.387334265997465,8.474692137434692,19.713269499792748,19.126519715093895,18.89366913453783,17.410666736456566,19.723234035188533,18.303867059946825,19.018262081862524\n119748,22.60041451183691,21.72496264072256,21.119361164032384,21.042251752301613,20.449474668254407,19.59161401169679,17.785408824070206,27.153665349280573,24.314715555498054,25.234572356988988,22.200162610407915,22.085262758621464,26.27648979406903,15.655763891879571,54.09903250382615,47.6222488492647,43.49423064681461,41.668320434704306,48.10325481694448,44.81294379682028,29.689165741672046\n119760,158.3823650746445,154.4684764489898,150.74000672918186,150.6927341015319,148.4830074713847,128.54384103698163,107.76304881753624,137.61428561754718,130.34930223977315,128.94523856902802,120.82834814539675,116.36661358369179,110.80222033265127,73.33793594100285,194.5217490860053,176.07352617993772,165.70156482464662,158.07132135061897,162.51285139379303,157.5687076073228,103.023075659543\n119763,149.80647127925513,148.82362251168058,145.78410473505878,145.4607274120624,143.43487521605286,94.8896702399587,85.753644935328,142.42848061522517,138.53238990400465,138.91718871500038,132.7394478856767,130.10892635095823,105.56280895367233,80.85346507521845,203.7445421727063,193.69419535782188,187.02156380042018,176.3957732380252,182.9051136026018,174.88697615390697,119.50720353009359\n119766,78.91539729302154,75.06727552145921,70.55522943076673,71.98959360492208,74.93410702574256,57.613531001189386,56.13017931979159,180.51327435594453,154.65508116191398,152.95692966658586,145.2172669234271,158.16233794040608,134.957945099118,133.91478261179253,323.46896521821697,315.9632302314358,298.1587825831286,277.8357978425512,293.5450368641899,223.2386654740703,221.76291406659854\n119787,57.862439558657705,57.76871964529279,54.76807312738136,54.588351916060894,53.03667963826535,48.21810555568052,45.16864129539238,68.82130645175182,64.58838504943435,64.09251482540421,59.782940615242616,60.87625119731837,62.86761582446928,64.09083676459527,113.92543463606548,112.20210258281261,105.87495420027771,101.68291690494247,106.58745325439631,106.89290769794869,110.5945058019349\n119789,7.114842361731006,6.797338479349636,6.457343537630559,6.38138610764805,6.208485565509047,7.1334347215446074,3.099632544438765,11.883114645645378,10.398584670165931,11.28764470196743,8.994170397672072,9.081744416665769,14.054091767834125,6.090327971040549,27.81455083518126,26.37888245429266,24.660430653615066,23.011730888671785,29.049382042592942,26.980867696818482,19.32606898348624\n119795,22.712448590027616,22.051738545005993,21.417123745999163,21.2455381770169,21.258522392518106,17.809779956077257,14.982681857241598,60.46682679164407,48.93866392214669,49.171933521506354,45.01057449261951,49.67424808435536,57.06703942263164,27.778777355583,121.61090242276694,109.02682928252959,98.81937112694993,94.21756788659624,104.9499440122186,98.54316850913561,61.83913230376979\n119811,39.53966591456435,39.292817707136344,38.467689834539726,38.42499074703708,39.20766692769076,30.232079603387728,28.93740030450196,36.977922435220826,34.24039936294144,33.4887576612507,32.25140020305069,31.479810785602695,26.000350012928937,21.207649610776585,51.255232618521916,48.82523137222582,43.53433814745725,41.326281275156006,43.14457186836438,37.74616047951486,26.691064078841737\n119813,11.614246541438959,10.837982001281482,10.63807087356778,10.675335785200792,10.327973979257902,9.116916158154,7.407970649874524,24.383263789677628,21.267255762548135,22.181249338909097,20.209918475486628,22.127075418105687,22.979304127443058,19.01466497871332,45.18424219954844,42.05762798964041,41.62193022905736,38.43016609954511,42.96930042887578,36.89229603692615,36.39490446734891\n119817,39.421705640119036,39.13029641016207,37.89720169797937,37.845680113498624,37.88033626386932,20.40506576943529,15.422373146202327,43.36332510651068,38.9687115092812,42.526329085364964,35.699987985953825,35.18214265578157,28.200619587110594,21.668575376494694,69.8160537450376,67.40621380976974,59.94456404068355,56.6308154337413,64.43491550696449,56.74885954573388,39.73376453286562\n119820,15.340993711483048,15.10540567915737,15.058012643955186,15.074630663477166,14.762416043163276,8.670569801541388,8.062200217680045,16.78983753468376,16.099750191278147,16.56593430671254,15.57537830636478,15.358662155891876,14.160573412054319,13.423793060343254,26.340367165442558,25.30531552205432,24.42106888293022,23.281105398912636,24.95656075983564,25.804866935836305,27.389865316672218\n119825,9.370425610896302,8.827826483606964,8.710169587862977,8.714496569752255,8.517307049458847,5.751991914784006,2.4625136470578872,25.86320366474669,20.39434216567138,21.325909534387396,18.67247124116048,20.37290451986695,22.747255105270174,14.50187030915947,54.58402522549655,48.26902241479073,43.65179124436995,42.00388995232833,49.07048722883107,40.69874138664282,38.23145034121831\n119835,7.267853807058112,6.927942387075605,6.840946954493202,6.894657670889021,6.76242764846122,4.807564469593182,3.0577853306263765,9.400274171129267,8.951865130738252,9.37959419705812,8.027214012841297,8.194751085654165,9.006790871890686,6.015529822907875,19.117146750078664,17.765530533842558,17.02883964193782,15.84104276340309,18.03286370414528,17.005847802400364,15.248702747796738\n119840,230.19276861343036,225.7417824518383,220.36781718036414,220.9426319811997,228.89921352271372,130.34010656435336,130.31346213466534,255.11165110716377,227.26163939356007,224.14356027315284,206.50316599183117,205.2822115450559,133.2700444748,147.80516263893338,379.24038555893867,371.4739544040855,320.4415507282134,302.4181667131886,333.96816377791873,224.59785722135572,228.51513831193333\n119858,13.015573400049206,12.793897719829582,12.326295513170288,12.27825510052727,12.04818743287935,9.032933720445973,7.396516956499797,18.272865832077425,15.730661547444903,15.826355314991401,14.157354242722006,15.148856763042005,13.877354763838076,13.33651854097815,29.37761044607279,28.538740110244603,25.275630605648843,24.355449006125514,26.866650429598774,25.20958577342156,22.59913703027022\n119876,363.4011008921466,352.01607626413,343.681189867547,341.2588243971023,323.7164172859946,202.24866309369662,217.30050298781796,428.1300820881645,405.3155796505228,413.16453285009146,366.9334150898234,380.4076294342207,258.46070546274956,251.7386500021878,752.4694691842589,712.9461526287307,693.6128957046755,650.1403429454177,692.4794950200885,445.10294602139504,451.97913827884287\n119892,95.68845321260099,94.83470381439767,92.79015802175569,93.08707384521607,92.89332969552876,71.97809787013355,71.72158376931195,74.09916404188144,74.73646307331936,75.81504920391029,72.3172003917655,69.05657053623885,54.83102599207805,51.15949303105827,94.66705566035716,91.18507223687912,86.40057801307991,83.21369544723231,84.6565014594311,74.92273875703087,63.17185892653302\n119893,12.18100742170734,12.085218886791285,11.147567407540269,11.165988499536637,10.867822122450999,9.3819443997452,3.2126849083804228,25.28019953587907,20.249690135175708,21.7423832546344,17.428139008751643,18.651502420496787,25.851862109838034,10.959287011939441,52.00704218736158,49.38897415034461,44.25690322875633,41.518364226007,45.81772184058389,50.690899655517526,29.915288476017995\n119919,181.58501211409288,173.43336459722894,173.15218796956452,172.43147555742507,168.58245380095613,142.8523173464976,116.98915657207343,175.10269617655382,172.0789343211802,175.55453836946864,170.84231340256193,168.1948688893576,151.35280954885843,141.35068436179876,269.04565434941725,263.63314014642623,252.576505919672,239.8163064265163,250.51478859449736,242.1339634540037,241.69745725009074\n119928,94.64129546890508,93.20845530644696,90.38799081776277,90.62780767883704,92.13643571715886,86.6914725796422,70.27193910581572,106.53162599893696,96.57719762708655,97.90471694054234,91.41454658702442,93.88494079159327,102.05246431163086,77.35509371693777,146.75592890055142,143.34209033867296,134.14599081076392,132.1497690182647,136.3572349995962,141.26427164700982,115.31401247545666\n119940,208.82963937909307,197.09490341016095,188.7755516070839,190.8841980419289,186.21931565134818,115.70677264997059,120.65929430773443,289.7428855394894,267.9564932960974,273.17676192853486,246.28574975206672,260.583379127527,180.6385405304102,179.66041002666594,541.1300170968409,509.9960184830533,488.65965290493887,455.70839470427177,488.23266073918455,283.8666757775218,296.3253363211578\n119963,17.306424445745137,16.953333195236677,16.218298225488763,15.994635811753204,15.734676676383115,11.64321435012672,10.539496568943255,17.753958371878017,16.728996357071203,16.97741769245387,15.685879244020473,15.512580889090593,14.619226802696577,15.169264741822744,28.930954203712638,29.222295098559133,26.59409232538239,25.18514791547887,26.617866203857005,27.317419490444955,27.28665085667597\n119968,188.6482577539145,185.80971616541882,180.50125916547725,179.5803399893245,175.9761375521074,151.05460968486386,132.92152919712086,175.5403351292363,165.97589104045844,170.0485650166696,163.66454161772577,158.25312806000406,155.3664231715983,112.5144200875058,246.9478144081901,229.2059702415774,214.28441706703165,203.9811845080232,213.73609602208026,226.71325323513727,144.7846456367897\n119969,41.70070566287078,40.2648737340465,39.35152447480394,39.288053199274366,38.60336047812807,20.731570690745237,12.914022958503049,51.94134977570696,45.34240023622952,49.04134279747372,42.05463823944976,41.6135638832339,44.64644263073227,18.118334866308047,103.2210503514276,88.75405273667712,79.52925111966948,77.90178085342772,90.69927154642814,84.33893303428762,53.433002797111705\n119975,87.92451520016242,86.61102136950669,85.13903303698015,85.03053415510793,84.28945050060308,56.54531451670837,47.10317824888835,91.71269801055477,84.51695827807404,85.40026120467435,81.8907954103883,80.00821592945078,69.12259292137487,49.59550836446761,138.54455603187745,122.38803687813004,116.72647855359035,110.76133225169146,112.91619199762393,112.39871383371926,71.2254327906685\n119994,23.613180960971253,22.781091676574828,22.366888167454395,22.32735373572823,22.532287221250435,9.979584746182622,6.617439065025638,17.29632801405154,17.156732177070417,17.27134530111024,16.719099620631585,16.112902251260987,6.166343309919557,3.596915647473259,19.81999379708207,19.27956374458407,17.148269206645192,16.014240584518213,16.485915392598017,8.3989059112954,3.934092439641337\n119997,2.637125329238962,2.4740785184429166,2.4732644949973452,2.4849724260570003,2.422500148100391,1.5045438706803789,0.935323267017919,4.763817939438254,4.322843935246576,4.540542913360309,4.015919446978877,4.366576445665911,5.103920881336413,4.141363020821544,9.321720480718186,8.791211091478948,8.695782010376684,8.00360568333785,9.177583618507887,8.792812636386042,8.967752129124118\n120000,8.671072205767537,8.366542888662792,8.145769206344106,8.13131301072197,7.998566125729263,7.164329341012769,6.44061463201572,21.33821070932561,19.169106230028554,19.43134613916355,17.32215650104724,20.059452101546324,24.03373911081845,16.917761045941337,41.132661355963876,38.436715799370226,37.63133444979763,35.80497868073805,41.55149715777249,36.18171205017169,34.90038427352223\n120005,5.126349354458467,5.090217828863682,4.997324452832778,4.991024468888815,4.877844490926798,2.3761223486895897,1.798550748179261,7.730302341099435,6.727922655744043,6.471461351625293,5.920708832347849,6.085151184055872,4.813518278787443,2.9314644158124787,14.163238332437077,12.357099367644242,11.59839780289772,10.904843434678584,12.151503885711014,9.430610929411834,7.833968082755086\n120009,447.6392671536878,423.19507224672543,419.4928168532066,406.17275636249116,387.09770935332955,228.0412059429676,243.39438543213564,824.407882823998,725.8395816645382,743.6210395874409,663.7116137476048,706.6997869793777,671.9399673522098,681.7549717968998,1612.4817831384905,1476.1331519786422,1419.9750015907462,1309.6470421732163,1407.0557556213387,1116.7328963577725,1163.8972702234626\n120011,4.701511830638701,4.468739791935209,4.390420787266248,4.3911803870401656,4.295052403904091,2.1668120723952535,1.356776867820594,13.849957763425293,11.5301702252595,11.948856687427826,10.858784580367761,11.946376189709635,12.936331892585102,11.344251303627095,28.626894584486028,27.746721052956122,25.65414071020377,24.055646096505235,27.63674045227155,25.332926652053196,25.32886657011676\n120012,19.13056476063657,18.0065510229072,17.53171557565495,17.472125066112298,16.99763677815089,15.08134862267947,12.802134214779498,31.826312442267167,28.791700801246435,29.45391398431584,27.301668263784368,28.84668749153374,30.60714126980314,26.704148415595228,60.928594919360826,57.705685477538545,56.256931783183646,51.87082729226907,56.90837293580871,52.65942199722444,49.88939878811052\n120019,46.21251879734561,43.893567946100426,44.60500956534894,44.6368775937472,43.319486920989625,30.48792708260203,28.361462614367994,47.70972792523662,45.58441698931927,47.77637937241674,43.69889053387533,42.56363796999822,35.54050820802662,23.153324714774623,89.23798384481726,81.43248230426053,78.82651911035641,72.73322862306244,82.79726790920968,66.70510763446991,52.75248169228783\n120051,3.677677562342192,3.499844861311842,3.4290675025279427,3.388849701124483,3.3072993432764775,2.378563754513542,1.7047150287952366,6.40507526636235,5.555628856002867,5.421285006959569,5.006989711534014,5.084913980299873,4.882352301396918,4.309712668739178,12.42617884874114,11.29592646233123,10.474717899390903,9.875215765944773,10.508248053281076,9.75772887959843,9.274365639851752\n120053,47.08878095053972,46.30286690278438,45.841510522824066,45.863959469297875,44.99403154822004,32.77941233557175,34.09909624775127,39.08435109353302,38.96652188679082,39.81182496942516,37.398928874605026,36.2858957495303,29.718895401925938,24.87062952623235,55.29965476474779,51.31796673715104,48.61206897323307,46.92677682653872,48.672995524162744,48.45997141844127,32.22902928698863\n120054,12.30509735304645,11.939349790591207,11.598690688212042,11.551418406348997,11.335177637084701,9.882329897768342,7.596672668303158,17.790655169154103,16.043194705447984,15.90716897326349,14.83744734839272,15.875060188381685,17.114723008496195,12.066451320728756,29.806924595514488,27.671015899806658,26.32200668704849,25.03623136184902,28.1788277971592,25.968974908126466,22.674251247236075\n120062,37.16494153555055,35.21379447994133,34.65094753379847,34.621478572812315,33.79820795420213,27.11384848794198,22.653648146307123,40.026493696911274,37.55718335432334,37.94065975930582,34.51479585799883,34.26358647896433,35.03436936198846,22.999363336124688,72.15309006333055,65.24933632810391,63.273115983403876,58.85910448892725,62.89659084801466,62.10780770348133,42.07638800346653\n120077,11.099562014488082,10.405189836129368,10.27846716407256,10.231662942837463,9.99472817775826,5.42953869855008,4.71438347267754,13.33356999488932,12.569013193154426,12.912253830338106,11.687951999601344,12.133766724616827,11.762569005082746,11.115349894169649,23.178821029016728,22.20086564290517,21.05278307106923,20.067375056795267,22.155741441550287,20.965684885210806,23.063836385607793\n120078,11.413053952431474,11.176916009999532,11.126245459573692,11.138321914936462,11.012418478339628,1.9813145470259843,2.1029629129286014,12.765350633968426,11.458175377311628,11.627680324731218,10.829864882093174,10.604111380736342,4.912879937382549,6.239702507008057,18.80411383017599,16.272965119870502,15.60290528246365,14.932233509436823,15.81816646982212,11.102874114633837,14.140237783252946\n120094,18.081074982328822,17.41397561167636,16.4640161408578,16.206883545036384,15.906953960862076,12.11477364189799,9.102385643615127,20.01418633193591,17.37396009226456,17.543358912653037,15.836852738514176,15.675280932569713,15.079186649375444,10.292567199351922,35.612998350256916,32.96536727368203,28.94844675937099,27.20913368952309,28.776185786072887,27.61169572171054,18.276243640490566\n120112,189.94671438949743,181.8029093642773,179.90293656914815,177.16501381387448,168.71410976794616,81.24644360317413,80.04772987813747,261.9866317460451,240.78329967698087,245.15772785628366,221.69943778309766,232.9648764815706,145.62256798109192,149.33555780509943,452.6264272799705,408.7773132955094,393.49317643259656,374.0706563731842,386.7314674665887,245.98708159455123,258.9474378734418\n120116,22.864052341583537,22.443088112477266,21.822905191174065,21.681700206506125,22.296004820058325,17.27563316592371,17.526959325042288,40.16301862899432,35.55247947210693,36.14800044818375,34.129428476011725,33.95318112827037,31.0280078010488,32.526566021386394,61.2711281002786,59.93957821942187,57.58976788918181,53.362784306313294,55.24059186659821,46.437898896770484,46.59782164649456\n120127,32.150474018307726,32.01568600763033,31.194520085155233,30.84741979680301,31.23659212437305,19.57899715523683,17.935520017549866,35.535627032211764,32.201834247864504,31.70035259327949,29.46991955248555,30.450879634453308,23.731507997439095,20.47011898868418,58.41645873118157,55.3923037443181,50.196178896076574,47.23417246167935,50.224047064972545,43.80260626164798,32.92747361211971\n120134,279.5961121889362,284.8399483801555,268.78798503817177,268.3099812770787,263.2088070430817,124.28024475133468,119.99929970071014,485.44853095564946,425.32083100150567,433.3799459104405,395.9165758001601,403.5576804405695,305.5354257346719,306.2646191554496,931.5578816644534,867.8529456911674,793.3718314310472,736.9026742565454,805.7099538122045,557.2629796689807,575.6836655291329\n120145,65.7675434718199,59.84924878215557,58.60723942460797,59.36508404721813,61.1344247917873,48.75511411743121,49.92137519953993,149.51957054158072,133.70906626086244,131.36462789466054,122.74792194908329,135.87754430882381,119.90110047456842,118.9814851505854,265.9528901973964,252.11676743239735,245.4471228653565,229.44087185554585,245.09584678639794,184.11231927864313,186.978512743538\n120162,13.348754532939788,12.555008206468068,12.264197731961536,12.21202586371829,11.824333525112573,11.204835101031087,9.848404279138144,14.976652420554556,13.840721621615918,13.981982394281863,12.523216912034021,12.8491725462875,13.012999463107269,10.376807839525826,28.462955629172857,26.19763057288818,24.627874917099877,23.137590401327742,25.2952028875367,23.78541558179195,18.816675380814843\n120169,49.51036300519287,48.94240556619815,48.16713117289917,48.34805178771218,47.50144913536438,30.82098429230476,31.017764703327476,96.08369137629028,86.26685581673858,86.3738840553285,83.17067665277784,90.54196013445954,96.90454343520118,73.63562060678397,171.4128541722845,162.39432378837176,157.09772242800497,146.7696853235677,155.00500725407818,160.26168795147922,126.14107729414383\n120170,19.996502231639525,19.362294679309056,18.95845538609183,19.06826775640253,19.28065491573821,9.722385203389178,8.039705734002688,22.303846143986508,18.866595446683448,18.75189637264992,17.27418176321472,16.894716954611198,8.528826886177173,7.214205939947319,35.89445816105832,33.00827795950447,30.373590577290766,27.476702828661143,30.505853348595,19.110783746928423,13.482461764003942\n120175,30.24733298902938,30.26385129821584,29.554014206500888,29.642189586750362,28.910902105502572,16.435080434000366,15.446113309790665,52.86863023191945,49.6920473940143,51.15989237126668,48.09972354938455,51.24422541683878,35.765711310321976,37.1266379766148,93.01207823672806,90.85718934980608,87.9830599339779,83.02461623592524,87.10030193828955,62.4177428063904,63.50326292854221\n120188,195.11792869723718,193.0374708085054,192.0729242088005,191.76538916946504,191.03830541959923,132.63934629828884,123.65248176526302,201.45760222268237,193.054381757749,191.86743942111366,186.93391138699891,187.46631392206953,161.01320056195658,142.84630877283115,254.82145468771446,244.152406552084,240.09866435474063,233.30105240375465,239.69595846307104,213.52655999040914,193.04325740674037\n120195,11.860140509554938,11.36256180037615,11.144989753697683,11.048222061458109,10.863970819349008,5.461236561383879,5.56868304455304,24.382544709823495,20.746201750631535,20.34588312249598,18.148572731843487,19.232113012651165,15.532070381106422,16.812338485163355,51.03328726895836,46.144693031456455,41.64480751168646,38.82318909221819,42.49735670263929,34.5484464211544,39.2171806608122\n120204,1276.294467973535,1253.7623095404717,1239.4484996024391,1241.7200909763221,1230.332346596985,1049.3804457147166,960.2808592724715,1156.7641876219902,1146.9486402367029,1157.8599279175194,1139.9325419558159,1111.5677095746748,988.7646892776716,924.7630667675359,1326.1999287345595,1288.9119824924464,1256.983135001053,1231.2348996191183,1252.1308049378028,1183.7705512461405,1088.3628640177349\n120207,15.155064535229842,14.966877875689802,14.741659063460276,14.769572277746184,14.53129998884991,10.377263803681233,10.549201678311867,17.69164391699528,15.912482089046668,16.081283337320855,14.974297944735984,14.851203918194251,12.425920221449745,8.950041255306106,27.20993792546973,23.73209045346548,22.726114132076432,21.3375602025123,22.23117745169398,18.937223611455707,13.228712497230388\n120232,106.69782029511966,109.27037805907737,104.69896164874339,106.12319303882344,107.1341754979245,66.77183265522766,68.1068385031808,168.15709085287108,155.08073541747737,160.04973649874458,138.83142214066237,138.0876701868277,109.59961453067815,115.62033834089905,340.87484846052945,365.8632814978349,332.582433084651,300.8158376779122,330.41162654490313,288.09283846077045,267.91054122868826\n120236,13.044372399357085,12.922311576686235,12.908401011939375,12.91367485143646,12.717129330632917,8.004683183374611,7.584284283009157,11.051137852809404,10.976955848009986,11.156709866019243,10.88555575685846,10.520112427032462,6.513570418258018,5.6997764714565555,14.99211942692987,13.352597052669134,12.778157305376576,12.46978685751072,12.6122091990193,9.851208990544986,7.805304245266317\n120241,84.09858807261949,83.25001192796954,80.16348807810375,81.07460593446032,82.052685609823,52.33929712745608,52.328385956140195,129.38120209096644,120.16911115016691,115.73125757143723,105.60597627832904,105.1215984289969,77.62388588727416,86.50269284955463,227.75788981738856,229.66539627502212,210.01955929384224,191.71922534561682,202.3035439086338,164.8893049808161,158.94522391076404\n120259,5.293131501004938,4.93092366283519,4.831180692724026,4.853018653380202,4.725019661751004,3.350999934040549,2.946344603313344,7.052016946038476,6.285522488470878,6.302230831383554,5.545314282211319,5.659380828941652,4.856654873474361,3.0131023942054274,14.053427864216705,12.445312136525272,11.778261637532525,10.979137564331038,11.873002787692055,9.789658995626922,6.151481828457591\n120260,12.559133766342923,12.518966947171641,12.437765661748147,12.38524518056922,12.476162258745147,7.235771846514929,5.329489055405397,14.295474038394559,12.884284203954678,12.377110369298737,11.458376757665222,11.060326992900384,7.576907036776925,6.015183296536572,23.317337007577308,21.74051509787514,18.768368666482402,17.218517220308925,19.2537744566674,16.717061055614522,12.937012441898256\n120280,21.012305355171012,21.06598316344219,20.28682706157096,20.452435286269306,20.395706331354585,16.005497212170663,14.152426104695872,16.590096619634828,16.29488232196129,16.162764809998308,14.85872180840373,14.09173809811669,12.412645085368657,7.593951668906384,24.469266030347814,22.037186078021552,20.437262537275373,18.79855786401973,19.113913756685218,20.660707406925145,8.6536703748192\n120282,84.48187262408858,85.45113595349804,82.54701339739138,81.99124643405203,79.14141946032207,58.27362609101004,57.21613549214368,67.14034185221443,67.3946215024552,68.36498759261359,65.18079327780687,60.936552459768066,48.86959008954359,42.8593496947082,95.97982644532478,93.15380347615444,86.5853889167312,82.5972551115352,86.17922999301061,86.71695450564943,68.61534758458858\n120291,14.620245005362536,13.856389306557222,13.683339994609137,13.785950845260231,13.944852645543492,12.082621924555419,11.100563740743441,23.03866959626894,20.716938081825138,19.843107697271453,18.383490128145635,19.73783729406026,20.884013524501235,11.802538982699748,36.61265881898222,33.54881950858979,32.97344169956251,31.66689881931112,33.26110959796749,31.91789233976696,21.17643657399876\n120293,9.087018761171059,8.98549413104525,8.686293618234556,8.737621655593061,8.798631940015014,5.7180515289425164,5.350525842726523,11.023556633095746,9.553483042364027,9.067000016519323,8.333392483923879,7.999485563797836,5.7180915558929195,5.238844031372548,16.888527199782175,15.838917217128431,13.822652783289909,12.756496144364297,14.057479160855722,10.648421416988542,8.52643310960405\n120306,1154.5029828723807,1138.31184091008,1132.1137756931203,1133.9102691791863,1125.156835134854,960.4051997251407,917.4726345893837,1114.9873581623592,1097.4793224182774,1105.4757368674286,1086.4586131517794,1083.6567127830442,992.8547445949386,917.0977934239329,1297.2634606627564,1252.7861978338456,1228.944508400012,1201.9297651077914,1220.878614055528,1158.969612093741,1021.2726147462623\n120347,169.82897426617157,165.14293258207408,164.072703092543,164.61602977767433,160.9543260392252,111.1172011804784,81.56103433239896,146.09152877838955,146.8235831208728,149.33792033977224,143.50978202025192,140.72075015637014,106.68814563106625,109.11291841096117,195.31802855639876,189.92837597416624,183.5471807133235,177.55879377073478,182.13856411058214,168.11430124283342,179.23068150095352\n120356,28.246743818758592,27.35859444544803,26.84128784979441,26.69931559148998,26.14860273510485,18.510055871040777,16.69638832176385,34.33417512720317,31.129893865894754,31.141332865032787,27.980832958662184,29.19091768952313,26.984533937044127,18.547964602141928,65.54437448930418,58.19730039581551,53.619883046764656,50.859422185080376,55.738432794932415,49.55143556698628,35.591052729937516\n120358,254.7367901100395,236.3589622018,233.71488044385438,236.04829923920965,229.4580174002438,141.75995798150967,146.268280227241,373.9479245279275,347.36280022102795,341.262067374172,314.1819018307005,336.128884936108,267.98564545901945,281.0529774668611,595.5924355186021,570.2378604530664,558.3497022403459,529.9903405628647,543.7318214435642,367.0307383270719,383.15938346969506\n120376,209.99712745092765,196.6668041624826,192.79373749166254,193.1408085066839,197.82963242653642,125.4810184902383,127.08145332162483,312.47728292434584,261.666972555236,262.0223777541459,237.41882795630684,234.67566094179728,187.708142322904,223.1638709238713,542.6918859740066,534.1310575806023,471.84354496937596,425.8729840997884,443.2866582831233,371.3955313004664,368.7379711834512\n120385,368.78605786923487,347.65973931620323,335.7559285895205,337.6567477870107,342.1695069058282,223.79189098716455,229.99587130182772,638.5398987048279,586.6297401281761,589.9970136621487,549.4480253900772,591.6993973613011,458.7687240666349,463.7066214733893,1066.161556130417,1021.4870387081841,999.7775041360925,921.7304935338368,988.3420076664349,661.0994160958412,682.1466205234196\n120400,15.178352489263311,15.346939718193214,14.952808544673516,14.932239301650434,14.966089248544144,9.36105317479123,7.256466557397879,22.179246935767463,20.113504463144277,21.30593159141256,18.490165494743,18.49445116775334,17.751076987442296,17.422264769519927,38.099265607678454,38.82643660380751,35.30403186494775,32.57224665449198,36.235815814995405,35.18776572344898,32.677276767705074\n120405,6.110868137819882,5.8094709798926285,5.582951796396424,5.6251213617980635,5.4579124059094895,4.861084801624874,5.107337188912605,10.24960729661102,9.133759506289367,9.30451417915961,7.726942361420614,8.494726045314149,9.205484341769694,5.133333154137015,22.326809760399417,20.182460456604012,19.022067781634775,17.90899069664704,20.816761873250666,17.003158071400964,12.174711229684256\n120416,220.29928417791183,209.6764609072837,202.77913396400507,206.2362394694041,204.14801347537852,146.04080015849016,150.5719990046679,279.13068812816573,265.6257775410452,261.0665744593409,239.31888317447886,248.894735153197,200.7579833242817,201.70214783940247,430.0869695044553,416.7967940552529,406.6524547554341,382.7211086191841,401.81910401611935,273.7822366127785,279.4529322732531\n120423,21.21960826124924,21.10476820336081,21.02768144686948,21.01089491890179,20.739839608098972,11.85817610710931,9.267017432364424,24.949443013297028,22.853033717269685,23.287390956848405,21.497483230646473,21.43443532267683,16.488242026142316,14.263645903465994,41.06165529143662,37.28549225073305,35.23856174069554,33.75245762794003,36.40312134635424,31.501061852376797,32.54248942470108\n120432,16.19323952073209,15.715547320869918,15.563477560182239,15.63646509498035,15.52076067263661,10.830844259642122,10.53343248186314,19.241328704616013,17.591234176748852,17.399077279130594,16.368845472805848,17.39300483090878,17.3508133254567,10.150212471119378,29.009496345055325,26.375265953887823,25.868454389070273,24.57552460898299,25.86697059671512,24.476527996216962,16.801757061360192\n120439,48.87596796799316,46.24904291108599,45.054238988024835,45.254075895742034,43.86258247149959,35.181763644062265,36.901065265602604,77.51084264110595,69.65771121281207,71.26396009403466,64.17181742670085,69.27355308242313,65.05907531892845,63.237226991054634,131.95712753655454,122.28601089353809,119.46092265645471,110.82633634201487,119.630726915005,97.94442064077583,100.05274161620883\n120457,309.4346928814524,307.7863836026013,291.0448063276496,296.14279766053244,298.8685092323277,208.61912048214688,208.97770422276838,505.8712929192571,460.7601936474634,454.59327890906593,424.71784240457094,444.1584677121007,351.45948415245965,365.78527434800066,792.4579138189009,781.7258258139052,745.6615142827627,719.0291386965265,749.4538344909137,491.43681986645436,494.40054510791634\n120463,194.0062722079887,189.35788915417928,184.67973243368206,185.52161988626517,184.37067402951504,121.66487599296633,121.64144808899219,261.62711918231327,246.2784805296898,246.1665090168068,225.85760623990356,232.46878867592244,170.23490615945016,171.64329775446313,420.34688059771344,408.9965432803733,396.80038595481943,367.2400528779493,388.0055149732447,248.65377121330602,254.7643501675975\n120468,101.19203373010777,98.62903603712684,96.95021055153714,95.9365541831244,94.3213636830248,87.77407674188129,74.83586805074712,101.14013374638184,97.6587273778573,100.45601571491135,97.27812931428768,96.10183944989078,87.93037624367068,88.82891287657256,170.2440748015465,169.6741764241411,157.57702444851654,149.2260899889382,154.53919494551928,150.43625333272027,155.7696557938179\n120491,297.08984697470044,283.3849531510151,278.1597533825805,279.37811777082015,271.59459530123627,197.58958969554587,203.82137889438124,288.44687001703056,272.047880658226,277.9762811701297,254.4974163338813,263.2268208451559,248.11247570632258,190.55355890989577,463.90453266052316,423.7287972668617,412.87102640781404,391.2433432148971,411.0937824036314,404.2620931051853,284.89379692614307\n120497,16.950149863153058,15.665998813223073,15.122697463390681,15.047294068616301,14.486306340638855,12.369059403588725,9.701286295328513,26.447164330053077,23.952445289259224,23.911935588912254,21.16008245448751,22.718502255356444,22.69486054106476,18.30663449262583,56.69626824956913,52.92726295343471,49.66917558017296,46.75117353834076,51.68103401660409,44.971681913696,40.44232486337169\n120505,39.306710745811856,38.40523403448927,37.818864237871225,37.85918036820771,36.77958627359859,20.073691903434657,16.224174961748037,50.690448066395085,45.723261177121145,46.95361351714983,42.124244621495066,44.240046441772414,37.27660246120567,22.470627718039154,95.86327759184452,85.7951945022024,81.04073732495557,76.60416272578662,86.73448527634972,70.22213678213849,58.12550029395223\n120519,4.129177588489307,3.9942241119144226,3.8707627116098307,3.8432567951478998,3.913327269093097,2.748939096918937,2.0817932324858472,5.824559918372654,5.215301184252618,4.795543015615906,4.577578336976842,4.503541657263732,4.016079097882666,2.58550411601557,9.775764519208584,9.400241481333488,8.699859737368241,8.065046234901976,8.370214643940459,8.01115096695807,5.250757258401042\n120531,146.62566910499322,142.59912012111505,138.89868062482174,135.98875434229416,136.04404354522637,65.94959878555652,65.31359930601408,244.49125831507,221.77200735956032,221.05622277944568,207.08814487917115,213.10795142712223,151.0343463338489,158.8010431314371,431.6382295589636,398.2529994978749,370.2347177172288,352.31081398820714,369.1561295116324,255.04710966825402,264.3572635536171\n120533,1.9667571526188672,1.8067265634618477,1.7231301300289101,1.7404300299128055,1.646394167035664,1.2706601123088594,0.8478128546983696,4.025533273025503,3.4637630823007033,3.605791976248856,3.2547763658245223,3.4982716780612537,3.8963263417429013,3.6064465147469713,7.823068001146249,7.510801816052423,6.914380623764128,6.579235307766064,7.466537164535016,6.6786090334692405,7.04093923964399\n120536,71.48953532175169,73.31502921786836,70.56217360252725,70.37052943311856,70.74883676907854,32.46297688176528,31.175505976060162,66.88372133484056,64.82212599220192,65.90626490418471,60.84055239715696,59.99480873112812,39.09863315693107,42.74162356634714,89.9330487424393,89.43691926574803,83.86722707067578,77.78055352817024,81.53102594967255,73.03836611570952,75.79099546942093\n120543,36.965960158839614,34.72023121969389,33.7898452039239,33.44455993383335,33.76977013020445,21.898038688566626,14.042630343239873,34.74720411114587,32.459974945808284,31.509873032917813,29.297313263727318,28.375233483097027,19.873723252775843,17.23324489274617,55.351972679575155,54.526145885793795,46.38732551780795,42.236854012012586,46.770696149581596,40.382797248290025,37.04235974058799\n120554,21.84484659146228,20.89257448904342,20.619375250806435,20.66182702525416,20.248147625647434,18.554940923661793,16.1360768123814,51.725685037764016,45.15759580550693,44.95681601982034,41.5219564338509,47.35616935684522,50.27147432187703,41.64656184368553,98.80377513268708,92.29791746692229,88.74965167798096,84.18565994406848,93.10176825358893,82.26668078165913,80.87795340838649\n120559,12.368187040411518,11.662799385061852,11.510983792320904,11.585944730178594,11.232987204046184,9.723560751699752,9.650101699853929,21.273026577480618,18.344680320392857,18.819781302031327,16.869098530657293,18.278817768378904,17.06520944714187,14.67665326008567,39.4598143376693,35.83356999255675,35.0446656839476,32.242472128953544,35.65819939533628,29.144085236782264,28.330173392376683\n120572,6.468456418980057,6.057222778499924,5.740310049795486,5.678099088288499,5.568613439316109,3.78211037682746,2.484685061182976,12.519609154862952,10.062898519158628,10.504386494961519,9.078176187483084,10.419934493093118,11.427693910528628,9.545950184429703,24.350026671476463,24.223000192239645,21.233451987383447,20.470559823135385,22.02320884169634,21.03041163868885,18.865692813837622\n120589,15.464717452543637,14.428467707783799,14.200929042452309,14.2008748539646,13.816617085271416,7.098472474576299,4.603120447627576,22.227884368564073,19.755170824031758,20.414318131806674,18.32356319577472,19.259487939466275,16.35307342385976,12.042481280835625,41.44845844891485,38.07967775675618,36.88530394724529,34.06325128681007,38.1271349753212,31.330757900040666,27.740435661049926\n120594,170.8979175778299,154.79340633984862,162.63596257951366,172.54527543142743,177.6887057637603,179.45554074680047,140.24596326550738,289.8595388579273,275.3902540121151,258.58293711753817,255.3022380184961,284.79498367234874,300.0215239493342,258.95188061303196,416.3784327667281,409.1355806695409,409.4891176161789,395.5498700673863,409.55072246823363,413.0053909863213,366.13182166771895\n120604,38.908706836269516,38.679616932939616,37.87915620195372,37.89866644373412,38.44162519653339,32.27274762573481,27.085415032185978,41.222497115054665,37.726763045298036,36.24162383014194,33.14511407403046,31.461983999955407,29.75842143440128,27.730144300791366,71.02059164297604,70.5398489135275,60.06020741661192,53.90559578988317,60.82190143086011,58.66529914306658,50.4649134560477\n120605,12.8348764153508,12.098762407355059,11.894377989205083,11.83361791984632,11.486322438144121,7.407657085866469,4.378217936805696,17.837814725936198,16.74519667772555,17.19441236115275,15.877008256126723,16.49096791874637,16.490435784635494,15.615956277300869,33.22501722558322,32.51914099069771,31.152401492540868,28.91002334217196,31.376507137752405,30.064203723036172,30.194606985235374\n120622,222.30075376997607,215.74428665219705,212.6399704583815,211.14708646773371,207.1359181368965,210.14057144140764,166.23682569050956,176.96834411652102,176.44665723426846,182.32467117348824,173.9900429450278,169.0590345528651,170.45180294709547,140.41617127037867,217.40410102274126,209.547615687102,200.63132587106747,196.5065282632911,201.9694937716056,206.22213829605832,167.9980996865622\n120625,13.951864838632007,13.783309361360903,13.258005460953383,13.12767319405519,12.917302248891572,12.19025940102423,9.0587796784307,23.334105712944147,20.51241251020776,21.276734833619216,19.10378535023572,19.59791717912795,23.090990978732435,14.608382455108856,44.68982614207701,41.33644456369848,38.38951093177651,36.85798738996677,40.79000182916168,39.39472289721589,30.30227392586247\n120626,44.90830192373313,43.655100309207505,43.3857480525509,43.300770438147985,42.769329406029904,23.287451071550002,17.212347900293913,38.30322623342245,38.3021906445954,38.796864560468876,37.87603964799434,37.12131634930807,20.34256555416487,15.842082958841955,41.59746137838333,39.693804515805056,38.24558825687862,38.09066651564235,38.31137349972802,25.343520388294234,21.563630657619218\n120641,7.87170715990442,7.73526379178339,7.64814442162109,7.624999162349735,7.686536567331225,3.7743877101363648,2.3836893811992304,8.581416994733896,7.668098367951701,7.556283537869605,7.1130300160285245,6.870876837100184,4.448299041066978,3.398457494914652,12.727407397825063,12.063532682856481,10.230641844356656,9.274193957318218,10.17804252504367,9.213404222511912,6.665821530546604\n120650,15.34125522204512,14.548241100714975,14.212984322041114,14.189174210549135,13.922372394576943,8.832684835324601,6.756759411885995,12.529829793340342,12.480456837826363,13.006064810918614,12.036204261662842,11.746907095460134,9.110280206118665,9.268693712640525,19.589216493339233,18.82192401470711,17.224332191259716,16.63659901784436,17.303284941601788,15.903942960334275,17.373397760753328\n120653,59.049722127519956,56.88065921816978,55.76665978929172,56.01797695297832,57.033439343885796,35.58092456167789,36.05479726638272,100.76741756253031,90.53957406685497,87.57371187257247,82.01832655832166,84.2284430432453,67.39108687752532,69.7860299632171,185.3767020220778,176.13594804427336,166.59037443553723,158.53916529233175,165.63500611295663,100.21462082453448,103.11181691450139\n120677,3.8839465859515063,3.4957078845458702,3.5141288771705166,3.534285998776422,3.437585897516038,2.7843079910147592,2.8426419481470875,9.986830574748092,8.593712128984823,8.595175692963384,7.975312436089932,9.594408754106258,10.566308815450867,9.066068972431426,19.38612227728017,18.422326300150015,18.201126541768787,16.721441241983452,18.383056760764962,17.27385747283842,16.87298335855944\n120682,28.836225069103207,28.56782667073036,28.071837998532896,28.065378968196793,27.553430647636464,18.228220800036503,16.029673648873107,40.5758569456124,36.263187106095884,37.595961002728686,33.88611930381791,34.15659204913667,34.544557357182164,30.142866995555057,80.74158496402754,75.35683828608164,68.17887670563397,64.68055765667452,71.93807208556694,68.55688707393429,68.25209996153565\n120688,16.790142343019685,15.779962425152544,15.478243430985534,15.463050866586665,15.544407610531968,8.85578119421654,5.233105906703402,14.874065803136828,13.915857023962438,13.818761303877032,13.262013013222129,12.927231495260711,7.413612962365335,6.248851818932119,18.42125730565622,18.668535109741462,17.36773932655685,15.57223568050199,15.695589209913235,14.124803967826786,12.08235217157598\n120693,5.082188081150103,4.906191377337527,4.742861916289001,4.712779914783887,4.5856640211134385,3.329568033622072,2.820082039247304,11.12648746218669,9.294117960573292,9.27333272758421,8.497341527681298,9.118390111497368,8.787628718129215,6.545055914545348,22.12628839593062,20.060205790910647,18.344678675688463,17.34372160215682,19.180724369656858,15.778231580719254,13.879220852746156\n120694,23.72217367167853,22.697421973639475,22.217606328288557,22.199844543088478,21.715446255241748,13.272810203299299,12.623388886247541,27.573601802938576,24.887855541669754,25.12619029996282,23.09247730255043,23.587527051234655,18.9045100728486,16.42433117285204,44.620947774085344,40.21593165463005,38.026418987784155,36.02933464068091,39.34963635145629,31.530300393216358,30.099979014355437\n120695,6.2048125516055235,5.885520690256563,5.8718672812785115,5.87429677619477,5.714023022620996,4.352130193981007,3.2583769954266497,8.979546600864214,8.44614700735693,8.73360647495259,7.752916097439667,8.168775031564094,8.887944080600208,7.2727430933320445,18.04636637038027,17.132539924950628,16.779711427467614,15.419837130842053,17.46551020367533,16.296124362712447,15.952432275222904\n120700,21.680550109773534,21.409385534796808,20.558832058470834,20.588931594290546,21.124580101157513,18.897920091142037,16.220945554664432,42.861079556623466,38.96357577521249,38.101116277417496,35.40757313156349,39.544704648462364,46.29008024154184,33.414869294797924,81.31256960356045,79.80523336984177,76.2054104890222,71.5609910387791,79.05212354125278,78.41452454754177,70.69022663423179\n120711,568.650327665965,542.0135272435618,530.0261116961075,526.7047958654139,532.7811917194512,506.5197260463681,184.19372478267346,720.3233949658517,674.6144208219761,643.8823556046191,620.1671608556329,631.1037330272145,730.8064343061113,256.6002350098043,1038.9384605284652,928.7024266021917,910.7643649817363,881.129840550353,911.8198616706526,980.2691928263142,484.0373871116827\n120729,29.59814497376799,28.50879784841997,27.967651503644205,27.82381573461503,27.302820232731435,12.07210192298812,8.133803364369694,35.03267343572412,30.295224558420347,30.248208835628724,27.827352233888746,27.16169099590163,13.871958399861882,8.201792067457905,62.270958042041805,52.81909402615284,46.4417973418345,43.562115452398025,46.668563374144405,29.654824880878788,20.790122335372587\n120743,463.3070492500828,455.5710187362272,449.89878626919204,442.73923331755554,434.9272665591148,398.566440526337,298.6050292360724,340.4060893850913,347.4841134720569,359.7480590300951,338.9210183617013,324.5727889114439,304.22729011632094,218.93171362450892,408.0254155094002,400.6979543557178,385.36219778996326,376.24098046623607,388.0767644035427,371.8961722277505,301.34331144843526\n120765,7.9687472481576735,7.406140038769441,7.197504597925708,7.237758057613688,7.020920877441582,5.722926792736347,3.470646140109824,19.02680360943235,16.442816591957303,17.419585529496278,15.366837258408182,16.695281274770682,19.231970653495562,10.588397662089548,39.415776028922004,36.19763027026122,35.18685253032988,32.408035503639404,37.02783540350517,33.8725510486153,25.935880736804496\n120766,16.151620828947383,15.526971991758929,14.988244052789318,14.781815744613523,14.604854700811172,9.234324382469392,6.94075611005502,16.36920458156468,14.905127700636056,15.092367261611052,13.657092639408454,13.6288697602444,10.532056907138783,8.799212770803893,27.07844547702908,25.948899443420444,22.828808941308953,22.2760653321083,23.04788621951896,20.298625116344727,17.38983119423384\n120768,49.27580790927832,45.6607098828975,45.142182853653885,44.63328707865044,43.40452922515687,26.542723360867672,27.67679649201258,90.80763762702243,77.76012784341775,77.50285221317829,68.92570876678631,74.23060737740066,62.19929518437142,61.95104500624259,175.667015757002,150.616678969014,144.48754344032213,135.8707505313012,145.42109349375855,108.08675813512244,111.85419318113443\n120792,7.186037224934699,7.180874689079381,7.013621728388458,6.961627342083212,6.856141615010226,4.375119153678647,4.260680464005766,9.056641378066534,8.163992919849049,8.28708828185266,7.73001362477796,7.836181356684868,7.1757816857055,5.492355903345828,14.288564427746563,13.117342734191585,12.197473811216602,11.62755225284891,12.379583028107975,11.084263202951565,9.300794872479432\n120797,52.36971549282015,47.93514259145391,49.293905798318484,48.88994971848311,46.67286567633295,55.64358153545281,28.404682560925217,45.088084138198035,45.188601719909414,46.97484823958884,43.47189894513303,40.39181265606152,51.62641968234804,27.625045686572367,87.77033376965647,84.02826617614676,82.25248020803312,75.7504226985435,84.94988930209615,88.67429024216082,66.26020246013891\n120804,28.74025888937802,28.4054523149772,28.190403890484816,28.21161971419318,27.59484785466191,10.996087357948348,9.125330924995566,30.510038234451432,29.343164497323865,30.13489600290612,27.993457836070075,27.925846758647772,19.714009968117253,18.443424056548206,47.35418426529165,45.7500207529446,44.10837050751693,41.59718993097184,44.28297858014303,36.82674203283457,36.047822306803376\n120809,4.828867399394053,4.565166171557703,4.399010913806605,4.314912241101753,4.295048254375851,2.641281487281076,1.8639583675164233,7.008366770885628,5.440562511450183,5.566043152087768,4.7619897121383765,4.854369151700069,3.681414104701044,2.507993684964722,12.089737830172611,10.578938685512016,9.398386315980954,8.673948911759956,9.846610621069317,7.621181517718241,5.628571754164681\n120831,141.5785217808163,135.90484825990075,133.52339792115305,134.69455940112496,138.0771422843564,94.88177379204159,96.14235876522314,187.43517440207575,164.82373988962348,161.21646494341493,149.9731907482327,143.87236585418043,113.62728696290861,138.71053521887038,310.6284472176899,303.729321479241,274.82964417628295,253.69198803734704,259.27840335383536,221.68869922319828,215.99550282764775\n120836,170.62561246082726,167.21492627637758,166.040415064479,165.3485918242334,160.69847604457863,101.82161203933586,45.790965308977725,185.56133984928667,176.5941398814876,181.03774399590668,168.56910362116585,168.26502172886313,155.58463461252566,99.36994217858836,333.8866468904046,314.7432890661388,296.1057097352789,280.23549018486136,305.76652676815064,293.58924989148875,246.70244416797374\n120846,13.464793596870916,13.50800694923213,12.695027902378072,12.594764145520328,12.363475842367928,11.075230732004378,10.484532410165517,23.80243656347663,19.527673357912203,20.531965269077997,17.651246717315114,18.90249639861126,20.841170057223373,17.460933254850495,47.43110396908392,47.104889861252225,41.84102361343212,38.962854295251354,42.30194122501951,40.87033019758399,35.662549484139475\n120855,41.5451605635476,40.723442939137776,39.96989264762471,39.11532595145511,40.179978950511504,25.849400730719662,25.633405000407723,56.63793950583362,49.86665512319654,49.34965135020228,46.68002096460392,45.66325693322132,21.554931126131073,25.367731716561547,80.88497672115143,77.39254092404525,67.40968894675784,62.44814083226189,67.31354612689772,32.96212627700275,37.48156066083362\n120859,11.845398824861029,11.283767645698118,11.024424946957636,11.012909535940944,10.789378969540723,5.868413835783708,4.219428866090801,13.559892568418407,12.141658476856255,12.251317715817121,11.242780680389753,11.156430062228658,7.847626952443282,4.856943652134996,21.615291793485458,18.958230090942195,18.18263977841727,17.019800148876836,18.091105149449998,13.125029108119529,9.406817093459823\n120894,19.941940707365642,19.451945687820686,19.00963246361629,19.01879990137548,18.504973436015195,15.612041198015266,16.353452625098924,25.88076228379126,24.203108753507756,24.684303218103537,21.6158868891215,22.634659542762133,24.652383185508796,17.250005097555093,48.535890724534354,44.64105946051008,42.80134879254963,40.36823755244043,45.87613162483561,41.47067872085875,33.406392240040375\n120910,6.3245469133555865,5.939627771663768,5.826657837474271,5.863005099932339,5.676911455647722,3.747314452484582,3.167707199165469,10.442128763938037,9.620419754299322,10.269891503809161,9.238675750414814,9.932776983089356,10.528708045338686,10.362924537204716,21.28566294339663,20.901168080954744,19.85706686527587,18.223408057680544,20.401313553693125,18.78983198789221,19.468134784618908\n120923,11.291531720728448,11.058119561952806,10.731524566262987,10.680321955709157,10.483073735629826,9.829520000260812,8.949911349271483,14.824922532099507,13.480383914711958,13.532251509343475,12.457650114057047,12.335990596507356,13.2831311943823,7.973424444808744,28.974146978051778,25.872046837935578,24.200009311946243,22.72478707884712,24.107003234036174,24.674695596375145,13.49339701006805\n120925,14.816101402590773,14.127232072392987,13.78489313845564,13.686368495247654,13.433998357452188,10.71298144862614,9.616560026842969,20.056625333109942,17.84756937661513,16.800545031437693,15.578349410962593,16.220426273252233,13.691394014380336,10.214032508749641,39.22831375502978,33.641956319523075,30.766122341165506,29.072849607778007,30.93087210816473,25.66967776722995,18.490831621824945\n120928,127.76965500768206,127.42923317599703,123.01555113558744,123.98790276658272,126.63918321447181,78.52704542620089,79.18121835572119,128.3159356115746,121.86742088177536,120.581362820917,112.74295865705669,110.44072180562652,79.72941244277062,89.41962883092967,173.98894418134788,174.68098002478615,156.74068659960525,146.8327281252533,151.92566580609423,123.3311435148001,124.55101880323764\n120929,1723.7605422030779,1695.7234105350453,1645.3239650784433,1645.8053765314373,1602.5421702751921,933.464139620401,674.9815992010545,1545.1247598347063,1485.2250474006255,1537.1913086474456,1373.6144762550123,1328.1555756124424,1193.0674991741807,546.6338009909871,2506.596977583336,2309.7226636122505,2212.372703788672,2018.9845535695972,2139.1785144058586,2039.8564735099026,1042.6834944221537\n120934,386.47297474403473,359.5353137788014,358.16915245126495,357.2133565987192,356.23292088820557,233.35754446921584,237.2407984765126,776.6804281943785,697.9184315497121,675.3682464112143,633.198666415031,686.9641566019428,647.939409887697,647.7536054324297,1302.5432494692966,1220.2413084460452,1198.885880262842,1126.0124221533645,1170.2431015676973,981.9189814338367,1006.239412267952\n120947,7.3048567767403005,7.050597422217184,6.887779079912311,6.805573161853699,6.805281513423258,3.4224678499246965,2.0187429483729216,10.00662080456416,8.49504489013853,8.72943422873366,7.667801119711503,7.871472684049899,6.5072201120717645,5.964507110150521,16.130057504827924,15.585575105612557,13.363660730495116,12.442877008955593,13.796558818293374,14.18675134231882,12.753652154238399\n"
  },
  {
    "path": "docker-compose.yml",
    "content": "services:\n  shell:\n    build: .\n    platform: linux/amd64\n    image: eemeter_shell\n    stdin_open: true\n    tty: true\n    entrypoint: /bin/sh\n    volumes:\n      - .:/app\n\n  docs:\n    image: eemeter_shell\n    stdin_open: true\n    tty: true\n    ports:\n      - \"127.0.0.1:${HOST_PORT_DOCS:-8000}:${HOST_PORT_DOCS:-8000}\"\n      - \"[::1]:${HOST_PORT_DOCS:-8000}:${HOST_PORT_DOCS:-8000}\"\n    entrypoint: mkdocs serve -f docs/mkdocs.yml --dev-addr=\"0.0.0.0:${HOST_PORT_DOCS:-8000}\"\n    volumes:\n      - .:/app\n\n  test:\n    image: eemeter_shell\n    entrypoint: py.test -n0\n    volumes:\n      - .:/app\n      - /app/tests/__pycache__/\n\n  jupyter:\n    image: eemeter_shell\n    platform: linux/amd64\n    stdin_open: true\n    tty: true\n    ports:\n      - \"127.0.0.1:${HOST_PORT_JUPYTER:-8888}:${HOST_PORT_JUPYTER:-8888}\"\n      - \"[::1]:${HOST_PORT_JUPYTER:-8888}:${HOST_PORT_JUPYTER:-8888}\"\n    entrypoint: |\n      jupyter lab scripts/ --ip=0.0.0.0 --port=${HOST_PORT_JUPYTER:-8888} --allow-root --no-browser\n    volumes:\n      - .:/app\n\n  uv:\n    image: eemeter_shell\n    entrypoint: uv\n    volumes:\n      - .:/app\n\n  blacken:\n    image: eemeter_shell\n    entrypoint: black .\n    volumes:\n      - .:/app\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.__version__.rst",
    "content": "=========================\n``gridmeter.__version__``\n=========================\n\n.. automodule:: gridmeter.__version__\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.__version__\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.bin_selection.rst",
    "content": "===========================\n``gridmeter.bin_selection``\n===========================\n\n.. automodule:: gridmeter.bin_selection\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.bin_selection\n\n\nClasses\n=======\n\n- :py:class:`StratifiedSamplingBinSelector`:\n  Undocumented.\n\n\n.. autoclass:: StratifiedSamplingBinSelector\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: StratifiedSamplingBinSelector\n      :parts: 1\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.bins.rst",
    "content": "==================\n``gridmeter.bins``\n==================\n\n.. automodule:: gridmeter.bins\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.bins\n\n\nClasses\n=======\n\n- :py:class:`BinnedData`:\n  Undocumented.\n\n- :py:class:`Binning`:\n  Contains list of multidimensional bins \n\n- :py:class:`Bin`:\n  Single-dimensional bin\n\n- :py:class:`MultiBin`:\n  Multi-dimensional bin -- intersection of n Bins\n\n\n.. autoclass:: BinnedData\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: BinnedData\n      :parts: 1\n\n.. autoclass:: Binning\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: Binning\n      :parts: 1\n\n.. autoclass:: Bin\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: Bin\n      :parts: 1\n\n.. autoclass:: MultiBin\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: MultiBin\n      :parts: 1\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.diagnostics.rst",
    "content": "=========================\n``gridmeter.diagnostics``\n=========================\n\n.. automodule:: gridmeter.diagnostics\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.diagnostics\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.distance_calc_selection.rst",
    "content": "=====================================\n``gridmeter.distance_calc_selection``\n=====================================\n\n.. automodule:: gridmeter.distance_calc_selection\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.distance_calc_selection\n\n\nClasses\n=======\n\n- :py:class:`DistanceMatching`:\n  Parameters\n\n\n.. autoclass:: DistanceMatching\n   :members:\n\n   .. rubric:: Inheritance\n   .. inheritance-diagram:: DistanceMatching\n      :parts: 1\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.equivalence.rst",
    "content": "=========================\n``gridmeter.equivalence``\n=========================\n\n.. automodule:: gridmeter.equivalence\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.equivalence\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.model.rst",
    "content": "===================\n``gridmeter.model``\n===================\n\n.. automodule:: gridmeter.model\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.model\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.param_selection.rst",
    "content": "=============================\n``gridmeter.param_selection``\n=============================\n\n.. automodule:: gridmeter.param_selection\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.param_selection\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.rst",
    "content": "=============\n``gridmeter``\n=============\n\n.. automodule:: gridmeter\n\n   .. contents::\n      :local:\n\n\nSubmodules\n==========\n\n.. toctree::\n\n   gridmeter.__version__\n   gridmeter.bin_selection\n   gridmeter.bins\n   gridmeter.diagnostics\n   gridmeter.distance_calc_selection\n   gridmeter.equivalence\n   gridmeter.model\n   gridmeter.param_selection\n   gridmeter.synthetic_data\n\n.. currentmodule:: gridmeter\n"
  },
  {
    "path": "docs/gridmeter/gridmeter.synthetic_data.rst",
    "content": "============================\n``gridmeter.synthetic_data``\n============================\n\n.. automodule:: gridmeter.synthetic_data\n\n   .. contents::\n      :local:\n\n.. currentmodule:: gridmeter.synthetic_data\n"
  },
  {
    "path": "opendsm/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport logging as _logging\n\nfrom importlib.metadata import metadata, PackageNotFoundError\n\ntry:\n    _meta = metadata(\"opendsm\")\nexcept PackageNotFoundError:\n    _meta = {}\n\n__title__ = _meta.get(\"Name\", \"opendsm\")\n__version__ = _meta.get(\"Version\", \"unknown\")\n__description__ = _meta.get(\"Summary\", \"\")\n__author__ = _meta.get(\"Author\", \"\")\n__author_email__ = _meta.get(\"Author-email\", \"\")\n__license__ = _meta.get(\"License\", \"\")\n__url__ = \"http://github.com/opendsm/opendsm\"\n__copyright__ = \"Copyright 2014-2025 OpenDSM contributors\"\n\nimport platform\nimport warnings\n\n# these happen during native code execution and segfault pytest when filterwarnings is set to error\nwarnings.filterwarnings(\"ignore\", module=\"importlib._bootstrap\")\nwarnings.filterwarnings(\n    \"ignore\", \"builtin type swigvarlink has no __module__ attribute\"\n)\nwarnings.filterwarnings(\n    \"ignore\", \"builtin type SwigPyPacked has no __module__ attribute\"\n)\n\nif platform.system() == \"Windows\":\n    # numba JIT breaks on Windows with int32/int64 return types\n    from numba import config\n\n    config.DISABLE_JIT = True\n\nfrom .common import test_data\nfrom . import (\n    eemeter,\n    drmeter,\n    comparison_groups,\n)\n\n# Set default logging handler to avoid \"No handler found\" warnings.\n_logging.getLogger(__name__).addHandler(_logging.NullHandler())\n\n# exclude built-in imports from namespace\n__all__ = [\n    \"__title__\",\n    \"__description__\",\n    \"__url__\",\n    \"__version__\",\n    \"__author__\",\n    \"__author_email__\",\n    \"__license__\",\n    \"__copyright__\",\n    \"eemeter\",\n    \"drmeter\",\n    \"comparison_groups\",\n    \"test_data\",\n]\n\ndef __dir__():\n    return __all__"
  },
  {
    "path": "opendsm/common/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.common.test_data import load_test_data\n\n__all__ = (\n    \"load_test_data\",\n)\n"
  },
  {
    "path": "opendsm/common/base_settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport pydantic\n\nfrom typing import Any\n\n\nclass BaseSettings(pydantic.BaseModel):\n    model_config = pydantic.ConfigDict(\n        frozen = True,\n        arbitrary_types_allowed=True,\n        str_to_lower = True,\n        str_strip_whitespace = True,\n    )\n\n    \"\"\"Make all property keys lowercase and strip whitespace\"\"\"\n    @pydantic.model_validator(mode=\"before\")\n    def __lowercase_property_keys__(cls, values: Any) -> Any:\n        def __lower__(value: Any) -> Any:\n            if isinstance(value, dict):\n                return {k.lower().strip() if isinstance(k, str) else k: __lower__(v) for k, v in value.items()}\n            return value\n\n        return __lower__(values)\n\n    \"\"\"Make all property values lowercase and strip whitespace before validation\"\"\"\n    @pydantic.field_validator(\"*\", mode=\"before\")\n    def lowercase_values(cls, v):\n        if isinstance(v, str):\n            return v.lower().strip()\n        return v\n\n\nclass MutableBaseSettings(BaseSettings):\n    model_config = pydantic.ConfigDict(\n        frozen = False,\n        arbitrary_types_allowed=True,\n        str_to_lower = True,\n        str_strip_whitespace = True,\n    )\n\n\n# add developer field to pydantic Field\ndef CustomField(developer=False, *args, **kwargs):\n    field = pydantic.Field(json_schema_extra={\"developer\": developer}, *args, **kwargs)\n    return field"
  },
  {
    "path": "opendsm/common/clustering/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .metrics import ClusterMetrics\nfrom .transform import (\n    normalize,\n    fpca_transform,\n    wavelet_transform,\n)\nfrom .cluster import cluster_features"
  },
  {
    "path": "opendsm/common/clustering/algorithms/__init__.py",
    "content": "from .bisect_k_means import bisect_k_means as _bisecting_kmeans_clustering\nfrom .birch import birch as _birch_clustering\nfrom .dbscan import dbscan as _dbscan_clustering\nfrom .hdbscan import hdbscan as _hdbscan_clustering\nfrom .spectral import spectral as _spectral_clustering"
  },
  {
    "path": "opendsm/common/clustering/algorithms/birch.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nfrom sklearn.cluster import Birch\n\nfrom opendsm.common.clustering import (\n    scoring as _scoring,\n    settings as _settings,\n    voting as _voting,\n)\n\n\n\ndef birch(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    Clusters features using Birch algorithm\n    \"\"\"\n\n    n_cluster_lower = settings.birch.n_cluster.lower\n    n_cluster_upper = settings.birch.n_cluster.upper\n    threshold = settings.birch.threshold\n    branching_factor = settings.birch.branching_factor\n\n    window_size = settings.birch.scoring.window_size\n\n    results = []\n    for n_clusters in range(n_cluster_lower, n_cluster_upper + 1):\n        algo = Birch(\n            n_clusters=n_clusters,\n            threshold=threshold,\n            branching_factor=branching_factor,\n        )\n        labels = algo.fit_predict(data)\n        \n        # Calculate score for the clusters\n        label_res = _scoring.score_clusters(data, labels, settings)\n        \n        results.append(label_res)\n\n    df_votes = _voting.construct_voting_df(results)\n    winner_idx = _voting.shulze_voting(\n        df_votes, \n        _scoring.score_council(settings), \n        window_size\n    )\n    # get labels of winner from results\n    winner_labels = results[winner_idx].labels\n\n    return winner_labels"
  },
  {
    "path": "opendsm/common/clustering/algorithms/bisect_k_means.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nfrom opendsm.common.clustering.algorithms import sklearn_bisect_k_means as _bisect_k_means\nfrom opendsm.common.clustering import (\n    scoring as _scoring,\n    settings as _settings,\n    voting as _voting,\n)\n\n\n\ndef bisect_k_means(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    clusters features using Bisecting K-Means algorithm\n    \"\"\"\n\n    algo_settings = settings.bisecting_kmeans\n    recluster_count = algo_settings.recluster_count\n    n_cluster_lower = algo_settings.n_cluster.lower\n    n_cluster_upper = algo_settings.n_cluster.upper\n    n_init = algo_settings.internal_recluster_count\n    inner_algorithm = algo_settings.inner_algorithm\n    bisecting_strategy = algo_settings.bisecting_strategy\n\n    window_size = algo_settings.scoring.window_size\n    min_cluster_size = algo_settings.scoring.min_cluster_size\n\n    seed = settings._seed\n\n    # Validate that we have enough samples to create the minimum number of clusters\n    n_samples = data.shape[0]\n    min_required_samples = n_cluster_lower * min_cluster_size\n    if n_samples <= min_required_samples:\n        raise ValueError(\n            f\"Insufficient samples for clustering: need more than {min_required_samples} samples \"\n            f\"(n_cluster_lower={n_cluster_lower} * min_cluster_size={min_cluster_size}), \"\n            f\"but only have {n_samples} samples\"\n        )\n\n    results = []\n    for i in range(recluster_count + 1):\n        algo = _bisect_k_means.BisectingKMeans(\n            n_clusters=n_cluster_upper,\n            init=\"k-means++\",  # does not benefit from k-means++ like other k-means\n            n_init=n_init,\n            random_state=seed + i,\n            algorithm=inner_algorithm,\n            bisecting_strategy=bisecting_strategy,\n        )\n        algo.fit(data)\n        labels_dict = algo.labels_full\n\n        # if specifying clusters, only score the specified clusters\n        if n_cluster_lower == n_cluster_upper:\n            labels_dict = {n_cluster_lower: labels_dict[n_cluster_lower]}\n\n        for n_cluster, labels in labels_dict.items():\n            label_res = _scoring.score_clusters(data, labels, settings)\n            results.append(label_res)\n\n    # Check if all results have score_unable_to_be_calculated == True\n    if all(all(result.score_unable_to_be_calculated.values()) for result in results):\n        return results[0].labels\n\n    # Construct voting df and perform voting to select best cluster count\n    df_votes = _voting.construct_voting_df(results)\n    winner_idx = _voting.shulze_voting(\n        df_votes, \n        _scoring.score_council(settings), \n        window_size\n    )\n    # get labels of winner from results\n    winner_labels = results[winner_idx].labels\n\n    return winner_labels"
  },
  {
    "path": "opendsm/common/clustering/algorithms/dbscan.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nfrom sklearn.cluster import DBSCAN\n\nfrom opendsm.common.clustering import settings as _settings\n\n\n\ndef dbscan(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    clusters features using DBSCAN algorithm\n    \"\"\"\n    algo = DBSCAN(\n        eps=settings.dbscan.epsilon, \n        min_samples=settings.dbscan.min_samples, \n        metric=settings.dbscan.distance_metric.value,\n        algorithm=settings.dbscan.nearest_neighbors_algorithm,\n        leaf_size=settings.dbscan.leaf_size,\n        p=settings.dbscan.minkowski_p,\n    )\n    labels = algo.fit_predict(data)\n\n    return labels"
  },
  {
    "path": "opendsm/common/clustering/algorithms/hdbscan.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\n\nfrom sklearn.cluster import HDBSCAN\n\nfrom opendsm.common.clustering import settings as _settings\n\n\n\ndef hdbscan(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    clusters features using HDBSCAN algorithm\n    \"\"\"\n    min_samples = settings.hdbscan.min_samples\n    if settings.hdbscan.min_samples == 1:\n        min_samples = 2\n\n    algo = HDBSCAN(\n        min_samples=settings.hdbscan.scoring_sample_count, \n        min_cluster_size=min_samples,\n        allow_single_cluster=settings.hdbscan.allow_single_cluster,\n        max_cluster_size=settings.hdbscan.max_cluster_size,\n        metric=settings.hdbscan.distance_metric,\n        cluster_selection_epsilon=settings.hdbscan.cluster_selection_epsilon,\n        alpha=settings.hdbscan.robust_single_linkage_scaling,\n        algorithm=settings.hdbscan.nearest_neighbors_algorithm,\n        leaf_size=settings.hdbscan.leaf_size,\n        cluster_selection_method=settings.hdbscan.cluster_selection_method,\n    )\n    labels = algo.fit_predict(data)\n\n    if settings.hdbscan.min_samples == 1:\n        # get count of -1 labels\n        outlier_count = np.sum(labels == -1)\n\n        if outlier_count == 0:\n            return labels\n\n        # add to all labels to make room for outliers\n        labels[labels != -1] += outlier_count\n\n        # make labels with -1 defined as arange(max_label+1, n_samples)\n        labels[labels == -1] = np.arange(0, outlier_count)\n\n    return labels"
  },
  {
    "path": "opendsm/common/clustering/algorithms/sklearn_bisect_k_means.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom copy import deepcopy as copy\n\nimport numpy as np\nimport scipy.sparse as sp\n\nfrom sklearn.cluster import BisectingKMeans as _sklearn_BisectingKMeans\nfrom sklearn.cluster import _bisect_k_means\nfrom sklearn.cluster._kmeans import (\n    _kmeans_single_elkan,\n    _kmeans_single_lloyd,\n) # type: ignore\nfrom sklearn.cluster._k_means_common import (\n    _inertia_dense,\n    _inertia_sparse,\n) # type: ignore\nfrom sklearn.utils.extmath import row_norms\nfrom sklearn.utils.validation import (\n    _check_sample_weight,\n    check_random_state,\n) # type: ignore\ntry:\n    from sklearn.utils.validation import validate_data # type: ignore\nexcept ImportError:\n    validate_data = None  # type: ignore\n# from sklearn.utils._openmp_helpers import _openmp_effective_n_threads\n\nimport logging\n\nlogger = logging.getLogger(__name__)\nlogging.basicConfig(level=logging.INFO)\n\n\nclass BisectingKMeans(_sklearn_BisectingKMeans):\n    \"\"\"\n    Override of sklearn class which simply saves the labels\n    of all intermediate cluster steps.\n\n    Only overrides fit\n\n    Should always take the upper bound of number of clusters to try.\n    Contains a new property named labels_full which is a dictionary where the key is the number of clusters\n    and the value is the labels using that number.\n\n    This should be used to score all the labels and determine the best number/labels to use/\n\n\n    \"\"\"\n\n    def fit(self, X, y=None, sample_weight=None):\n        \"\"\"Compute bisecting k-means clustering.\n\n        Parameters\n        ----------\n        X : {array-like, sparse matrix} of shape (n_samples, n_features)\n\n            Training instances to cluster.\n\n            .. note:: The data will be converted to C ordering,\n                which will cause a memory copy\n                if the given data is not C-contiguous.\n\n        y : Ignored\n            Not used, present here for API consistency by convention.\n\n        sample_weight : array-like of shape (n_samples,), default=None\n            The weights for each observation in X. If None, all observations\n            are assigned equal weight.\n\n        Returns\n        -------\n        self\n            Fitted estimator.\n        \"\"\"\n        # return self._fit_test(X, y, sample_weight)\n\n        self._validate_params()  # type: ignore\n\n        if validate_data is not None:\n            X = validate_data(  # type: ignore\n                self,\n                X,\n                accept_sparse=\"csr\",\n                dtype=[np.float64, np.float32],\n                order=\"C\",\n                copy=self.copy_x,  # type: ignore\n                accept_large_sparse=False,\n            )\n        else:\n            X = self._validate_data(  # type: ignore\n                X,\n                accept_sparse=\"csr\",\n                dtype=[np.float64, np.float32],\n                order=\"C\",\n                copy=self.copy_x,  # type: ignore\n                accept_large_sparse=False,\n            )\n\n        self._check_params_vs_input(X)  # type: ignore\n\n        self._random_state = check_random_state(self.random_state)  # type: ignore\n        sample_weight = _check_sample_weight(sample_weight, X, dtype=X.dtype)\n        # self._n_threads = _openmp_effective_n_threads()\n        self._n_threads = 1  # OVERRIDE OF ABOVE SO THAT RESULTS ARE DETERMINISTIC\n\n        if self.algorithm == \"lloyd\" or self.n_clusters == 1:  # type: ignore\n            self._kmeans_single = _kmeans_single_lloyd\n            self._check_mkl_vcomp(X, X.shape[0])  # type: ignore\n        else:\n            self._kmeans_single = _kmeans_single_elkan\n\n        # Subtract of mean of X for more accurate distance computations\n        if not sp.issparse(X):\n            self._X_mean = X.mean(axis=0)\n            X -= self._X_mean\n\n        # Initialize the hierarchical clusters tree\n        self._bisecting_tree = _bisect_k_means._BisectingTree(\n            indices=np.arange(X.shape[0]),\n            center=X.mean(axis=0),\n            score=0,\n        )\n\n        x_squared_norms = row_norms(X, squared=True)\n        self.labels_full = {}\n        for i in range(self.n_clusters - 1):  # type: ignore\n            # Chose cluster to bisect\n            try:\n                cluster_to_bisect = self._bisecting_tree.get_cluster_to_bisect()\n            except RecursionError:\n                logger.warn(\n                    f\"encountered Recursion error during bisection for cluster size {i + 2}. Returning early\"\n                )\n                return self\n\n            # Split this cluster into 2 subclusters\n            try:\n                self._bisect(X, x_squared_norms, sample_weight, cluster_to_bisect)  # type: ignore\n            except IndexError:\n                logger.warn(\n                    f\"encountered IndexError during bisection for cluster size {i + 2}\"\n                )\n                return self  # return early so that calculated labels can be returned until an error arose\n\n            # Aggregate final labels and centers from the bisecting tree\n            labels = np.full(X.shape[0], -1, dtype=np.int32)\n\n            for j, cluster_node in enumerate(self._bisecting_tree.iter_leaves()):\n                labels[cluster_node.indices] = j  # type: ignore\n\n            self.labels_full[i + 2] = copy(labels)\n\n        # Aggregate final labels and centers from the bisecting tree\n        self.labels_ = np.full(X.shape[0], -1, dtype=np.int32)\n        self.cluster_centers_ = np.empty((self.n_clusters, X.shape[1]), dtype=X.dtype)  # type: ignore\n\n        for i, cluster_node in enumerate(self._bisecting_tree.iter_leaves()):\n            self.labels_[cluster_node.indices] = i  # type: ignore\n            self.cluster_centers_[i] = cluster_node.center  # type: ignore\n            cluster_node.label = i  # type: ignore\n            cluster_node.indices = None  # type: ignore\n\n        # Restore original data\n        if not sp.issparse(X):\n            X += self._X_mean\n            self.cluster_centers_ += self._X_mean\n\n        _inertia = _inertia_sparse if sp.issparse(X) else _inertia_dense\n        self.inertia_ = _inertia(\n            X, sample_weight, self.cluster_centers_, self.labels_, self._n_threads\n        )\n\n        self._n_features_out = self.cluster_centers_.shape[0]\n\n        return self\n"
  },
  {
    "path": "opendsm/common/clustering/algorithms/spectral.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport warnings\n\nimport numpy as np\n\nfrom scipy.spatial.distance import pdist\nfrom scipy.sparse.linalg import eigsh\nfrom scipy.sparse import csgraph\n\nfrom sklearn.cluster import SpectralClustering\nfrom sklearn.metrics.pairwise import pairwise_kernels\n\nfrom opendsm.common.clustering import (\n    scoring as _scoring,\n    settings as _settings,\n    voting as _voting,\n)\n\n\n\ndef eigenDecomposition(A, topK = 5):\n    \"\"\"\n    :param A: Affinity matrix\n    :param plot: plots the sorted eigen values for visual inspection\n    :return A tuple containing:\n    - the optimal number of clusters by eigengap heuristic\n    - all eigen values\n    - all eigen vectors\n    \n    This method performs the eigen decomposition on a given affinity matrix,\n    following the steps recommended in the paper:\n    1. Construct the normalized affinity matrix: L = D−1/2ADˆ −1/2.\n    2. Find the eigenvalues and their associated eigen vectors\n    3. Identify the maximum gap which corresponds to the number of clusters\n    by eigengap heuristic\n    \n    References:\n    https://papers.nips.cc/paper/2619-self-tuning-spectral-clustering.pdf\n    http://www.kyb.mpg.de/fileadmin/user_upload/files/publications/attachments/Luxburg07_tutorial_4488%5b0%5d.pdf\n    \"\"\"\n    L = csgraph.laplacian(A, normed=True)\n    n_components = A.shape[0]\n    \n    # LM parameter : Eigenvalues with largest magnitude (eigs, eigsh), that is, largest eigenvalues in \n    # the euclidean norm of complex numbers.\n#     eigenvalues, eigenvectors = eigsh(L, k=n_components, which=\"LM\", sigma=1.0, maxiter=5000)\n    eigenvalues, eigenvectors = np.linalg.eig(L)\n        \n    # Identify the optimal number of clusters as the index corresponding\n    # to the larger gap between eigen values\n    index_largest_gap = np.argsort(np.diff(eigenvalues))[::-1]\n    nb_clusters = index_largest_gap + 1\n        \n    return nb_clusters, eigenvalues, eigenvectors\n\n\ndef eigendecomp_cluster_count(\n    X,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    Votes on the optimal number of clusters using the eigen decomposition\n    \"\"\"\n    min_clusters = settings.spectral.n_cluster.lower\n    max_clusters = settings.spectral.n_cluster.upper\n\n    nb_clusters, _, _ = eigenDecomposition(X)\n\n    # only include clusters in the range of min_clusters to max_clusters\n    nb_clusters = nb_clusters[nb_clusters >= min_clusters]\n    nb_clusters = nb_clusters[nb_clusters <= max_clusters]\n\n    return nb_clusters\n\n\ndef _affinity_matrix(\n    data: np.ndarray,\n    algo: SpectralClustering,\n):\n    \"\"\"\n    Computes the affinity matrix for the given data\n    \"\"\"\n    params = algo.kernel_params\n    if params is None:\n        params = {}\n    if not callable(algo.affinity):\n        params[\"gamma\"] = algo.gamma\n        params[\"degree\"] = algo.degree\n        params[\"coef0\"] = algo.coef0\n\n    X = pairwise_kernels(\n        data, metric=algo.affinity, filter_params=True, **params\n    )\n\n    return X\n\n\ndef _single_spectral_clustering(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    clusters features using Spectral Clustering algorithm\n    \"\"\"\n\n    n_cluster_lower = settings.spectral.n_cluster.lower\n    n_cluster_upper = settings.spectral.n_cluster.upper\n\n    window_size = settings.spectral.scoring.window_size\n\n    algo = SpectralClustering(\n        n_clusters=n_cluster_lower,\n        eigen_solver=settings.spectral.eigen_solver,\n        n_components=settings.spectral.n_components,\n        affinity=settings.spectral.affinity,\n        n_neighbors=settings.spectral.nearest_neighbors,\n        gamma=settings.spectral.gamma,\n        eigen_tol=settings.spectral.eigen_tol,\n        assign_labels=settings.spectral.assign_labels,\n        random_state=settings._seed\n    )\n\n    # transform data as spectral clustering doesn't like negative values\n    # data = np.exp(-data / np.std(data))\n\n    # For nearest_neighbors affinity, let sklearn handle it internally\n    # For other affinities, precompute the affinity matrix\n    if settings.spectral.affinity == \"nearest_neighbors\":\n        X = data\n    else:\n        # X = _local_affinity_matrix(data)\n        X = _affinity_matrix(data, algo)\n        algo.affinity = \"precomputed\"\n\n    results = []\n    n_clusters_range = np.arange(n_cluster_lower, n_cluster_upper + 1)\n    for n_clusters in n_clusters_range:\n        if n_clusters > n_cluster_lower:\n            algo.n_clusters = n_clusters\n\n        np_state = np.random.get_state()\n        np.random.seed(settings._seed)\n\n        # hide UserWarning from sklearn\n        with warnings.catch_warnings():\n            warnings.filterwarnings(\"ignore\", category=UserWarning)\n            labels = algo.fit_predict(X)\n\n        # Calculate a score for the clustering\n        label_res = _scoring.score_clusters(data, labels, settings)\n\n        np.random.set_state(np_state)\n        \n        results.append(label_res)\n\n    df_votes = _voting.construct_voting_df(results)\n    # df_votes.index = n_clusters_range\n\n    # # drop single cluster from df_votes\n    # df_votes = df_votes.drop(index=1, errors='ignore')\n\n    winner_idx, df_votes = _voting.shulze_voting(\n        df_votes, \n        _scoring.score_council(settings), \n        window_size,\n        return_preference_df=True\n        )\n    df_votes.index = n_clusters_range\n\n    # get labels of winner from results\n    label_res = results[winner_idx]\n\n    return label_res, df_votes\n    \n\ndef _local_affinity_matrix(X):\n    dim = X.shape[0]\n    dist_ = pdist(X)\n    pd = np.zeros([dim, dim])\n    dist = iter(dist_)\n    for i in range(dim):\n        for j in range(i+1, dim):  \n            d = next(dist)\n            pd[i,j] = d\n            pd[j,i] = d\n            \n    #calculate local sigma\n    sigmas = np.zeros(dim)\n    for i in range(len(pd)):\n        sigmas[i] = sorted(pd[i])[7]\n    \n    A = np.zeros([dim, dim])\n    dist = iter(dist_)\n    for i in range(dim):\n        for j in range(i+1, dim):  \n            d = np.exp(-1*next(dist)**2/(sigmas[i]*sigmas[j]))\n\n            A[i,j] = d\n            A[j,i] = d\n\n    return A\n\n\n# defunct/experimental\nclass RobustSpectralClustering:\n    \"\"\"\n    Implementation of the method proposed in the paper:\n    'Robust Spectral Clustering for Noisy Data: Modeling Sparse Corruptions Improves Latent Embeddings'\n\n    If you publish material based on algorithms or evaluation measures obtained from this code,\n    then please note this in your acknowledgments and please cite the following paper:\n        Aleksandar Bojchevski, Yves Matkovic, and Stephan Günnemann.\n        2017. Robust Spectral Clustering for Noisy Data.\n        In Proceedings of KDD’17, August 13–17, 2017, Halifax, NS, Canada.\n\n    Copyright (C) 2017\n    Aleksandar Bojchevski\n    Yves Matkovic\n    Stephan Günnemann\n    Technical University of Munich, Germany\n    \"\"\"\n\n    def __init__(self, k, nn=15, theta=20, m=0.5, laplacian=1, n_iter=50, normalize=False, affinity=\"local\", verbose=False):\n        \"\"\"\n        :param k: number of clusters\n        :param nn: number of neighbours to consider for constructing the KNN graph (excluding the node itself)\n        :param theta: number of corrupted edges to remove\n        :param m: minimum percentage of neighbours to keep per node (omega_i constraints)\n        :param n_iter: number of iterations of the alternating optimization procedure\n        :param laplacian: which graph Laplacian to use: 0: L, 1: L_rw, 2: L_sym\n        :param normalize: whether to row normalize the eigen vectors before performing k-means\n        :param verbose: verbosity\n        \"\"\"\n\n        self.k = k\n        self.nn = nn\n        self.theta = theta\n        self.m = m\n        self.n_iter = n_iter\n        self.normalize = normalize\n        self.verbose = verbose\n        self.laplacian = laplacian\n        self.affinity = affinity\n\n        if laplacian == 0:\n            if self.verbose:\n                print('Using unnormalized Laplacian L')\n        elif laplacian == 1:\n            if self.verbose:\n                print('Using random walk based normalized Laplacian L_rw')\n        elif laplacian == 2:\n            raise NotImplementedError('The symmetric normalized Laplacian L_sym is not implemented yet.')\n        else:\n            raise ValueError('Choice of graph Laplacian not valid. Please use 0, 1 or 2.')\n\n    def __affinity_matrix(self, X):\n        # compute the KNN graph\n        A = kneighbors_graph(X=X, n_neighbors=self.nn, metric='euclidean', include_self=False, mode='connectivity')\n        A = A.maximum(A.T)  # make the graph undirected\n\n        return A\n\n    def __local_affinity_matrix(self, X):\n        dim = X.shape[0]\n        dist_ = pdist(X)\n        pd = np.zeros([dim, dim])\n        dist = iter(dist_)\n        for i in range(dim):\n            for j in range(i+1, dim):  \n                d = next(dist)\n                pd[i,j] = d\n                pd[j,i] = d\n                \n        #calculate local sigma\n        sigmas = np.zeros(dim)\n        for i in range(len(pd)):\n            sigmas[i] = sorted(pd[i])[7]\n        \n        A = np.zeros([dim, dim])\n        dist = iter(dist_)\n        for i in range(dim):\n            for j in range(i+1, dim):  \n                d = np.exp(-1*next(dist)**2/(sigmas[i]*sigmas[j]))\n\n                A[i,j] = d\n                A[j,i] = d\n\n        return A\n\n    def __latent_decomposition(self, X):\n        # compute the KNN graph\n        if self.affinity != \"local\":\n            A = self.__affinity_matrix(X)\n        else:   \n            A = self.__local_affinity_matrix(X)\n            \n        N = A.shape[0]  # number of nodes\n        deg = A.sum(0).A1  # node degrees\n\n        prev_trace = np.inf  # keep track of the trace for convergence\n        Ag = A.copy()\n\n        for it in range(self.n_iter):\n\n            # form the unnormalized Laplacian\n            D = sp.diags(Ag.sum(0).A1).tocsc()\n            L = D - Ag\n\n            # solve the normal eigenvalue problem\n            if self.laplacian == 0:\n                h, H = eigsh(L, self.k, which='SM')\n            # solve the generalized eigenvalue problem\n            elif self.laplacian == 1:\n                h, H = eigsh(L, self.k, D, which='SM')\n\n            trace = h.sum()\n\n            if self.verbose:\n                print('Iter: {} Trace: {:.4f}'.format(it, trace))\n\n            if self.theta == 0:\n                # no edges are removed\n                Ac = sp.coo_matrix((N, N), [np.int])\n                break\n\n            if prev_trace - trace < 1e-10:\n                # we have converged\n                break\n\n            allowed_to_remove_per_node = (deg * self.m).astype(np.int)\n            prev_trace = trace\n\n            # consider only the edges on the lower triangular part since we are symmetric\n            edges = sp.tril(A).nonzero()\n            removed_edges = []\n\n            if self.laplacian == 1:\n                # fix for potential numerical instability of the eigenvalues computation\n                h[np.isclose(h, 0)] = 0\n\n                # equation (5) in the paper\n                p = np.linalg.norm(H[edges[0]] - H[edges[1]], axis=1) ** 2 \\\n                    - np.linalg.norm(H[edges[0]] * np.sqrt(h), axis=1) ** 2 \\\n                    - np.linalg.norm(H[edges[1]] * np.sqrt(h), axis=1) ** 2\n            else:\n                # equation (4) in the paper\n                p = np.linalg.norm(H[edges[0]] - H[edges[1]], axis=1) ** 2\n\n            # greedly remove the worst edges\n            for ind in p.argsort()[::-1]:\n                e_i, e_j, p_e = edges[0][ind], edges[1][ind], p[ind]\n\n                # remove the edge if it satisfies the constraints\n                if allowed_to_remove_per_node[e_i] > 0 and allowed_to_remove_per_node[e_j] > 0 and p_e > 0:\n                    allowed_to_remove_per_node[e_i] -= 1\n                    allowed_to_remove_per_node[e_j] -= 1\n                    removed_edges.append((e_i, e_j))\n                    if len(removed_edges) == self.theta:\n                        break\n\n            removed_edges = np.array(removed_edges)\n            Ac = sp.coo_matrix((np.ones(len(removed_edges)), (removed_edges[:, 0], removed_edges[:, 1])), shape=(N, N))\n            Ac = Ac.maximum(Ac.T)\n            Ag = A - Ac\n\n        return Ag, Ac, H\n\n    def fit_predict(self, X):\n        \"\"\"\n        :param X: array-like or sparse matrix, shape (n_samples, n_features)\n        :return: cluster labels ndarray, shape (n_samples,)\n        \"\"\"\n\n        Ag, Ac, H = self.__latent_decomposition(X)\n        self.Ag = Ag\n        self.Ac = Ac\n\n        if self.normalize:\n            self.H = H / np.linalg.norm(H, axis=1)[:, None]\n        else:\n            self.H = H\n\n        centroids, labels, *_ = k_means(X=self.H, n_clusters=self.k)\n\n        self.centroids = centroids\n        self.labels = labels\n\n        return labels\n\n\ndef spectral(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    clusters features using Spectral Clustering algorithm\n    \"\"\"\n    recluster_count = settings.spectral.recluster_count\n    \n    results = []\n    df_votes_recluster = []\n    for n in range(recluster_count + 1):\n        if n > 0:\n            settings_dict = settings.model_dump()\n            settings_dict[\"seed\"] = settings_dict[\"seed\"] + 1\n            settings = _settings.ClusteringSettings(**settings_dict)\n        \n        label_res, df_votes = _single_spectral_clustering(data, settings)\n        \n        results.append(label_res)\n        df_votes_recluster.append(df_votes)\n\n    winner_idx = 0\n    if recluster_count > 0:\n        df_votes = _voting.construct_voting_df(results)\n        winner_idx = _voting.shulze_voting(\n            df_votes, \n            _scoring.score_council(settings), \n            window_size=0\n        )\n\n    # get labels of winner from results\n    winner_labels = results[winner_idx].labels\n    # df_votes_recluster = df_votes_recluster[winner_idx]\n\n    return winner_labels"
  },
  {
    "path": "opendsm/common/clustering/cluster.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nfrom scipy.signal import find_peaks\nfrom scipy.spatial.distance import cdist\n\nfrom opendsm.common.clustering import (\n    settings as _settings,\n    transform as _transform,\n)\n\nfrom opendsm.common.clustering.algorithms import (\n    _bisecting_kmeans_clustering,\n    _birch_clustering,\n    _dbscan_clustering,\n    _hdbscan_clustering,\n    _spectral_clustering,\n)\n\n\n\ndef _cluster_merge(\n    cluster_labels: np.ndarray,\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings,\n    W: float = 0.5,\n):\n    \n    # get unique labels\n    unique_labels = np.unique(cluster_labels)\n\n    # get the distance between all rows in data\n    distances = cdist(data, data)\n\n    intra_cluster_similarity = np.zeros(len(unique_labels))\n    inter_cluster_similarity = np.zeros((len(unique_labels), len(unique_labels)))\n    for i in range(len(unique_labels)):\n        idx_i = np.where(cluster_labels == unique_labels[i])[0]\n        for j in range(len(unique_labels)):\n            idx_j = np.where(cluster_labels == unique_labels[j])[0]\n\n            if i == j:\n                intra_cluster_similarity[i] = np.mean(distances[idx_i, :][:, idx_i])\n                inter_cluster_similarity[i, j] = 0\n                continue\n            elif i < j:\n                continue\n\n            inter_cluster_similarity[i, j] = np.sum(distances[idx_i, :][:, idx_j])\n            inter_cluster_similarity[j, i] = np.nan\n\n    # if there are only two clusters, merge them if the similarity is less than W\n    if unique_labels.shape[0] == 2:\n        cluster_similarity = inter_cluster_similarity[0, 1]\n        mean_similarity = np.mean(distances[distances != 0])\n\n        ratio = cluster_similarity / mean_similarity\n\n        if ratio < W:\n            return np.zeros(len(cluster_labels))\n        \n        return cluster_labels\n\n    # if there are more than two clusters, merge them if the similarity is less than W\n    mean_similarity = np.mean(inter_cluster_similarity)\n\n    for i in reversed(range(len(unique_labels))):\n        for j in reversed(range(len(unique_labels))):\n            if i == j:\n                continue\n\n            ratio = inter_cluster_similarity[i, j] / mean_similarity\n\n            if ratio < W:\n                cluster_labels[cluster_labels == unique_labels[j]] = unique_labels[i]\n\n    return cluster_labels\n\n\ndef cluster_reorder(\n    data: pd.DataFrame, \n    cluster_labels: np.ndarray,\n    settings: _settings.ClusteringSettings,\n):\n    sort_method = settings.cluster_sort.method\n    agg_type = settings.cluster_sort.aggregation\n    reverse = settings.cluster_sort.reverse\n\n    # assign labels to data\n    df = data.copy()\n    df[\"label\"] = cluster_labels\n     # exclude label -1 (outliers) from reordering\n    df = df[df['label'] >= 0]\n\n    # calculate n_clusters after filtering out outliers\n    uniq_labels = df['label'].unique()\n    n_clusters = len(uniq_labels)\n\n    if sort_method == \"size\":\n        # sort clusters by count\n        cluster_size = df['label'].value_counts()\n        cluster_size = cluster_size.sort_values()\n\n        features = cluster_size\n\n    elif sort_method == \"peak\":\n        # TODO: This is a work in progress\n\n        # group by cluster and aggregate\n        df_cluster = df.groupby('label').agg(agg_type)\n\n        # subtract each cluster's median from the cluster's median\n        df_cluster_norm = df_cluster.sub(df_cluster.agg(agg_type, axis=1), axis=0)\n        cluster_max = df_cluster_norm.abs().max().max()\n        df_cluster_norm = df_cluster_norm/cluster_max\n\n        # define threshold for peak and valley\n        threshold = np.quantile(abs(df_cluster.values), 0.75)\n\n        # find peaks and valleys\n        peak = {}\n        valley = {}\n        norm = {}\n        for i in range(n_clusters):\n            cluster_normal = df_cluster.iloc[i]\n            norm[i] = cluster_normal.agg(agg_type)\n            df_cluster_norm = cluster_normal - norm[i]\n            thresh = threshold - norm[i]\n\n            peak[i] = find_peaks(df_cluster_norm.values, height=thresh, width=1)[0]\n            valley[i] = find_peaks(-df_cluster_norm.values, height=thresh, width=1)[0]\n\n            if len(peak[i]) == 0:\n                peak[i] = None\n            else:\n                peak[i] = peak[i][0]\n\n            if len(valley[i]) == 0:\n                valley[i] = None\n            else:\n                valley[i] = valley[i][0]\n        \n        # create df with peak and valley\n        features = pd.DataFrame({'peak': peak, 'valley': valley, \"norm\": norm})\n\n        features = features.sort_values(by=[\"peak\", \"valley\", \"norm\"], na_position='first')\n\n    # create dictionary to remap cluster numbers to features order\n    cluster_map = {i: i for i in cluster_labels}\n\n    if not reverse:\n        cluster_map.update({features.index[i]: i for i in range(n_clusters)})\n    else:\n        # Reverse the mapping: smallest feature gets highest index, largest gets lowest\n        cluster_map.update({features.index[i]: n_clusters - 1 - i for i in range(n_clusters)})\n\n    return cluster_map\n\n\ndef _cluster_features(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings,\n) -> np.ndarray:\n\n    # adjust upper cluster count if necessary\n    if settings.algorithm_selection not in [\"dbscan\", \"hdbscan\"]:\n        algo = f\"{settings.algorithm_selection.value}\"\n        algo_settings = getattr(settings, algo)\n\n        data_count = len(data)\n        cluster_count = algo_settings.n_cluster.upper\n        min_cluster_size = algo_settings.scoring.min_cluster_size\n        min_required_data = min_cluster_size * cluster_count\n\n        if data_count < min_required_data:\n            settings_dict = settings.model_dump()\n            settings_dict[algo][\"n_cluster\"][\"upper\"] = data_count // min_cluster_size\n            settings = _settings.ClusteringSettings(**settings_dict)\n    \n    # cluster the pca features\n    if settings.algorithm_selection == \"bisecting_kmeans\":\n        cluster_fcn = _bisecting_kmeans_clustering\n    elif settings.algorithm_selection == \"birch\":\n        cluster_fcn = _birch_clustering\n    elif settings.algorithm_selection == \"dbscan\":\n        cluster_fcn = _dbscan_clustering\n    elif settings.algorithm_selection == \"hdbscan\":\n        cluster_fcn = _hdbscan_clustering\n    elif settings.algorithm_selection == \"spectral\":\n        cluster_fcn = _spectral_clustering\n    else:\n        raise ValueError(f\"Unknown clustering algorithm: {settings.algorithm_selection}\")\n    \n    cluster_labels = cluster_fcn(data, settings)\n\n    return cluster_labels\n\n\ndef cluster_features(\n    df: pd.DataFrame,\n    settings: _settings.ClusteringSettings,\n):\n    # convert data to numpy array\n    data = df.to_numpy()\n    \n    # bypass clustering if cluster count is >= data\n    if settings.algorithm_selection not in [\"dbscan\", \"hdbscan\"]:\n        algo = f\"{settings.algorithm_selection.value}\"\n        algo_settings = getattr(settings, algo)\n        if algo_settings.n_cluster.lower >= len(data):\n            return np.arange(len(data))\n\n    data = _transform.transform_features(data, settings)\n\n    cluster_labels = _cluster_features(data, settings)\n\n    skip_merge = True\n    if not skip_merge and np.unique(cluster_labels).shape[0] == 2:\n        cluster_labels = _cluster_merge(cluster_labels, data, settings)\n\n    if settings.cluster_sort.enable:\n        cluster_remap_dict = cluster_reorder(df, cluster_labels, settings)\n\n        # remap cluster labels using cluster_remap_dict\n        cluster_labels = np.vectorize(cluster_remap_dict.get)(cluster_labels)\n\n    return cluster_labels"
  },
  {
    "path": "opendsm/common/clustering/metrics/__init__.py",
    "content": "from .cluster_metrics import ClusterMetrics"
  },
  {
    "path": "opendsm/common/clustering/metrics/cluster_metrics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2025 OpenDSM contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\nfrom __future__ import annotations\n\nimport pydantic\nfrom typing import Optional, Literal\nfrom enum import Enum\n\nimport numpy as np\n\nfrom scipy.spatial.distance import cdist, pdist, squareform\n\nfrom opendsm.common.stats.basic import median_absolute_deviation\nfrom opendsm.common.pydantic_utils import (\n    ArbitraryPydanticModel,\n    computed_field_cached_property,\n)\nfrom opendsm.common.clustering.metrics.density_based_clustering_validation import dbcv\n\n\n\nclass DistanceMetric(str, Enum):\n    \"\"\"\n    what distance method to use\n    \"\"\"\n    EUCLIDEAN = \"euclidean\"\n    STANDARDIZED_EUCLIDEAN = \"seuclidean\"\n    SQUARED_EUCLIDEAN = \"sqeuclidean\"\n    MANHATTAN = \"manhattan\"\n    COSINE = \"cosine\"\n\n\nclass ClusterPairDistanceMetrics(ArbitraryPydanticModel):\n    \"\"\"\n    Metrics between clusters\n    \"\"\"\n\n    cluster_ids: Optional[tuple[int, int]] = pydantic.Field(\n        default=None,\n        description=\"The two clusters to compare\"\n    )\n\n    distance: np.ndarray = pydantic.Field(\n        exclude=True,\n        repr=False,\n    )\n\n    @computed_field_cached_property()\n    def n(self) -> int:\n        return self.distance.size\n\n    @computed_field_cached_property()\n    def sum_of_squares(self) -> float:\n        return np.sum(self.distance**2)\n\n    @computed_field_cached_property()\n    def mean(self) -> float:\n        return np.mean(self.distance)\n\n    @computed_field_cached_property()\n    def median(self) -> float:\n        return np.median(self.distance)\n\n    @computed_field_cached_property()\n    def var(self) -> float:\n        return np.var(self.distance)\n\n    @computed_field_cached_property()\n    def std(self) -> float:\n        return np.std(self.distance)\n\n    @computed_field_cached_property()\n    def mad(self) -> float:\n        return median_absolute_deviation(self.distance)\n    \n    @computed_field_cached_property()\n    def lower_quantile(self) -> float:\n        return np.quantile(self.distance, 0.05)\n\n    @computed_field_cached_property()\n    def upper_quantile(self) -> float:\n        return np.quantile(self.distance, 0.95)\n\n    @computed_field_cached_property()\n    def min(self) -> float:\n        return np.min(self.distance[self.distance > 0])\n    \n    @computed_field_cached_property()\n    def max(self) -> float:\n        return np.max(self.distance)\n\n\nclass SingleClusterMetrics(ArbitraryPydanticModel):\n    \"\"\"\n    Metrics within a single cluster\n    \"\"\"\n    cluster_id: int | None = pydantic.Field(\n        default=None,\n    )\n\n    n: int = pydantic.Field()\n\n    mean: np.ndarray = pydantic.Field()\n\n    median: np.ndarray = pydantic.Field()\n\n    var: Optional[np.ndarray] = pydantic.Field(\n        default=None,\n    )\n\n    distance: dict[int, ClusterPairDistanceMetrics] | ClusterPairDistanceMetrics = pydantic.Field()\n\n    distance_to_mean: dict[int | str, ClusterPairDistanceMetrics] | ClusterPairDistanceMetrics = pydantic.Field()\n\n    distance_to_median: dict[int | str, ClusterPairDistanceMetrics] | ClusterPairDistanceMetrics = pydantic.Field()\n\n    mean_distance_intra_cluster: Optional[np.ndarray] = pydantic.Field(\n        default=None,\n        exclude=True,\n        repr=False,\n    )\n\n    median_distance_intra_cluster: Optional[np.ndarray] = pydantic.Field(\n        default=None,\n        exclude=True,\n        repr=False,\n    )\n\n    mean_distance_to_nearest_cluster: Optional[np.ndarray] = pydantic.Field(\n        default=None,\n        exclude=True,\n        repr=False,\n    )\n\n    median_distance_to_nearest_cluster: Optional[np.ndarray] = pydantic.Field(\n        default=None,\n        exclude=True,\n        repr=False,\n    )\n\n    @computed_field_cached_property()\n    def var_norm(self) -> float | None:\n        \"\"\"Norm of the per-dimension variance vector: ||σ||\"\"\"\n        if self.var is None:\n            return None\n        \n        return np.linalg.norm(self.var)\n\n    @computed_field_cached_property()\n    def within_pairwise_distances(self) -> np.ndarray | None:\n        \"\"\"Upper triangle of intra-cluster pairwise distances (no diagonal, no duplicates)\"\"\"\n        if self.cluster_id is None or not isinstance(self.distance, dict):\n            return None\n\n        d = self.distance[self.cluster_id].distance\n        return d[np.triu_indices_from(d, k=1)]\n\n    @computed_field_cached_property()\n    def between_pairwise_distances(self) -> np.ndarray | None:\n        \"\"\"All pairwise distances from this cluster to other clusters\"\"\"\n        if self.cluster_id is None or not isinstance(self.distance, dict):\n            return None\n\n        parts = []\n        for label_j, pair_metrics in self.distance.items():\n            if label_j != self.cluster_id:\n                parts.append(pair_metrics.distance.ravel())\n\n        return np.concatenate(parts) if parts else np.array([])\n\n    @computed_field_cached_property()\n    def mean_silhouette_coefficient(self) -> np.ndarray:\n        if self.mean_distance_intra_cluster is None:\n            return None\n\n        a = self.mean_distance_intra_cluster\n        b = self.mean_distance_to_nearest_cluster\n        \n        return (b - a) / np.maximum(a, b)\n\n    @computed_field_cached_property()\n    def median_silhouette_coefficient(self) -> np.ndarray:\n        if self.median_distance_intra_cluster is None:\n            return None\n\n        a = self.median_distance_intra_cluster\n        b = self.median_distance_to_nearest_cluster\n\n        return (b - a) / np.maximum(a, b)\n\n\nclass ClusterMetrics(ArbitraryPydanticModel):\n    # TODO: Update the doc string\n    \"\"\"Input dataframe to be used for metrics calculations\"\"\"\n    data: np.ndarray = pydantic.Field(\n        exclude=True,\n        repr=False,\n    )\n\n    labels: np.ndarray = pydantic.Field(\n        exclude=True,\n        repr=False,\n    )\n\n    distance_metric: DistanceMetric = pydantic.Field(\n        default=DistanceMetric.EUCLIDEAN,\n    )\n\n    index_direction: Literal[\"minimize\", \"maximize\"] = pydantic.Field(\n        default=\"minimize\",\n        description=\"Force the indice direction to `minimize` or `maximize` as best\",\n    )\n\n    _eps: float = 1e-10\n    _all: int = -999\n\n    @pydantic.model_validator(mode='after')\n    def _validate_data(self) -> 'ClusterMetrics':\n        if self.data.shape[0] == 0:\n            raise ValueError(\"Data must have at least one row\")\n\n        if self.labels.shape[0] == 0:\n            raise ValueError(\"Labels must have at least one row\")\n\n        if self.labels.shape[0] != self.data.shape[0]:\n            raise ValueError(\"Labels and data must have the same length\")\n\n        label_min = self.labels.min()\n\n        # Ensure _all sentinel doesn't collide with actual labels\n        if label_min < self._all:\n            len_label_min = len(str(abs(int(label_min))))\n            self._all = -int('9' * len_label_min)\n\n            # and just in case in case\n            if self._all == label_min:\n                self._all = -int('9' * (len_label_min + 1))\n\n        return self\n\n    @computed_field_cached_property()\n    def n_total(self) -> int:\n        return self.data.shape[0]\n\n    @computed_field_cached_property()\n    def unique_labels(self) -> np.ndarray:\n        return np.unique(self.labels)\n\n    @computed_field_cached_property()\n    def label_count(self) -> int:\n        return len(self.unique_labels)\n\n    @computed_field_cached_property()\n    def _label_indices(self) -> dict[int, np.ndarray]:\n        return {label: np.where(self.labels == label)[0]\n                for label in self.unique_labels}\n\n    @computed_field_cached_property()\n    def _n(self) -> np.ndarray:\n        cluster_sizes = [len(self._label_indices[label]) for label in self.unique_labels]\n        return np.array([self.n_total, *cluster_sizes])\n\n    @computed_field_cached_property()\n    def _mean(self) -> np.ndarray:\n        means = [np.mean(self.data, axis=0)]\n        for label in self.unique_labels:\n            means.append(np.mean(self.data[self._label_indices[label]], axis=0))\n\n        return np.array(means)\n\n    @computed_field_cached_property()\n    def _median(self) -> np.ndarray:\n        medians = [np.median(self.data, axis=0)]\n        for label in self.unique_labels:\n            medians.append(np.median(self.data[self._label_indices[label]], axis=0))\n\n        return np.array(medians)\n\n    @computed_field_cached_property()\n    def _var(self) -> np.ndarray:\n        variances = [np.var(self.data, axis=0)]\n        for label in self.unique_labels:\n            variances.append(np.var(self.data[self._label_indices[label]], axis=0))\n\n        return np.array(variances)\n\n    @computed_field_cached_property()\n    def _distance(self) -> np.ndarray:\n        return squareform(pdist(self.data))\n\n    @computed_field_cached_property()\n    def _distance_to_mean(self) -> np.ndarray:\n        return cdist(self.data, self._mean)\n\n    @computed_field_cached_property()\n    def _distance_to_median(self) -> np.ndarray:\n        return cdist(self.data, self._median)\n    \n    @computed_field_cached_property()\n    def _labeled_distance(self) -> dict[tuple[int, int], np.ndarray]:\n        data = {}\n        for label_i in self.unique_labels:\n            idx_i = self._label_indices[label_i]\n\n            for label_j in self.unique_labels:\n                idx_j = self._label_indices[label_j]\n                data[label_i, label_j] = self._distance[np.ix_(idx_i, idx_j)]\n\n        return data\n\n    def _labeled_distance_to_centroid(self, distance_matrix: np.ndarray) -> dict[tuple[int, int], np.ndarray]:\n        unique_labels = [self._all, *self.unique_labels]\n        all_idx = np.arange(self.n_total)\n\n        data = {}\n        for label_i in unique_labels:\n            idx_i = all_idx if label_i == self._all else self._label_indices[label_i]\n\n            for col_idx, label_j in enumerate(unique_labels):\n                data[label_i, label_j] = distance_matrix[idx_i, col_idx]\n\n        return data\n\n    @computed_field_cached_property()\n    def _labeled_distance_to_mean(self) -> dict[tuple[int, int], np.ndarray]:\n        return self._labeled_distance_to_centroid(self._distance_to_mean)\n\n    @computed_field_cached_property()\n    def _labeled_distance_to_median(self) -> dict[tuple[int, int], np.ndarray]:\n        return self._labeled_distance_to_centroid(self._distance_to_median)\n\n    def _labeled_distance_to_nearest_cluster(self, agg: str = \"mean\") -> dict[int, np.ndarray]:\n        agg_fcn = np.mean if agg == \"mean\" else np.median\n\n        data = {}\n        for label_i in self.unique_labels:\n            n = self._labeled_distance[label_i, label_i].shape[0]\n            dist_to_nearest = np.full(n, np.inf)\n\n            for label_j in self.unique_labels:\n                if label_i == label_j:\n                    continue\n\n                dist_matrix = self._labeled_distance[label_i, label_j]\n                avg_dists = agg_fcn(dist_matrix, axis=1)\n                dist_to_nearest = np.minimum(dist_to_nearest, avg_dists)\n\n            data[label_i] = dist_to_nearest\n\n        return data\n    \n    @computed_field_cached_property()\n    def _labeled_mean_distance_to_nearest_cluster(self) -> dict[int, np.ndarray]:\n        return self._labeled_distance_to_nearest_cluster(agg=\"mean\")\n    \n    @computed_field_cached_property()\n    def _labeled_median_distance_to_nearest_cluster(self) -> dict[int, np.ndarray]:\n        return self._labeled_distance_to_nearest_cluster(agg=\"median\")\n\n    def _labeled_distance_intra_cluster(self, agg: str = \"mean\") -> dict[int, np.ndarray]:\n        data = {}\n        for label_i in self.unique_labels:\n            distance_array = self._labeled_distance[label_i, label_i]\n            n = distance_array.shape[0]\n\n            if agg == \"mean\":\n                # Mean excluding self: row_sum / (n-1), diagonal is 0\n                data[label_i] = np.sum(distance_array, axis=1) / (n - 1)\n            else:\n                # Median excluding self: mask diagonal with nan, use nanmedian\n                masked = distance_array.copy()\n                np.fill_diagonal(masked, np.nan)\n                data[label_i] = np.nanmedian(masked, axis=1)\n\n        return data\n    \n    @computed_field_cached_property()\n    def _labeled_mean_distance_intra_cluster(self) -> dict[int, np.ndarray]:\n        return self._labeled_distance_intra_cluster(agg=\"mean\")\n    \n    @computed_field_cached_property()\n    def _labeled_median_distance_intra_cluster(self) -> dict[int, np.ndarray]:\n        return self._labeled_distance_intra_cluster(agg=\"median\")\n        \n    @computed_field_cached_property()\n    def all(self) -> SingleClusterMetrics:\n        key = (self._all, self._all)\n\n        distance = ClusterPairDistanceMetrics(\n            distance=self._distance,\n        )\n\n        distance_to_mean = ClusterPairDistanceMetrics(\n            distance=self._labeled_distance_to_mean[key],\n        )\n        \n        distance_to_median = ClusterPairDistanceMetrics(\n            distance=self._labeled_distance_to_median[key],\n        )\n\n        return SingleClusterMetrics(\n            cluster_id=None,\n            n=self._n[0],\n            mean=self._mean[0],\n            median=self._median[0],\n            var=self._var[0],\n            distance=distance,\n            distance_to_mean=distance_to_mean,\n            distance_to_median=distance_to_median,\n        )\n\n    @computed_field_cached_property()\n    def cluster(self) -> dict[int, SingleClusterMetrics]:\n        data = {}\n        for i, label in enumerate(self.unique_labels):\n            # single cluster metrics\n            n = self._n[i + 1]\n            mean = self._mean[i + 1]\n            median = self._median[i + 1]\n            var = self._var[i + 1]\n\n            # pair distance metrics\n            distance = {}\n            distance_to_mean = {\"all\": ClusterPairDistanceMetrics(\n                cluster_ids=(label, self._all),\n                distance=self._labeled_distance_to_mean[(label, self._all)],\n            )}\n            distance_to_median = {\"all\": ClusterPairDistanceMetrics(\n                cluster_ids=(label, self._all),\n                distance=self._labeled_distance_to_median[(label, self._all)],\n            )}\n\n            for label_j in self.unique_labels:\n                key = (label, label_j)\n\n                distance[label_j] = ClusterPairDistanceMetrics(\n                    cluster_ids=key,\n                    distance=self._labeled_distance[key],\n                )\n\n                distance_to_mean[label_j] = ClusterPairDistanceMetrics(\n                    cluster_ids=key,\n                    distance=self._labeled_distance_to_mean[key],\n                )\n\n                distance_to_median[label_j] = ClusterPairDistanceMetrics(\n                    cluster_ids=key,\n                    distance=self._labeled_distance_to_median[key],\n                )\n\n            mean_distance_intra_cluster = self._labeled_mean_distance_intra_cluster[label]\n            median_distance_intra_cluster = self._labeled_median_distance_intra_cluster[label]\n            mean_distance_to_nearest_cluster = self._labeled_mean_distance_to_nearest_cluster[label]\n            median_distance_to_nearest_cluster = self._labeled_median_distance_to_nearest_cluster[label]\n            \n            data[label] = SingleClusterMetrics(\n                cluster_id=label,\n                n=n,\n                mean=mean,\n                median=median,\n                var=var,\n                distance=distance,\n                distance_to_mean=distance_to_mean,\n                distance_to_median=distance_to_median,\n                mean_distance_intra_cluster=mean_distance_intra_cluster,\n                median_distance_intra_cluster=median_distance_intra_cluster,\n                mean_distance_to_nearest_cluster=mean_distance_to_nearest_cluster,\n                median_distance_to_nearest_cluster=median_distance_to_nearest_cluster,\n            )\n\n        return data\n\n    # -------------------------------------------------------------------------\n    # Private infrastructure: scatter matrices, sum-of-squares, pairwise vectors\n    # -------------------------------------------------------------------------\n\n    @computed_field_cached_property()\n    def _WCSM(self) -> dict[int, np.ndarray]:\n        \"\"\"\n        Within-Cluster Scatter Matrices\n        Returns a dictionary mapping cluster labels to their scatter matrices\n        \"\"\"\n        # Compute scatter matrix for each cluster\n        scatter_matrices = {}\n        for i, label in enumerate(self.unique_labels):\n            cluster_data = self.data[self._label_indices[label]]\n            cluster_mean = self._mean[i + 1]\n\n            # Compute scatter matrix for this cluster: Σ(x - mean)(x - mean)^T\n            centered_data = cluster_data - cluster_mean\n            scatter_matrices[label] = centered_data.T @ centered_data\n\n        return scatter_matrices\n\n    @computed_field_cached_property()\n    def _sum_WCSM(self) -> np.ndarray:\n        \"\"\"\n        Pooled Within-Cluster Scatter Matrix\n        Returns the sum of all within-cluster scatter matrices\n        \"\"\"\n        return sum(self._WCSM.values())\n\n    @computed_field_cached_property()\n    def _TSM(self) -> np.ndarray:\n        \"\"\"\n        Total Scatter Matrix\n        \"\"\"\n        centered_data = self.data - self._mean[0]\n        return centered_data.T @ centered_data\n\n    @computed_field_cached_property()\n    def _WCSS(self) -> float:\n        \"\"\"\n        Within-Cluster Sum of Squares\n        Computed as the trace of the summed within-cluster scatter matrix\n        \"\"\"\n        return np.trace(self._sum_WCSM)\n\n    @computed_field_cached_property()\n    def _BCSS(self) -> float:\n        \"\"\"\n        Between-Cluster Sum of Squares\n        \"\"\"\n        diffs = self._mean[1:] - self._mean[0]\n        sq_dists = np.sum(diffs ** 2, axis=1)\n        BCSS = np.dot(self._n[1:], sq_dists)\n\n        return BCSS\n\n    @computed_field_cached_property()\n    def _WC_pairwise_distances(self) -> np.ndarray:\n        \"\"\"Aggregated within-cluster pairwise distances across all clusters\"\"\"\n        parts = [\n            c.within_pairwise_distances\n            for c in self.cluster.values()\n            if c.within_pairwise_distances is not None and len(c.within_pairwise_distances) > 0\n        ]\n        return np.concatenate(parts) if parts else np.array([])\n\n    @computed_field_cached_property()\n    def _BC_pairwise_distances(self) -> np.ndarray:\n        \"\"\"Aggregated between-cluster pairwise distances (deduplicated across cluster pairs)\"\"\"\n        parts = []\n        labels = list(self.unique_labels)\n        for i, label_i in enumerate(labels):\n            for label_j in labels[i+1:]:\n                parts.append(self.cluster[label_i].distance[label_j].distance.ravel())\n        return np.concatenate(parts) if parts else np.array([])\n\n    @computed_field_cached_property()\n    def _mean_scatter(self) -> float:\n        \"\"\"Average scattering: (1/K) × Σ_k ||σ(C_k)|| / ||σ(D)||\"\"\"\n        total_var_norm = self.all.var_norm\n        if total_var_norm is None or total_var_norm < self._eps:\n            return 0.0\n\n        cluster_var_norms = np.array([\n            self.cluster[label].var_norm\n            for label in self.unique_labels\n        ])\n        return np.mean(cluster_var_norms) / total_var_norm\n\n    # -------------------------------------------------------------------------\n    # Compactness indices (within-cluster quality only)\n    # -------------------------------------------------------------------------\n\n    @computed_field_cached_property()\n    def sum_of_squared_errors_index(self) -> float:\n        # Sum of Squared Errors (SSE) Index\n        # Range is 0 to inf, 0 is the best\n        # Formula: SSE = Σ_k Σ_{x_i ∈ C_k} ||x_i - c_k||²\n        # This is equivalent to the Within-Cluster Sum of Squares (WCSS)\n\n        # Within Cluster Sum of Squares (WCSS) = SSE\n        res = self._WCSS\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def mean_squared_error_index(self) -> float:\n        # Mean Squared Error (MSE) Index\n        # Range is 0 to inf, 0 is the best\n        # Formula: MSE = SSE / n = WCSS / n\n        # where SSE = sum of squared errors,\n        #       n = total number of data points\n\n        n = self.n_total  # number of data points\n        WCSS = self._WCSS\n\n        res = WCSS / n\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def ball_hall_index(self) -> float:\n        # Ball and Hall Index\n        # Range is 0 to inf, 0 is the best\n        # Formula: (1/K) * Σ(sum of squared distances from points to cluster centroids)\n\n        k = self.label_count  # number of clusters\n        WCSS = self._WCSS  # Within Cluster Sum of Squares (WCSS)\n        res = WCSS / k\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def banfeld_raftery_index(self) -> float:\n        # Banfeld-Raftery Index\n        # Range is -inf to inf, -inf is the best\n        # Formula: Σ [n_k × log(trace(W_k) / n_k)]\n        # where n_k = number of points in cluster k,\n        #       trace(W_k) = sum of squared distances to centroid for cluster k\n\n        n_k = self._n[1:]  # cluster sizes\n        traces = np.array([np.trace(self._WCSM[label]) for label in self.unique_labels])\n\n        # Replace zero traces with _eps to avoid log(0)\n        traces_safe = np.where(traces > 0, traces, self._eps)\n        res = np.sum(n_k * np.log(traces_safe / n_k))\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def scott_symons_index(self) -> float:\n        # Scott-Symons Index\n        # Range is -inf to inf, -inf is the best (minimize)\n        # Formula: Σ n_k × log(det(W_k / n_k))\n        # where W_k = scatter matrix for cluster k,\n        #       n_k = number of points in cluster k\n\n        res = 0.0\n        for i, label in enumerate(self.unique_labels):\n            n_k = self._n[i + 1]\n            W_k = self._WCSM[label]\n\n            try:\n                sign, logdet = np.linalg.slogdet(W_k / n_k)\n                if sign <= 0:\n                    logdet = np.log(self._eps)\n                res += n_k * logdet\n            except np.linalg.LinAlgError:\n                res += n_k * np.log(self._eps)\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def trace_w_index(self) -> float:\n        # Trace W Index\n        # Range is 0 to inf, 0 is the best (minimize)\n        # Formula: trace(W)\n        # where W = pooled within-cluster scatter matrix\n        # Equivalent to WCSS but formalized as a matrix trace measure\n\n        res = np.trace(self._sum_WCSM)\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    # -------------------------------------------------------------------------\n    # Compactness + Separation indices (combined)\n    # -------------------------------------------------------------------------\n\n    def _silhouette_coefficients(self, variant: str = \"mean\") -> np.ndarray:\n        if variant == \"mean\":\n            intra = self._labeled_mean_distance_intra_cluster\n            nearest = self._labeled_mean_distance_to_nearest_cluster\n        else:\n            intra = self._labeled_median_distance_intra_cluster\n            nearest = self._labeled_median_distance_to_nearest_cluster\n\n        idx = 0\n        coefficients = np.empty(self.n_total)\n        for label in self.unique_labels:\n            a = intra[label]\n            b = nearest[label]\n            coefs = (b - a) / np.maximum(a, b)\n            n_points = len(coefs)\n            coefficients[idx:idx+n_points] = coefs\n            idx += n_points\n\n        return coefficients\n\n    @computed_field_cached_property()\n    def silhouette_index(self) -> float:\n        # range is -1 to 1, 1 is the best\n        res = np.mean(self._silhouette_coefficients(\"mean\"))\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def silhouette_median_index(self) -> float:\n        # range is -1 to 1, 1 is the best\n        res = np.median(self._silhouette_coefficients(\"median\"))\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def davies_bouldin_index(self) -> float:\n        # range is 0 to inf, 0 is the best\n        k = self.label_count\n        # Could abstract with scipy.stats.moment\n\n        intracluster_distance = np.empty(k)\n        for i, label in enumerate(self.unique_labels):\n            intracluster_distance[i] = np.mean(self._labeled_distance_to_mean[(label, label)])\n\n        intercluster_distance = squareform(pdist(self._mean[1:]))\n\n        # Compute similarity matrix using broadcasting\n        # similarity[i,j] = (dist_i + dist_j) / dist_ij for i != j\n        with np.errstate(divide='ignore', invalid='ignore'):\n            similarity = (intracluster_distance[:, None] + intracluster_distance[None, :]) / intercluster_distance\n\n        # Set diagonal to 0 (i == j case)\n        np.fill_diagonal(similarity, 0)\n\n        res = np.sum(np.max(similarity, axis=1)) / k\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def calinski_harabasz_index(self) -> float:\n        # range is 0 to inf, inf is the best\n        k = self.label_count # number of clusters\n        n = self.n_total # number of data points\n\n        BCSS = self._BCSS  # Between Cluster Sum of Squares (BCSS)\n        WCSS = self._WCSS  # Within Cluster Sum of Squares (WCSS)\n\n        if WCSS < self._eps:\n            return 1.0\n\n        res = (BCSS / WCSS) * ((n - k) / (k - 1.0))\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def variance_ratio_criterion(self) -> float:\n        return self.calinski_harabasz_index\n\n    @computed_field_cached_property()\n    def dunn_index(self) -> float:\n        # Dunn Index\n        # Range is 0 to inf, inf is the best\n        # Formula: min(inter-cluster distance) / max(intra-cluster diameter)\n        # where inter-cluster distance = min distance between points in different clusters\n        #       intra-cluster diameter = max distance between points in same cluster\n\n        n_clusters = len(self.unique_labels)\n\n        use_modified = True\n\n        # Find minimum inter-cluster distance\n        min_inter_distance = np.inf\n        if use_modified:\n            # Modified: min distance from any point in cluster k1 to centroid of cluster k0\n            # Reuse precomputed distances to means: column i+1 is distance to cluster i's mean\n            for k0 in range(n_clusters - 1):\n                for k1 in range(k0 + 1, n_clusters):\n                    label_k1 = self.unique_labels[k1]\n                    # _distance_to_mean column k0+1 has distances to cluster k0's centroid\n                    dists = self._distance_to_mean[self._label_indices[label_k1], k0 + 1]\n                    min_inter_distance = min(min_inter_distance, np.min(dists))\n        else:\n            # Standard: distance between all point pairs in different clusters\n            for i, label_i in enumerate(self.unique_labels):\n                for label_j in self.unique_labels[i+1:]:\n                    min_dist = np.min(self._labeled_distance[label_i, label_j])\n                    min_inter_distance = min(min_inter_distance, min_dist)\n\n        # Find maximum intra-cluster diameter\n        max_intra_diameter = max(\n            np.max(self._labeled_distance[label, label])\n            for label in self.unique_labels\n        )\n\n        # Avoid division by zero\n        if max_intra_diameter < self._eps:\n            res = np.inf\n        else:\n            res = min_inter_distance / max_intra_diameter\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def xie_beni_index(self) -> float:\n        # Xie-Beni Index\n        # Range is 0 to inf, 0 is the best\n        # Formula: WCSS / (n × d_min²)\n        # where WCSS = within-cluster sum of squares,\n        #       n = number of data points,\n        #       d_min = minimum distance between cluster centroids\n\n        n = self.n_total  # number of data points\n        cluster_means = self._mean[1:] # Skip the overall mean at index 0\n\n        # define numerator\n        use_assigned_cluster_centroids = True\n\n        if use_assigned_cluster_centroids:\n            num = self._WCSS\n        else: # uses nearest centroid instead\n            # Compute WGSS: sum of squared distances from each point to its nearest centroid\n            # This matches the standard Xie-Beni definition\n            d_sq_to_centroids = cdist(\n                self.data,\n                cluster_means,\n                metric='sqeuclidean'\n            )\n            min_d_sq_to_centroids = np.min(d_sq_to_centroids, axis=1)\n            num = np.sum(min_d_sq_to_centroids)\n\n        # Calculate squared pairwise distances between centroids\n        if len(cluster_means) > 1:\n            d_sq = pdist(\n                cluster_means,\n                metric='sqeuclidean'\n            )\n            d_min_squared = np.min(d_sq)\n        else:\n            # If only one cluster, return infinity (worst score)\n            return np.inf if self.index_direction == \"minimize\" else -np.inf\n\n        # Avoid division by zero\n        if d_min_squared < self._eps:\n            res = np.inf\n        else:\n            res = num / (n * d_min_squared)\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def duda_hart_index(self) -> float:\n        # Duda and Hart Index\n        # Range is 0 to inf, 0 is the best\n\n        intracluster_distance = 0\n        intercluster_distance = 0\n        for label_i in self.unique_labels:\n            intracluster_distance += np.mean(self._labeled_distance_to_mean[(label_i, label_i)])\n\n            # Mean distance from points in label_i to all points NOT in label_i\n            inter_dists = [self._labeled_distance[label_i, label_j]\n                           for label_j in self.unique_labels if label_j != label_i]\n            intercluster_distance += np.mean(np.hstack(inter_dists))\n\n        res = intracluster_distance / intercluster_distance\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def c_index(self) -> float:\n        # C-Index\n        # Range is 0 to 1, 0 is the best\n        # Formula: (S_w - S_min) / (S_max - S_min)\n        # where S_w = sum of within-cluster pairwise distances,\n        #       S_min = sum of the N_w smallest pairwise distances overall,\n        #       S_max = sum of the N_w largest pairwise distances overall,\n        #       N_w = number of within-cluster pairs\n\n        within_dists = self._WC_pairwise_distances\n        n_w = len(within_dists)\n\n        if n_w == 0:\n            return 0.0\n\n        S_w = np.sum(within_dists)\n\n        all_dists = self._distance[np.triu_indices(self.n_total, k=1)]\n        all_dists_sorted = np.sort(all_dists)\n\n        S_min = np.sum(all_dists_sorted[:n_w])\n        S_max = np.sum(all_dists_sorted[-n_w:])\n\n        denom = S_max - S_min\n        if denom < self._eps:\n            res = 0.0\n        else:\n            res = (S_w - S_min) / denom\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def mcclain_rao_index(self) -> float:\n        # McClain-Rao Index\n        # Range is 0 to inf, 0 is the best\n        # Formula: mean(within-cluster distances) / mean(between-cluster distances)\n\n        within_dists = self._WC_pairwise_distances\n        between_dists = self._BC_pairwise_distances\n\n        if len(within_dists) == 0 or len(between_dists) == 0:\n            return np.inf\n\n        mean_within = np.mean(within_dists)\n        mean_between = np.mean(between_dists)\n\n        if mean_between < self._eps:\n            res = np.inf\n        else:\n            res = mean_within / mean_between\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def i_index(self) -> float:\n        # I-Index (Maulik-Bandyopadhyay)\n        # Range is 0 to inf, inf is the best\n        # Formula: I(K) = (1/K × E_1/E_K × D_K)^p\n        # where E_1 = Σ ||x_i - grand_centroid|| (total distance to grand mean),\n        #       E_K = Σ_k Σ_{x ∈ C_k} ||x - c_k|| (total distance to cluster centroids),\n        #       D_K = max inter-centroid distance,\n        #       p = 2\n\n        k = self.label_count\n        p = 2\n\n        # E_1: sum of distances from all points to grand centroid\n        E_1 = np.sum(self._labeled_distance_to_mean[(self._all, self._all)])\n\n        # E_K: sum of distances from each point to its assigned cluster centroid\n        E_K = 0.0\n        for label in self.unique_labels:\n            E_K += np.sum(self._labeled_distance_to_mean[(label, label)])\n\n        # D_K: max distance between any pair of cluster centroids\n        cluster_means = self._mean[1:]\n        if len(cluster_means) > 1:\n            D_K = np.max(pdist(cluster_means))\n        else:\n            return 0.0\n\n        if E_K < self._eps:\n            res = np.inf\n        else:\n            res = ((1.0 / k) * (E_1 / E_K) * D_K) ** p\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def log_ss_ratio_index(self) -> float:\n        # Log SS Ratio Index (Log Sum of Squares Ratio)\n        # Range is -inf to inf, inf is the best\n        # Formula: log(BCSS / WCSS) = log(BCSS) - log(WCSS)\n        # where BCSS = between-cluster sum of squares,\n        #       WCSS = within-cluster sum of squares\n\n        BCSS = self._BCSS  # Between Cluster Sum of Squares (BCSS)\n        WCSS = self._WCSS  # Within Cluster Sum of Squares (WCSS)\n\n        # Avoid log of zero or division by zero\n        if WCSS < self._eps:\n            res = np.inf\n        elif BCSS < self._eps:\n            res = -np.inf\n        else:\n            # log(BCSS / WCSS) = log(BCSS) - log(WCSS)\n            res = np.log(BCSS) - np.log(WCSS)\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    # -------------------------------------------------------------------------\n    # Statistical / Correlation indices\n    # -------------------------------------------------------------------------\n\n    @computed_field_cached_property()\n    def gamma_index(self) -> float:\n        # Hubert's Gamma Index\n        # Range is -1 to 1, 1 is the best\n        # Concordance measure: Γ = (s+ - s-) / (s+ + s-)\n        # where s+ = concordant pairs (within < between),\n        #       s- = discordant pairs (within > between)\n\n        within_dists = self._WC_pairwise_distances\n        between_dists = self._BC_pairwise_distances\n\n        n_b = len(between_dists)\n        if n_b == 0 or len(within_dists) == 0:\n            return 0.0\n\n        between_sorted = np.sort(between_dists)\n\n        # For each within_dist, count concordant (between > within) and discordant (between < within)\n        left_indices = np.searchsorted(between_sorted, within_dists, side='left')\n        right_indices = np.searchsorted(between_sorted, within_dists, side='right')\n\n        s_plus = np.sum(n_b - right_indices)   # between > within (concordant)\n        s_minus = np.sum(left_indices)          # between < within (discordant)\n\n        denom = s_plus + s_minus\n        if denom == 0:\n            res = 0.0\n        else:\n            res = float(s_plus - s_minus) / float(denom)\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def point_biserial_index(self) -> float:\n        # Point-Biserial Correlation\n        # Range is -1 to 1, 1 is the best\n        # Formula: r_pb = (M_b - M_w) / s_d × sqrt(n_w × n_b / n_t²)\n        # where M_b = mean between-cluster distance,\n        #       M_w = mean within-cluster distance,\n        #       s_d = std of all pairwise distances,\n        #       n_w, n_b = number of within/between pairs\n\n        within_dists = self._WC_pairwise_distances\n        between_dists = self._BC_pairwise_distances\n\n        n_w = len(within_dists)\n        n_b = len(between_dists)\n        n_t = n_w + n_b\n\n        if n_w == 0 or n_b == 0:\n            return 0.0\n\n        mean_within = np.mean(within_dists)\n        mean_between = np.mean(between_dists)\n\n        all_dists = np.concatenate([within_dists, between_dists])\n        std_all = np.std(all_dists)\n\n        if std_all < self._eps:\n            return 0.0\n\n        res = ((mean_between - mean_within) / std_all) * np.sqrt(n_w * n_b / (n_t ** 2))\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    # -------------------------------------------------------------------------\n    # Matrix / Determinant indices\n    # -------------------------------------------------------------------------\n\n    @computed_field_cached_property()\n    def _det_ratio(self) -> float:\n        \"\"\"\n        Raw det(T) / det(W)\n        \"\"\"\n        try:\n            det_T = np.linalg.det(self._TSM)\n            det_W = np.linalg.det(self._sum_WCSM)\n\n            if np.abs(det_W) < self._eps:\n                return np.inf\n            else:\n                return det_T / det_W\n\n        except np.linalg.LinAlgError:\n            return np.inf\n\n    @computed_field_cached_property()\n    def ksq_detw_index(self) -> float:\n        # KSq-DetW Index (K² × det(W))\n        # Range is -inf to inf, inf is the best\n        # Formula: K² × det(W)\n        # where K = number of clusters,\n        #       W = summed within-cluster scatter matrix (normalized by default),\n        #       det(W) = determinant of W\n\n        k = self.label_count  # number of clusters\n        normalize_scatter_matrix = True\n\n        # Get summed within-cluster scatter matrix\n        W = self._sum_WCSM\n\n        # Apply normalization if enabled\n        if normalize_scatter_matrix:\n            W_min = np.min(W)\n            W_max = np.max(W)\n            W = (W - W_min) / (W_max - W_min)\n\n        # Compute determinant\n        try:\n            det_W = np.linalg.det(W)\n        except np.linalg.LinAlgError:\n            # Handle singular matrix\n            det_W = 0\n\n        # KSq-DetW = K² × det(W)\n        res = (k ** 2) * det_W\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def det_ratio_index(self) -> float:\n        # Det Ratio Index\n        # Range is 0 to inf, inf is the best\n        # Formula: det(T) / det(W)\n        # where T = total scatter matrix (covariance of all data),\n        #       W = summed within-cluster scatter matrix\n\n        res = self._det_ratio\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def log_det_ratio_index(self) -> float:\n        # Log Det Ratio Index\n        # Range is -inf to inf, inf is the best\n        # Formula: n * log(det(T) / det(W)) = n * (log(det(T)) - log(det(W)))\n        # where T = total scatter matrix,\n        #       W = summed within-cluster scatter matrix,\n        #       n = number of data points\n\n        n = self.n_total\n        res = n * np.log(np.abs(self._det_ratio))\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def trace_wb_index(self) -> float:\n        # Trace WB Index (Trace of W^-1 × B)\n        # Range is 0 to inf, inf is the best (maximize)\n        # Formula: trace(W^-1 × B) where B = T - W\n        # Multivariate generalization of Calinski-Harabasz\n\n        W = self._sum_WCSM\n        B = self._TSM - W\n\n        try:\n            W_inv = np.linalg.inv(W)\n        except np.linalg.LinAlgError:\n            W_inv = np.linalg.pinv(W)\n\n        res = np.trace(W_inv @ B)\n\n        if self.index_direction == \"minimize\":\n            res *= -1\n\n        return res\n\n    # -------------------------------------------------------------------------\n    # Density-based indices\n    # -------------------------------------------------------------------------\n\n    @computed_field_cached_property()\n    def s_dbw_index(self) -> float:\n        # S_Dbw Index (Halkidi and Vazirgiannis, 2001)\n        # Range is 0 to inf, 0 is the best (minimize)\n        # Formula: S_Dbw = Scat + Dens_bw\n        # Scat = average scattering (cluster variance / total variance)\n        # Dens_bw = average inter-cluster density at midpoints between centroids\n\n        k = self.label_count\n        if k < 2:\n            return self._mean_scatter\n\n        scat = self._mean_scatter\n\n        # stdev: average norm of cluster variance vectors (neighborhood radius)\n        cluster_var_norms = np.array([\n            self.cluster[label].var_norm\n            for label in self.unique_labels\n        ])\n        stdev = np.mean(cluster_var_norms)\n\n        if stdev < self._eps:\n            # All clusters are single points; no inter-cluster density\n            res = scat\n            if self.index_direction == \"maximize\":\n                res *= -1\n            return res\n\n        cluster_means = self._mean[1:]\n\n        # Compute Dens_bw: for each pair (i, j), evaluate density at midpoint\n        # relative to density at the denser centroid\n        dens_bw_sum = 0.0\n        for i in range(k):\n            label_i = self.unique_labels[i]\n            idx_i = self._label_indices[label_i]\n\n            for j in range(k):\n                if i == j:\n                    continue\n\n                label_j = self.unique_labels[j]\n                idx_j = self._label_indices[label_j]\n\n                # Union of points in clusters i and j\n                union_idx = np.concatenate([idx_i, idx_j])\n                union_data = self.data[union_idx]\n\n                # Midpoint between centroids\n                u_ij = (cluster_means[i] + cluster_means[j]) / 2.0\n\n                # Count points within stdev of each reference point\n                density_midpoint = np.sum(\n                    np.linalg.norm(union_data - u_ij, axis=1) <= stdev\n                )\n                density_ci = np.sum(\n                    np.linalg.norm(union_data - cluster_means[i], axis=1) <= stdev\n                )\n                density_cj = np.sum(\n                    np.linalg.norm(union_data - cluster_means[j], axis=1) <= stdev\n                )\n\n                max_density = max(density_ci, density_cj)\n                if max_density > 0:\n                    dens_bw_sum += density_midpoint / max_density\n\n        dens_bw = dens_bw_sum / (k * (k - 1))\n\n        res = scat + dens_bw\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def sd_validity_index(self) -> float:\n        # SD Validity Index (Halkidi, Vazirgiannis, Batistakis, 2000)\n        # Range is 0 to inf, 0 is the best (minimize)\n        # Formula: SD = α × Scat(K) + Dis(K)\n        # Scat = (1/K) × Σ_k ||σ(C_k)|| / ||σ(D)||\n        # Dis = (D_max/D_min) × Σ_k (Σ_j ||c_k - c_j||)^{-1}\n        # α = 1.0 (default; in multi-K sweeps this is set to Dis(K_max))\n\n        k = self.label_count\n        scat = self._mean_scatter\n\n        if k < 2:\n            res = scat\n            if self.index_direction == \"maximize\":\n                res *= -1\n            return res\n\n        # Dis component: separation based on centroid distances\n        cluster_means = self._mean[1:]\n        centroid_dists = squareform(pdist(cluster_means))\n\n        D_max = np.max(centroid_dists)\n\n        # D_min: minimum non-zero inter-centroid distance\n        centroid_dists_no_diag = centroid_dists.copy()\n        np.fill_diagonal(centroid_dists_no_diag, np.inf)\n        D_min = np.min(centroid_dists_no_diag)\n\n        if D_min < self._eps:\n            dis = np.inf\n        else:\n            # Σ_k (Σ_j ||c_k - c_j||)^{-1}\n            row_sums = np.sum(centroid_dists, axis=1)\n            row_sums_safe = np.where(row_sums > self._eps, row_sums, self._eps)\n            dis = (D_max / D_min) * np.sum(1.0 / row_sums_safe)\n\n        alpha = 1.0\n        res = alpha * scat + dis\n\n        if self.index_direction == \"maximize\":\n            res *= -1\n\n        return res\n\n    @computed_field_cached_property()\n    def density_based_clustering_validation_index(self) -> float:\n        # Density-Based Clustering Validation Index\n        # https://epubs.siam.org/doi/pdf/10.1137/1.9781611973440.96\n        # Metric is between -1 and 1, 1 is the best\n\n        precomputed_distances = self._distance\n        # if self.distance_metric == DistanceMetric.EUCLIDEAN:\n        #     precomputed_distances = np.power(precomputed_distances, 2)\n\n        dbcvi = dbcv(\n            X = self.data,\n            y = self.labels,\n            precomputed_distances = precomputed_distances,\n            metric = self.distance_metric,\n            noise_id = -1, # what label is the noise index\n            check_duplicates = False,\n            n_processes = 1,\n            enable_dynamic_precision = False,\n            bits_of_precision = 512,\n            use_original_mst_implementation = False\n        )\n\n        if self.index_direction == \"minimize\":\n            dbcvi *= -1\n\n        return dbcvi"
  },
  {
    "path": "opendsm/common/clustering/metrics/density_based_clustering_validation.py",
    "content": "\"\"\"\nFrom https://github.com/FelSiq/DBCV\n\nMIT License\n\nCopyright (c) 2024 Felipe Alves Siqueira\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\"\"\"\n\nimport multiprocessing\nimport typing as t\nimport itertools\nimport functools\n\nimport numpy as np\nimport numpy.typing as npt\nimport sklearn.neighbors\nimport scipy.spatial.distance\nimport scipy.sparse.csgraph\nimport scipy.stats\nimport mpmath\n\n\n\n_MP = mpmath.mp.clone()\n\n\ndef prim_mst(\n    graph: npt.NDArray[np.float32], ind_root: int = 0\n) -> npt.NDArray[np.float32]:\n    \"\"\"Python translation of the original implementation of Prim's MST in MATLAB.\n\n    Reference source: https://github.com/pajaskowiak/dbcv/blob/main/src/MST_Edges.m\n    \"\"\"\n\n    n = len(graph)\n    intree = np.full(n, fill_value=False)\n    d = np.full(n, fill_value=np.inf)\n\n    d[ind_root] = 0\n    v = ind_root\n    counter = 0\n\n    G = {\n        \"MST_edges\": {\n            \"node_inds\": np.zeros((n - 1, 2), dtype=int),\n            \"weights\": np.zeros(n - 1, dtype=float),\n        },\n        \"MST_degrees\": np.zeros(n, dtype=int),\n        \"MST_parent\": np.arange(n),\n    }\n\n    while counter < n - 1:\n        intree[v] = True\n        dist = np.inf\n\n        for w in np.arange(n):\n            if w != v and not intree[w]:\n                weight = graph[v, w]\n\n                if d[w] > weight:\n                    d[w] = weight\n                    G[\"MST_parent\"][w] = v\n\n                if dist > d[w]:\n                    dist = d[w]\n                    next_v = w\n\n        counter += 1\n        G[\"MST_edges\"][\"node_inds\"][counter - 1, :] = (G[\"MST_parent\"][next_v], next_v)\n        G[\"MST_edges\"][\"weights\"][counter - 1] = graph[G[\"MST_parent\"][next_v], next_v]\n        G[\"MST_degrees\"][G[\"MST_parent\"][next_v]] += 1\n        G[\"MST_degrees\"][next_v] += 1\n        v = next_v\n\n    (inds_a, inds_b) = G[\"MST_edges\"][\"node_inds\"].T\n    weights = G[\"MST_edges\"][\"weights\"]\n\n    mst = np.zeros_like(graph)\n    mst[inds_a, inds_b] = weights\n    mst[inds_b, inds_a] = weights\n\n    return mst\n\n\ndef compute_pair_to_pair_dists(\n    X: npt.NDArray[np.float64], metric: str\n) -> npt.NDArray[np.float64]:\n    dists = scipy.spatial.distance.cdist(X, X, metric=metric)\n    np.maximum(dists, 1e-12, out=dists)\n    # NOTE: set self-distance to +inf to prevent points being self-neighbors.\n    np.fill_diagonal(dists, val=np.inf)\n    return dists\n\n\ndef get_subarray(\n    arr: npt.NDArray[np.float64],\n    /,\n    inds_a: t.Optional[npt.NDArray[np.int32]] = None,\n    inds_b: t.Optional[npt.NDArray[np.int32]] = None,\n) -> npt.NDArray[np.float64]:\n    if inds_a is None:\n        return arr\n    if inds_b is None:\n        inds_b = inds_a\n    inds_a_mesh, inds_b_mesh = np.meshgrid(inds_a, inds_b)\n    return arr[inds_a_mesh, inds_b_mesh].T\n\n\ndef get_internal_objects(\n    mutual_reach_dists: npt.NDArray[np.float64], use_original_mst_implementation: bool\n) -> npt.NDArray[np.float64]:\n    if use_original_mst_implementation:\n        mutual_reach_dists = np.copy(mutual_reach_dists)\n        np.fill_diagonal(mutual_reach_dists, 0.0)\n        mst = prim_mst(mutual_reach_dists)\n\n    else:\n        mst = scipy.sparse.csgraph.minimum_spanning_tree(mutual_reach_dists)\n        mst = mst.toarray()\n        mst += mst.T\n\n    is_mst_edges = (mst > 0.0).astype(int, copy=False)\n\n    internal_node_inds = is_mst_edges.sum(axis=0) > 1\n    internal_node_inds = np.flatnonzero(internal_node_inds)\n\n    internal_edge_weights = get_subarray(mst, inds_a=internal_node_inds)\n\n    graph_has_internal_nodes = bool(internal_node_inds.size > 0)\n    graph_has_at_least_two_internal_nodes = bool(internal_edge_weights.size > 1)\n\n    return (\n        (\n            internal_node_inds\n            if graph_has_internal_nodes\n            else np.arange(mutual_reach_dists.shape[0])\n        ),\n        internal_edge_weights if graph_has_at_least_two_internal_nodes else mst,\n    )\n\n\ndef compute_cluster_core_distance(\n    dists: npt.NDArray[np.float64], d: int, enable_dynamic_precision: bool\n) -> npt.NDArray[np.float64]:\n    n, _ = dists.shape\n    orig_dists_dtype = dists.dtype\n\n    if enable_dynamic_precision:\n        dists = np.asarray(_MP.matrix(dists), dtype=object).reshape(*dists.shape)\n\n    core_dists = np.power(dists, -d).sum(axis=-1, keepdims=True) / (n - 1)\n\n    if not enable_dynamic_precision:\n        np.clip(core_dists, a_min=0.0, a_max=1e12, out=core_dists)\n\n    np.power(core_dists, -1.0 / d, out=core_dists)\n\n    if enable_dynamic_precision:\n        core_dists = np.asarray(core_dists, dtype=orig_dists_dtype)\n\n    return core_dists\n\n\ndef compute_mutual_reach_dists(\n    dists: npt.NDArray[np.float64],\n    d: float,\n    enable_dynamic_precision: bool,\n) -> npt.NDArray[np.float64]:\n    core_dists = compute_cluster_core_distance(\n        d=d, dists=dists, enable_dynamic_precision=enable_dynamic_precision\n    )\n    mutual_reach_dists = dists.copy()\n    np.maximum(mutual_reach_dists, core_dists, out=mutual_reach_dists)\n    np.maximum(mutual_reach_dists, core_dists.T, out=mutual_reach_dists)\n    return (core_dists, mutual_reach_dists)\n\n\ndef fn_density_sparseness(\n    cls_inds: npt.NDArray[np.int32],\n    dists: npt.NDArray[np.float64],\n    d: int,\n    enable_dynamic_precision: bool,\n    use_original_mst_implementation: bool,\n) -> t.Tuple[float, npt.NDArray[np.float32], npt.NDArray[np.int32]]:\n    (core_dists, mutual_reach_dists) = compute_mutual_reach_dists(\n        dists=dists, d=d, enable_dynamic_precision=enable_dynamic_precision\n    )\n    (internal_node_inds, internal_edge_weights) = get_internal_objects(\n        mutual_reach_dists,\n        use_original_mst_implementation=use_original_mst_implementation,\n    )\n    dsc = float(internal_edge_weights.max())\n    internal_core_dists = core_dists[internal_node_inds]\n    internal_node_inds = cls_inds[internal_node_inds]\n    return (dsc, internal_core_dists, internal_node_inds)\n\n\ndef fn_density_separation(\n    cls_i: int,\n    cls_j: int,\n    dists: npt.NDArray[np.float64],\n    internal_core_dists_i: npt.NDArray[np.float64],\n    internal_core_dists_j: npt.NDArray[np.float64],\n) -> t.Tuple[int, int, float]:\n    sep = dists.copy()\n    np.maximum(sep, internal_core_dists_i, out=sep)\n    np.maximum(sep, internal_core_dists_j.T, out=sep)\n    dspc_ij = float(sep.min()) if sep.size else np.inf\n    return (cls_i, cls_j, dspc_ij)\n\n\ndef _check_duplicated_samples(X: npt.NDArray[np.float64], threshold: float = 1e-9):\n    if X.shape[0] <= 1:\n        return\n\n    nn = sklearn.neighbors.NearestNeighbors(n_neighbors=1)\n    nn.fit(X)\n    dists, _ = nn.kneighbors(return_distance=True)\n\n    if np.any(dists < threshold):\n        raise ValueError(\"Duplicated samples have been found in X.\")\n\n\ndef _convert_singleton_clusters_to_noise(\n    y: npt.NDArray[np.int32], noise_id: int\n) -> npt.NDArray[np.int32]:\n    \"\"\"Cast clusters containing a single instance as noise.\"\"\"\n    cluster_ids, cluster_sizes = np.unique(y, return_counts=True)\n    singleton_clusters = cluster_ids[cluster_sizes == 1]\n\n    if singleton_clusters.size == 0:\n        return y\n\n    return np.where(np.isin(y, singleton_clusters), noise_id, y)\n\n\ndef dbcv(\n    X: npt.NDArray[np.float64],\n    y: npt.NDArray[np.int32],\n    precomputed_distances: t.Optional[npt.NDArray[np.float64]] = None,\n    metric: str = \"sqeuclidean\",\n    noise_id: int = -1,\n    check_duplicates: bool = True,\n    n_processes: t.Union[int, str] = \"auto\",\n    enable_dynamic_precision: bool = False,\n    bits_of_precision: int = 512,\n    use_original_mst_implementation: bool = False,\n) -> float:\n    \"\"\"Compute DBCV metric.\n\n    Density-Based Clustering Validation (DBCV) is an intrinsic (= unsupervised/unlabeled)\n    relative metric. See reference [1] for the original reference.\n\n    Parameters\n    ----------\n    X : npt.NDArray[np.float64] of shape (N, D)\n        Sample embeddings.\n\n    y : npt.NDArray[np.int32] of shape (N,)\n        Cluster IDs assigned for each sample in X.\n\n    metric : str, default=\"sqeuclidean\"\n        This parameter specifies the metric function to compute dissimilarities between observations.\n        The DBCV metric estimation may vary depending on the distance metric used.\n        This argument is passed to `scipy.spatial.distance.cdist`.\n        The default value is the squared Euclidean distance, which is also employed in the original\n        MATLAB implementation (see reference [2]).\n\n    noise_id : int, default=-1\n        The noise \"cluster\" ID refers to instances where `y[i] = noise_id`, which are considered noise.\n        Additionally, singleton clusters, meaning clusters containing only a single instance, are automatically\n        classified as noise.\n\n    check_duplicates : bool, default=True\n        If set to True, check for duplicated samples before execution.\n        Instances with Euclidean distance to their nearest neighbor below 1e-9 are considered\n        duplicates.\n\n    n_processes : int or \"auto\", default=\"auto\"\n        Maximum number of parallel processes for processing clusters and cluster pairs.\n        If `n_processes=\"auto\"`, the number of parallel processes will be set to 1 for\n        datasets with 500 or fewer instances, and 4 for datasets with more than 500 instances.\n\n    enable_dynamic_precision : bool, default=False\n        If set to True, this activates a dynamic quantity of bits of precision for floating point during\n        density calculation, as defined by the `bits_of_precision` argument below. Enabling this argument\n        ensures proper density calculation for very high-dimensional data, although it significantly slows\n        down the process compared to standard calculations.\n\n    bits_of_precision : int, default=512\n        Bits of precision for density calculation. High values are necessary for high\n        dimensions to avoid underflow/overflow.\n\n    use_original_mst_implementation : bool, default=False\n        If set to False, the function will use Scipy's MST implementation (Kruskal's implementation).\n        If set to True, the function will use an exact replica of the original MATLAB implementation.\n        This version is a variant of Prim's MST algorithm.\n        The original implementation is slower than Scipy's implementation and tends to create hub nodes\n        much more often.\n        Since these implementations are not equivalent, the DBCV metric estimation tends to vary depending\n        on the MST algorithm used.\n\n    Returns\n    -------\n    DBCV : float\n        DBCV metric estimation.\n\n    Source\n    ------\n    .. [1] \"Density-Based Clustering Validation\". Davoud Moulavi, Pablo A. Jaskowiak,\n           Ricardo J. G. B. Campello, Arthur Zimek, Jörg Sander.\n           https://www.dbs.ifi.lmu.de/~zimek/publications/SDM2014/DBCV.pdf\n    .. [2] https://github.com/pajaskowiak/dbcv/\n    \"\"\"\n    X = np.asarray(X, dtype=np.float64)\n\n    if X.ndim == 1:\n        X = X.reshape(-1, 1)\n\n    y = np.asarray(y, dtype=int)\n\n    n, d = X.shape  # NOTE: 'n' must be calculated before removing noise.\n\n    if n != y.size:\n        raise ValueError(f\"Mismatch in {X.shape[0]=} and {y.size=} dimensions.\")\n\n    if y.size == 0:\n        return 0.0\n    \n    if precomputed_distances is None:\n        y = _convert_singleton_clusters_to_noise(y, noise_id=noise_id)\n\n        non_noise_inds = y != noise_id\n        X = X[non_noise_inds, :]\n        y = y[non_noise_inds]\n\n        if y.size == 0:\n            return 0.0\n\n        if check_duplicates:\n            _check_duplicated_samples(X)\n\n        dists = compute_pair_to_pair_dists(X=X, metric=metric)\n\n    else:\n        dists = precomputed_distances.copy()\n        np.maximum(dists, 1e-12, out=dists)\n        # NOTE: set self-distance to +inf to prevent points being self-neighbors.\n        np.fill_diagonal(dists, val=np.inf)\n\n    y = scipy.stats.rankdata(y, method=\"dense\") - 1\n    cluster_ids, cluster_sizes = np.unique(y, return_counts=True)\n\n    # DSC: 'Density Sparseness of a Cluster'\n    dscs = np.zeros(cluster_ids.size, dtype=float)\n\n    # DSPC: 'Density Separation of a Pair of Clusters'\n    min_dspcs = np.full(cluster_ids.size, fill_value=np.inf)\n\n    # Internal objects = Internal nodes = nodes such that degree(node) > 1 in MST.\n    internal_objects_per_cls: t.Dict[int, npt.NDArray[np.int32]] = {}\n\n    # internal core distances = core distances of internal nodes\n    internal_core_dists_per_cls: t.Dict[int, npt.NDArray[np.float32]] = {}\n\n    cls_inds = [np.flatnonzero(y == cls_id) for cls_id in cluster_ids]\n\n    if n_processes == \"auto\":\n        n_processes = 4 if y.size > 500 else 1\n\n    with _MP.workprec(bits_of_precision), multiprocessing.Pool(\n        processes=min(n_processes, cluster_ids.size)\n    ) as ppool:\n        fn_density_sparseness_ = functools.partial(\n            fn_density_sparseness,\n            d=d,\n            enable_dynamic_precision=enable_dynamic_precision,\n            use_original_mst_implementation=use_original_mst_implementation,\n        )\n\n        args = [(cls_ind, get_subarray(dists, inds_a=cls_ind)) for cls_ind in cls_inds]\n\n        for cls_id, (dsc, internal_core_dists, internal_node_inds) in enumerate(\n            ppool.starmap(fn_density_sparseness_, args)\n        ):\n            internal_objects_per_cls[cls_id] = internal_node_inds\n            internal_core_dists_per_cls[cls_id] = internal_core_dists\n            dscs[cls_id] = dsc\n\n    n_cls_pairs = (cluster_ids.size * (cluster_ids.size - 1)) // 2\n\n    if n_cls_pairs > 0:\n        with _MP.workprec(bits_of_precision), multiprocessing.Pool(\n            processes=min(n_processes, n_cls_pairs)\n        ) as ppool:\n            args = [\n                (\n                    cls_i,\n                    cls_j,\n                    get_subarray(\n                        dists,\n                        inds_a=internal_objects_per_cls[cls_i],\n                        inds_b=internal_objects_per_cls[cls_j],\n                    ),\n                    internal_core_dists_per_cls[cls_i],\n                    internal_core_dists_per_cls[cls_j],\n                )\n                for cls_i, cls_j in itertools.combinations(cluster_ids, 2)\n            ]\n\n            for cls_i, cls_j, dspc_ij in ppool.starmap(fn_density_separation, args):\n                min_dspcs[cls_i] = min(min_dspcs[cls_i], dspc_ij)\n                min_dspcs[cls_j] = min(min_dspcs[cls_j], dspc_ij)\n\n    np.nan_to_num(min_dspcs, copy=False, posinf=1e12)\n    vcs = (min_dspcs - dscs) / (1e-12 + np.maximum(min_dspcs, dscs))\n    np.nan_to_num(vcs, copy=False, nan=0.0)\n    dbcv = float(np.sum(vcs * cluster_sizes)) / n\n\n    return dbcv"
  },
  {
    "path": "opendsm/common/clustering/scoring.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport sys\n\nimport numpy as np\n\nfrom pydantic import BaseModel, ConfigDict\n\nimport sklearn.metrics as _metrics\n\nfrom opendsm.common.clustering.metrics.cluster_metrics import ClusterMetrics\nfrom opendsm.common.clustering import settings as _settings\n\n\ndef get_max_score_from_system_size() -> float:\n    \"\"\"\n    recreates the call to sys.float_info.max in order to\n    follow what was used in ads repo.\n\n    Making into function which executes each time\n    so unforseen issues are less likely when running on\n    distributed env\n    \"\"\"\n\n    return sys.float_info.max**0.5\n\n\ndef renumber_clusters(clusters: np.ndarray, reorder: bool):\n    \"\"\"Takes in cluster identifiers and renumbers them.\n        After merging or reclustering there are many cluster numbers left blank and need to be renumbered\n            Example: [0, 1, 2, 5, 7]\n        Additionally the clusters are reordered from largest cluster to smallest\n\n    Args:\n        clusters (list|np.array): an array in which a cluster number is defined for each load shape\n\n    Returns:\n        clusters (np.array): an array in which a cluster number is defined for each load shape\n    \"\"\"\n\n    if reorder:\n        # if outlier cluster exists, don't include it in the ordering\n        uniq_id, counts = np.unique(clusters[clusters != -1], return_counts=True)\n        count_order = np.argsort(counts)[::-1]\n\n        uniq_id = uniq_id[count_order]\n\n    else:\n        uniq_id = np.unique(clusters)\n\n    # if outlier cluster exists, don't change it\n    conv = {-1: -1}\n    conv.update({uniq_id[i]: i for i in range(len(uniq_id))})\n\n    clusters = np.array([conv[idx] for idx in clusters])\n\n    return clusters\n\n\ndef merge_small_clusters(clusters: np.ndarray, min_cluster_size: int):\n    \"\"\"\n    OG DOCSTRING:\n    Merges clusters which consist of less than the minumum number into the outlier cluster\n\n    Args:\n        clusters (list|np.array): A list defining what cluster each load shape belongs to\n        min_cluster_size (int): Minumum number of meters for a cluster\n            Options: 2 < val\n\n    Returns:\n        _type_: _description_\n    \"\"\"\n\n    uniq_ids, uniq_counts = np.unique(clusters, return_counts=True)\n\n    uniq_counts = uniq_counts[uniq_ids != -1]\n    uniq_ids = uniq_ids[uniq_ids != -1]\n\n    outlier_ids = uniq_ids[uniq_counts < min_cluster_size]\n    clusters[np.isin(clusters, outlier_ids)] = -1\n\n    return renumber_clusters(clusters, reorder=True)\n\n\nclass LabelResult(BaseModel):\n    \"\"\"\n    contains metrics about a cluster label\n    \"\"\"\n\n    model_config = ConfigDict(arbitrary_types_allowed=True)\n\n    labels: np.ndarray\n    score: dict[str, float]\n    score_unable_to_be_calculated: dict[str, bool]\n    n_clusters: int\n\n\n_score_council_init = {\n    'calinski_harabasz_index': 1.0,\n    'davies_bouldin_index': 1.0,\n    'density_based_clustering_validation_index': 1.0,\n    'dunn_index': 1.0,\n    'silhouette_index': 1.0,\n    'silhouette_median_index': 1.0,\n    'xie_beni_index': 1.0,\n}\n\ndef _score_clusters(\n    data: np.ndarray,\n    labels: np.ndarray,\n    n_cluster_lower: int,\n    score_council: dict[str, float] = _score_council_init, \n    dist_metric=\"euclidean\",\n    min_cluster_size=2,\n    max_non_outlier_cluster_count=200,\n) -> LabelResult:\n    \"\"\"\n    ---\n    Original docstring:\n\n    Score clusters of the given data with the selected choices.\n    Small clusters are first merged to only score clusters above the minimum size\n    and not in the outlier cluster.\n\n    Args:\n        data (np.array): Load shapes being clustered\n        labels (list|np.array): A list defining what cluster each load shape belongs to\n\n    Returns:\n        score (float): Lower is better\n        unable_to_calc_score (bool): Boolean that if true, means max score was used\n    \"\"\"\n\n    n_clusters = len(np.unique(labels))\n\n    # merge clusters to outlier cluster\n    labels = merge_small_clusters(labels, min_cluster_size)\n\n    non_outlier_cluster_count = labels.max() + 1\n    invalid = False\n    if non_outlier_cluster_count < n_cluster_lower:\n        invalid = True\n    elif non_outlier_cluster_count > max_non_outlier_cluster_count:\n        invalid = True\n        \n    if invalid:\n        return LabelResult(\n            labels=labels,\n            score={voter: np.inf for voter in score_council.keys()},\n            score_unable_to_be_calculated={voter: True for voter in score_council.keys()},\n            n_clusters=n_clusters,\n        )\n\n    # don't include outlier cluster in scoring\n    idx = np.argwhere(labels != -1).flatten()\n    data_non_outlier = data[idx, :]\n    labels_non_outlier = labels[idx]\n\n    metrics = ClusterMetrics(\n        data=data_non_outlier,\n        labels=labels_non_outlier,\n        distance_metric=dist_metric,\n    )\n    score = {}\n    score_unable_to_be_calculated = {}\n    for score_choice, score_weight in score_council.items():\n        score[score_choice] = np.inf\n        score_unable_to_be_calculated[score_choice] = True\n\n        if score_weight > 0:\n            try:\n                score[score_choice] = getattr(metrics, score_choice)\n\n                if np.isfinite(score[score_choice]):\n                    score_unable_to_be_calculated[score_choice] = False\n            except:\n                continue\n    \n    label_res = LabelResult(\n        labels=labels,\n        score=score,\n        score_unable_to_be_calculated=score_unable_to_be_calculated,\n        n_clusters=n_clusters,\n    )\n\n    return label_res\n\n\ndef score_council(settings: _settings.ClusteringSettings):\n    \"\"\"\n    Set the score council for the given settings.\n    \"\"\"\n\n    algo_settings = getattr(settings, settings.algorithm_selection)\n    algo_scoring = algo_settings.scoring\n\n    score_council = {\n        'calinski_harabasz_index': algo_scoring.calinski_harabasz_weight,\n        'davies_bouldin_index': algo_scoring.davies_bouldin_weight,\n        'density_based_clustering_validation_index': algo_scoring.density_based_clustering_validation_weight,\n        'dunn_index': algo_scoring.dunn_weight,\n        'silhouette_index': algo_scoring.silhouette_weight,\n        'silhouette_median_index': algo_scoring.silhouette_median_weight,\n        'xie_beni_index': algo_scoring.xie_beni_weight,\n    }\n\n    return score_council\n\n\ndef score_clusters(\n    data: np.ndarray,\n    labels: np.ndarray,\n    settings: _settings.ClusteringSettings\n):\n    \"\"\"\n    Score clusters of the given data with the selected choices.\n    \"\"\"\n    algo_settings = getattr(settings, settings.algorithm_selection)\n    algo_scoring = algo_settings.scoring\n\n    n_cluster_lower = algo_settings.n_cluster.lower\n    dist_metric = algo_scoring.distance_metric\n    min_cluster_size = algo_scoring.min_cluster_size\n    max_non_outlier_cluster_count = algo_scoring.max_non_outlier_cluster_count\n    \n    label_res = _score_clusters(\n        data,\n        labels,\n        n_cluster_lower,\n        score_council(settings),\n        dist_metric,\n        min_cluster_size,\n        max_non_outlier_cluster_count,\n    )\n\n    return label_res"
  },
  {
    "path": "opendsm/common/clustering/settings.py",
    "content": "from __future__ import annotations\n\nimport numpy as np\n\nimport pydantic\n\nfrom enum import Enum\nfrom typing import Optional, Literal, Union\n\nimport pywt\n\nfrom opendsm.common.base_settings import BaseSettings\n\n\nclass NormalizeChoice(str, Enum):\n    MIN_MAX_QUANTILE = \"min_max_quantile\"\n    STANDARDIZE = \"standardize\"\n    MED_MAD = \"med_mad\"\n\n\nclass NormalizeSettings(BaseSettings):\n    \"\"\"normalization method for data\"\"\"\n    method: Optional[NormalizeChoice] = pydantic.Field(\n        default=NormalizeChoice.STANDARDIZE,\n    )\n\n    pre_transform: bool = pydantic.Field(\n        default=True,\n    )\n\n    post_transform: bool = pydantic.Field(\n        default=True,\n    )\n\n    quantile: Optional[float] = pydantic.Field(\n        default=None,\n        gt=0.0,\n        lt=0.5,\n    )\n\n    axis: Optional[int] = pydantic.Field(\n        default=None,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_quantile(self):\n        if self.method == NormalizeChoice.MIN_MAX_QUANTILE:\n            if self.quantile is None:\n                raise ValueError(\n                    \"'quantile' must be specified when 'method' is 'min_max_quantile'\"\n                )\n        else:\n            if self.quantile is not None:\n                raise ValueError(\n                    \"'quantile' should only be specified when 'method' is 'min_max_quantile'\"\n                )\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_enable(self):\n        if self.method is None:\n            if self.pre_transform or self.post_transform:\n                raise ValueError(\n                    \"'method' cannot be None if 'pre_transform' or 'post_transform' is True\"\n                )\n        else:\n            if not (self.pre_transform or self.post_transform):\n                raise ValueError(\n                    \"'pre_transform' or 'post_transform' must be True if 'method' is specified\"\n                )\n\n        return self\n\n\nclass TransformChoice(str, Enum):\n    FPCA = \"fpca\"\n    WAVELET = \"wavelet\"\n    \n\nclass fPCATransformSettings(BaseSettings):\n    \"\"\"explained variance ratio for fPCA clustering\"\"\"\n    min_var_ratio: float = pydantic.Field(\n        default=0.97,\n        ge=0.5,\n        le=1.0,\n    )\n\n\nclass PCASelection(str, Enum):\n    PCA = \"pca\"\n    KERNEL_PCA = \"kernel_pca\"\n\n\nclass WaveletSelection(str, Enum):\n    BIOR1_1 = \"bior1.1\"\n    COIF6 = \"coif6\"\n    COIF17 = \"coif17\"    # Best error/speed mix\n    DB1 = \"db1\"          # Best error metrics\n    DB16 = \"db16\"\n    DB26 = \"db26\"\n    DB29 = \"db29\"\n    HAAR = \"haar\"\n    RBIO1_1 = \"rbio1.1\"\n    SYM11 = \"sym11\"\n\n\nclass WaveletTransformSettings(BaseSettings):\n    \"\"\"wavelet decomposition level\"\"\"\n    wavelet_n_levels: Optional[int] = pydantic.Field(\n        default=None,\n        ge=1,\n    )\n\n    \"\"\"wavelet choice for wavelet decomposition\"\"\"\n    wavelet_name: WaveletSelection = pydantic.Field(\n        default=WaveletSelection.DB1,\n    )\n\n    \"\"\"signal extension mode for wavelet decomposition\"\"\"\n    wavelet_mode: str = pydantic.Field(\n        default=\"smooth\",\n    )\n\n    \"\"\"PCA method\"\"\"\n    pca_method: PCASelection = pydantic.Field(\n        default=PCASelection.PCA,\n    )\n\n    \"\"\"minimum variance ratio for PCA clustering\"\"\"\n    pca_min_variance_ratio_explained: Optional[float] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"number of components to keep for PCA clustering\"\"\"\n    pca_n_components: Optional[Union[int, Literal[\"mle\"]]] = pydantic.Field(\n        default=\"mle\",\n    )\n\n    \"\"\"add scale to features\"\"\"\n    include_scale_feature: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"seed for random state assignment\"\"\"\n    seed: Optional[int] = pydantic.Field(\n        default=None,\n        ge=0,\n    )\n\n    _seed: Optional[int] = pydantic.PrivateAttr(\n        default=None\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_seed(self):\n        if self.seed is None and self._seed is None:\n            self._seed = np.random.randint(0, 2**32 - 1, dtype=np.int64)\n        else:\n            self._seed = self.seed\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_wavelet(self):\n        all_wavelets = pywt.wavelist(kind=\"discrete\")\n        if self.wavelet_name not in all_wavelets:\n            raise ValueError(\n                f\"'wavelet_name' must be a valid wavelet in PyWavelets: \\n{all_wavelets}\"\n            )\n\n        all_modes = pywt.Modes.modes\n        if self.wavelet_mode not in all_modes:\n            raise ValueError(\n                f\"'wavelet_mode' must be a valid mode in PyWavelets: \\n{all_modes}\"\n            )\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_pca_settings(self):\n        if self.pca_n_components is None and self.pca_min_variance_ratio_explained is None:\n            raise ValueError(\n                \"Must specify either 'pca_min_variance_ratio_explained' or 'pca_n_components'\"\n            )\n\n        if self.pca_n_components is not None:\n            if self.pca_min_variance_ratio_explained is not None:\n                raise ValueError(\n                    \"Cannot specify both 'pca_min_variance_ratio_explained' and 'pca_n_components'\"\n                )\n            \n            if isinstance(self.pca_n_components, int):\n                if self.pca_n_components < 1:\n                    raise ValueError(\n                        \"'pca_n_components' must be >= 1\"\n                    )\n\n            if (self.pca_n_components == \"mle\") and (self.pca_method == PCASelection.KERNEL_PCA):\n                raise ValueError(\n                    \"Cannot use 'mle' with 'kernel_pca'\"\n                )\n\n        if self.pca_min_variance_ratio_explained is not None:\n            if not 0.5 <= self.pca_min_variance_ratio_explained <= 1:\n                raise ValueError(\n                    \"'pca_min_variance_ratio_explained' must be between 0.5 and 1\"\n                )\n            \n        return self\n\nclass DistanceMetric(str, Enum):\n    \"\"\"\n    what distance method to use\n    \"\"\"\n    EUCLIDEAN = \"euclidean\"\n    SEUCLIDEAN = \"seuclidean\"\n    MANHATTAN = \"manhattan\"\n    COSINE = \"cosine\"\n\nclass ScoreSettings(BaseSettings):\n    \"\"\"minimum cluster size\"\"\"\n    min_cluster_size: int = pydantic.Field(\n        default=2,\n        ge=2, # \n    )\n\n    \"\"\"maximum number of non-outlier clusters\"\"\"\n    max_non_outlier_cluster_count: int = pydantic.Field(\n        default=200,\n        ge=1,\n    )\n\n    \"\"\"scoring methods\"\"\"\n    calinski_harabasz_weight: float = pydantic.Field(\n        default=1.0,\n        ge=0,\n    )\n\n    davies_bouldin_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    density_based_clustering_validation_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    dunn_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    silhouette_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    silhouette_median_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    xie_beni_weight: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    window_size: float = pydantic.Field(\n        default=0,\n        ge=0,\n    )\n\n    \"\"\"distance metric for clustering\"\"\"\n    distance_metric: DistanceMetric = pydantic.Field(\n        default=DistanceMetric.EUCLIDEAN,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_weights(self):\n        weights = [\n            self.calinski_harabasz_weight,\n            self.davies_bouldin_weight,\n            self.density_based_clustering_validation_weight,\n            self.dunn_weight,\n            self.silhouette_weight,\n            self.silhouette_median_weight,\n            self.xie_beni_weight,\n        ]\n\n        if not any(w > 0 for w in weights):\n            raise ValueError(\"At least one scoring weight must be greater than 0\")\n        \n        return self\n\n\nclass ClusterRangeSettings(BaseSettings):\n    \"\"\"lower bound for number of clusters\"\"\"\n    lower: int = pydantic.Field(\n        default=2,\n        ge=2,\n    )\n\n    \"\"\"upper bound for number of clusters\"\"\"\n    upper: int = pydantic.Field(\n        default=24,\n        ge=2,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_n_cluster_range(self):\n        if self.lower > self.upper:\n            raise ValueError(\n                \"'n_cluster_lower' must be <= 'n_cluster_upper'\"\n            )\n\n        return self\n\n\nclass BiKmeansInnerAlgorithms(str, Enum):\n    ELKAN = \"elkan\"\n    LLOYD = \"lloyd\"\n\n\nclass BiKmeansBisectingStrategies(str, Enum):\n    BIGGEST_INERTIA = \"biggest_inertia\"\n    LARGEST_CLUSTER = \"largest_cluster\"\n\n\nclass BisectingKMeansSettings(BaseSettings):\n    \"\"\"number of times to recluster\"\"\"\n    recluster_count: int = pydantic.Field(\n        default=3,\n        ge=1,\n    )\n\n    \"\"\"number of times to recluster internally\"\"\"\n    internal_recluster_count: int = pydantic.Field(\n        default=5,\n        ge=1,\n    )\n\n    \"\"\"Inner KMeans algorithm used in bisection\"\"\"\n    inner_algorithm: BiKmeansInnerAlgorithms = pydantic.Field(\n        default=BiKmeansInnerAlgorithms.ELKAN,\n    )\n\n    \"\"\"Bisection strategy\"\"\"\n    bisecting_strategy: BiKmeansBisectingStrategies = pydantic.Field(\n        default=BiKmeansBisectingStrategies.LARGEST_CLUSTER,\n    )\n\n    n_cluster: ClusterRangeSettings = pydantic.Field(\n        default_factory=ClusterRangeSettings\n    )\n\n    scoring: ScoreSettings = pydantic.Field(\n        default_factory=ScoreSettings\n    )\n\n    \nclass BirchSettings(BaseSettings):\n    \"\"\"radius of the subcluster to merge a new sample in\"\"\"\n    threshold: float = pydantic.Field(\n        default=0.5,\n        ge=0,\n    )\n\n    \"\"\"maximum number of CF subclusters in each node\"\"\"\n    branching_factor: int = pydantic.Field(\n        default=50,\n        ge=1,\n    )\n\n    n_cluster: ClusterRangeSettings = pydantic.Field(\n        default_factory=ClusterRangeSettings\n    )\n\n    scoring: ScoreSettings = pydantic.Field(\n        default_factory=ScoreSettings\n    )\n\n\nclass DbscanDistanceAlgorithm(str, Enum):\n    AUTO = \"auto\"\n    BRUTE = \"brute\"\n    KD_TREE = \"kd_tree\"\n    BALL_TREE = \"ball_tree\"\n\nclass DBSCANSettings(BaseSettings):\n    \"\"\"maximum distance between two samples for one to be considered as in the neighborhood of the other\"\"\"\n    epsilon: float = pydantic.Field(\n        default=0.5,\n        gt=0,\n    )\n\n    \"\"\"minimum number of samples in a neighborhood for a point to be considered as a cluster\"\"\"\n    min_samples: int = pydantic.Field(\n        default=1, # sklearn default is 5\n        ge=1,\n    )\n\n    \"\"\"distance metric for calculating distance between samples\"\"\"\n    distance_metric: DistanceMetric = pydantic.Field(\n        default=DistanceMetric.EUCLIDEAN,\n    )\n\n    \"\"\"distance algorithm to use for nearest neighbors\"\"\"\n    nearest_neighbors_algorithm: DbscanDistanceAlgorithm = pydantic.Field(\n        default=DbscanDistanceAlgorithm.AUTO,\n    )\n\n    \"\"\"leaf size for KDTree or BallTree\"\"\"\n    leaf_size: Optional[int] = pydantic.Field(\n        default=30,\n    )\n\n    \"\"\"Minkowski p-norm distance power\"\"\"\n    minkowski_p: float = pydantic.Field(\n        default=2,\n        ge=1,\n    )\n\n\nclass HdbscanClusterSelectionMethod(str, Enum):\n    LEAF = \"leaf\"\n    EXCESS_OF_MASS = \"eom\"\n\n\nclass HDBSCANSettings(BaseSettings):\n    \"\"\"allow single cluster\"\"\"\n    allow_single_cluster: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"maximum cluster count\"\"\"\n    max_cluster_size: Optional[int] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"minimum number of samples in a group for it to be considered as a cluster\"\"\"\n    min_samples: int = pydantic.Field(\n        default=1,\n        ge=1,\n    )\n\n    \"\"\"distance metric for calculating distance between samples\"\"\"\n    distance_metric: DistanceMetric = pydantic.Field(\n        default=DistanceMetric.EUCLIDEAN,\n    )\n\n    \"\"\"samples to calculate distance between neighbors\"\"\"\n    scoring_sample_count: Optional[int] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"clusters below this distance threshold will be merged\"\"\"\n    cluster_selection_epsilon: float = pydantic.Field(\n        default=0.0,\n        ge=0,\n    )\n\n    \"\"\"distance scaling factor for robust single linkage\"\"\"\n    robust_single_linkage_scaling: float = pydantic.Field(\n        default=1.0,\n        gt=0,\n    )\n\n    \"\"\"distance algorithm to use\"\"\"\n    nearest_neighbors_algorithm: DbscanDistanceAlgorithm = pydantic.Field(\n        default=DbscanDistanceAlgorithm.AUTO,\n    )\n\n    \"\"\"leaf size for KDTree or BallTree\"\"\"\n    leaf_size: Optional[int] = pydantic.Field(\n        default=40,\n    )\n\n    \"\"\"cluster selection method\"\"\"\n    cluster_selection_method: HdbscanClusterSelectionMethod = pydantic.Field(\n        default=HdbscanClusterSelectionMethod.EXCESS_OF_MASS,\n    )\n\n\nclass SpectralEigenSolver(str, Enum):\n    ARPACK = \"arpack\"\n    LOBPCG = \"lobpcg\"\n    # AMG = \"amg\" # disabled due to additional installation requirements\n\nclass AffinityMatrixOptions(str, Enum):\n    # Some of these are currently disabled. Can be added later after debugging\n    NEAREST_NEIGHBORS = \"nearest_neighbors\"\n    RBF = \"rbf\"\n    # ADDITIVE_CHI2 = \"additive_chi2\"\n    CHI2 = \"chi2\"\n    # LINEAR = \"linear\"\n    # POLY = \"poly\"\n    # POLYNOMIAL = \"polynomial\"\n    LAPLACIAN = \"laplacian\"\n    # SIGMOID = \"sigmoid\"\n    # COSINE = \"cosine\"\n\nclass SpectralAssignLabels(str, Enum):\n    KMEANS = \"kmeans\"\n    DISCRETIZE = \"discretize\"\n    CLUSTER_QR = \"cluster_qr\"\n    \nclass SpectralSettings(BaseSettings):\n    \"\"\"number of times to recluster\"\"\"\n    recluster_count: int = pydantic.Field(\n        default=0,\n        ge=0,\n    )\n\n    \"\"\"eigen solver to use\"\"\"\n    eigen_solver: Optional[SpectralEigenSolver] = pydantic.Field(\n        default=SpectralEigenSolver.ARPACK,\n    )\n\n    \"\"\"number of eigenvectors to use, defaults to n_clusters\"\"\"\n    n_components: Optional[int] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"affinity matrix algorithm to use\"\"\"\n    affinity: AffinityMatrixOptions = pydantic.Field(\n        default=AffinityMatrixOptions.RBF,\n    )\n\n    \"\"\"number of nearest neighbors to use for nearest neighbors kernel\"\"\"\n    nearest_neighbors: int = pydantic.Field(\n        default=5,\n        ge=1,\n    )\n\n    \"\"\"gamma for RBF, polynomial, sigmoid, laplacian, and chi2 kernels\"\"\"\n    gamma: float = pydantic.Field(\n        default=1.05,\n        gt=0,\n    )\n\n    \"\"\"stopping criterion for eigen decomposition\"\"\"\n    eigen_tol: Union[float, Literal[\"auto\"]] = pydantic.Field(\n        default=\"auto\",\n    )\n\n    \"\"\"label assignment method\"\"\"\n    assign_labels: SpectralAssignLabels = pydantic.Field(\n        default=SpectralAssignLabels.CLUSTER_QR,\n    )\n\n    n_cluster: ClusterRangeSettings = pydantic.Field(\n        default_factory=ClusterRangeSettings\n    )\n\n    scoring: ScoreSettings = pydantic.Field(\n        default_factory=ScoreSettings\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_eigen_tol(self):\n        if self.eigen_tol != \"auto\":\n            if self.eigen_tol < 0:\n                raise ValueError(\n                    \"'eigen_tol' must be >= 0\"\n                )\n\n        return self\n\n\nclass SortMethod(str, Enum):\n    SIZE = \"size\"\n    PEAK = \"peak\"\n    # VARIANCE = \"variance\"\n\n\nclass AggregateMethod(str, Enum):\n    MEAN = \"mean\"\n    MEDIAN = \"median\"\n\n\nclass ClusterSortSettings(BaseSettings):\n    \"\"\"enable cluster sorting\"\"\"\n    enable: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"sort method\"\"\"\n    method: SortMethod = pydantic.Field(\n        default=SortMethod.PEAK,\n    )\n\n    \"\"\"aggregate method\"\"\"\n    aggregation: AggregateMethod = pydantic.Field(\n        default=AggregateMethod.MEAN\n    )\n\n    \"\"\"sort order\"\"\"\n    reverse: bool = pydantic.Field(\n        default=False,\n    )\n\n\nclass ClusterAlgorithms(str, Enum):\n    BISECTING_KMEANS = \"bisecting_kmeans\"\n    BIRCH = \"birch\"\n    DBSCAN = \"dbscan\"\n    HDBSCAN = \"hdbscan\"\n    SPECTRAL = \"spectral\"\n\n\nclass ClusteringSettings(BaseSettings):\n    \"\"\"pretransform data rescale settings\"\"\"\n    normalize: NormalizeSettings = pydantic.Field(\n        default_factory=NormalizeSettings\n    )\n\n    \"\"\"transform method\"\"\"\n    transform_selection: Optional[TransformChoice] = pydantic.Field(\n        default=TransformChoice.WAVELET,\n    )\n\n    \"\"\"fPCA transform settings\"\"\"\n    fpca_transform: Optional[fPCATransformSettings] = pydantic.Field(\n        default_factory=fPCATransformSettings\n    )\n\n    \"\"\"wavelet transform settings\"\"\"\n    wavelet_transform: Optional[WaveletTransformSettings] = pydantic.Field(\n        default_factory=WaveletTransformSettings\n    )\n\n    \"\"\"clustering choice\"\"\"\n    algorithm_selection: ClusterAlgorithms = pydantic.Field(\n        default=ClusterAlgorithms.SPECTRAL,\n    )\n\n    \"\"\"BisectingKMeans settings\"\"\"\n    bisecting_kmeans: Optional[BisectingKMeansSettings] = pydantic.Field(\n        default_factory=BisectingKMeansSettings,\n    )\n\n    \"\"\"Birch settings\"\"\"\n    birch: Optional[BirchSettings] = pydantic.Field(\n        default_factory=BirchSettings,\n    )\n\n    \"\"\"DBSCAN settings\"\"\"\n    dbscan: Optional[DBSCANSettings] = pydantic.Field(\n        default_factory=DBSCANSettings,\n    )\n\n    \"\"\"HDBSCAN settings\"\"\"\n    hdbscan: Optional[HDBSCANSettings] = pydantic.Field(\n        default_factory=HDBSCANSettings,\n    )\n\n    \"\"\"Spectral settings\"\"\"\n    spectral: Optional[SpectralSettings] = pydantic.Field(\n        default_factory=SpectralSettings,\n    )\n\n    \"\"\"sort clusters \"\"\"\n    cluster_sort: ClusterSortSettings = pydantic.Field(\n        default_factory=ClusterSortSettings,\n    )\n\n    \"\"\"seed for random state assignment\"\"\"\n    seed: Optional[int] = pydantic.Field(\n        default=None,\n        ge=0,\n    )\n\n    _seed: Optional[int] = pydantic.PrivateAttr(\n        default=None\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_seed(self):\n        if self.seed is None and self._seed is None:\n            self._seed = np.random.randint(0, 2**32 - 1, dtype=np.int64)\n        else:\n            self._seed = self.seed\n\n        for transform in [self.wavelet_transform, self.fpca_transform]:\n            if transform is not None:\n                transform._seed = self._seed\n\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _remove_unselected_algorithms(self):\n        self.model_config[\"frozen\"] = False\n\n        algo_dict = {\n            ClusterAlgorithms.BISECTING_KMEANS: self.bisecting_kmeans,\n            ClusterAlgorithms.BIRCH: self.birch,\n            ClusterAlgorithms.DBSCAN: self.dbscan,\n            ClusterAlgorithms.HDBSCAN: self.hdbscan,\n            ClusterAlgorithms.SPECTRAL: self.spectral,\n        }\n\n        for k in algo_dict.keys():\n            if k != self.algorithm_selection:\n                setattr(self, k, None)\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _remove_unselected_transform(self):\n        self.model_config[\"frozen\"] = False\n\n        transform_dict = {\n            TransformChoice.WAVELET: self.wavelet_transform,\n            TransformChoice.FPCA: self.fpca_transform,\n        }\n\n        for k in transform_dict.keys():\n            if k != self.transform_selection:\n                setattr(self, f\"{k.value}_transform\", None)\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n\n\nif __name__ == \"__main__\":\n    settings = ClusteringSettings()\n\n    print(settings)\n\n    print(settings._algorithm)"
  },
  {
    "path": "opendsm/common/clustering/transform.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport warnings\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\n\nfrom skfda.representation.grid import FDataGrid as _FDataGrid\nfrom skfda.representation.basis import Fourier as _Fourier\nfrom skfda.preprocessing.dim_reduction import FPCA as _FPCA\n\nimport pywt\n\nfrom sklearn.decomposition import PCA, KernelPCA\n\nfrom opendsm.common.stats import basic as _basic\nfrom opendsm.common.clustering import settings as _settings\n\n\n\ndef _safe_standardize(\n    data: np.ndarray,\n    center: np.ndarray,\n    scale: np.ndarray,\n    threshold: float = 1e-10\n) -> np.ndarray:\n    \"\"\"Safely standardize data by centering and scaling.\n\n    If the scale (e.g., standard deviation or MAD) is near zero, only centers\n    the data without scaling to avoid division by near-zero values.\n\n    Parameters\n    ----------\n    data : np.ndarray\n        Input data to standardize.\n    center : np.ndarray\n        Centering values (e.g., mean or median) to subtract from data.\n    scale : np.ndarray\n        Scaling values (e.g., std or MAD) to divide by. Can be scalar or array.\n    threshold : float, optional\n        Minimum threshold for scale values. If scale is below this, only\n        centering is performed. Default is 1e-10.\n\n    Returns\n    -------\n    np.ndarray\n        Standardized data. If scale is near zero for any element, those\n        elements are only centered without scaling.\n    \"\"\"\n    centered = data - center\n\n    # Handle scalar scale\n    if np.isscalar(scale) or scale.ndim == 0:\n        if scale > threshold:\n            return centered / scale\n        else:\n            return centered\n\n    # Handle array scale with broadcasting\n    # Replace near-zero scales with 1 for safe division, but track which were replaced\n    scale_safe = np.where(scale > threshold, scale, 1.0)\n    result = centered / scale_safe\n\n    # For positions where scale was near zero, use only centered value\n    near_zero_mask = scale <= threshold\n    if np.any(near_zero_mask):\n        # Use broadcasting to apply mask\n        if centered.ndim == 2 and scale.ndim == 1:\n            # Expand mask to match data dimensions\n            result = np.where(near_zero_mask, centered, result)\n        else:\n            result[near_zero_mask] = centered[near_zero_mask]\n\n    return result\n\n\ndef normalize(\n    data: np.ndarray,\n    settings: _settings.NormalizeSettings\n) -> np.ndarray:\n    method = settings.method\n    axis = settings.axis\n\n    if method == _settings.NormalizeChoice.STANDARDIZE:\n        mean = np.mean(data, axis=axis)\n        std = np.std(data, axis=axis)\n        data = _safe_standardize(data, mean, std)\n\n    elif method == _settings.NormalizeChoice.MED_MAD:\n        median = np.median(data, axis=axis)\n        mad = _basic.median_absolute_deviation(data, median=median, axis=axis)\n        data = _safe_standardize(data, median, mad)\n\n    elif method == _settings.NormalizeChoice.MIN_MAX_QUANTILE:\n        q = settings.quantile\n        a, b = [-1, 1]  # range to normalize to\n\n        min_val, max_val = np.quantile(data, [q, 1 - q], axis=axis)\n\n        # Handle different axis cases\n        if axis is None:\n            # Global normalization\n            if min_val == max_val:\n                data = np.full_like(data, (a + b) / 2)\n            else:\n                data = (b - a) * (data - min_val) / (max_val - min_val) + a\n        else:\n            # Axis-specific normalization\n            idx_same = np.argwhere(min_val == max_val).flatten()\n\n            # Determine which axis we're normalizing over\n            # If axis=0, we normalize columns (iterate over axis 1)\n            # If axis=1, we normalize rows (iterate over axis 0)\n            other_axis = 1 - axis if axis in [0, 1] else None\n\n            if other_axis is not None:\n                n_elements = data.shape[other_axis]\n                idx_diff = np.array([idx for idx in range(n_elements) if idx not in idx_same])\n\n                if len(idx_diff) > 0:\n                    # Reshape min_val and max_val for proper broadcasting\n                    shape = [1, 1]\n                    shape[other_axis] = len(idx_diff)\n                    min_val_reshaped = min_val[idx_diff].reshape(shape)\n                    max_val_reshaped = max_val[idx_diff].reshape(shape)\n\n                    # Create slice objects for indexing\n                    slices = [slice(None), slice(None)]\n                    slices[other_axis] = idx_diff\n                    slices = tuple(slices)\n\n                    # Normalize\n                    data[slices] = (b - a) * (data[slices] - min_val_reshaped) / (max_val_reshaped - min_val_reshaped) + a\n\n                if len(idx_same) > 0:\n                    slices = [slice(None), slice(None)]\n                    slices[other_axis] = idx_same\n                    slices = tuple(slices)\n                    data[slices] = (a + b) / 2\n\n    return data\n\n\nclass FpcaError(Exception):\n    pass\n\ndef _fpca_base(\n    x: np.ndarray, \n    y: np.ndarray, \n    min_var_ratio: float\n) -> np.ndarray:\n    \"\"\"\n    applies fpca to concatenated transform loadshape dataframe values\n\n    x -> time converted to np array taken from loadshape dataframe\n    y -> transformed values\n\n    assumes mixture_components return and fourier basis\n\n    also may return a string as second return value. if it is not None, it implies an error occurred\n    \"\"\"\n\n    if 0 >= min_var_ratio or min_var_ratio >= 1:\n        raise FpcaError(\"min_var_ratio but be greater than 0 and less than 1\")\n\n    if not np.all(np.isfinite(x)) or not np.all(np.isfinite(y)):\n        raise FpcaError(\"provided non finite values for fpca\")\n\n    if len(x) == 0 or len(y) == 0:\n        raise FpcaError(\"provided empty values for fpca\")\n\n    n_min = 1\n    # get maximum n components\n\n    # smallest 1  || min(largest = number of samples - 1, # time points)\n    n_max = np.min(np.array(np.shape(y)) - [1, 5])\n    if n_max < n_min:\n        n_max = n_min\n\n    n_max = int(n_max)\n\n    # get maximum principle components\n    fd = _FDataGrid(grid_points=x, data_matrix=y)\n    basis_fcn = _Fourier\n\n    basis_fd = fd.to_basis(basis_fcn(n_basis=n_max + 4))\n    fpca = _FPCA(n_components=n_max, components_basis=basis_fcn(n_basis=n_max + 4))\n    fpca.fit(basis_fd)\n\n    var_ratio = np.cumsum(fpca.explained_variance_ratio_) - min_var_ratio\n    n = int(np.argmin(var_ratio < 0.0) + 1)\n\n    basis_fd = fd.to_basis(basis_fcn(n_basis=n + 4))\n    fpca = _FPCA(n_components=n, components_basis=basis_fcn(n_basis=n + 4))\n    fpca.fit(basis_fd)\n\n    mixture_components = fpca.transform(basis_fd)\n\n    return mixture_components\n\n\ndef fpca_transform(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n) -> np.ndarray:\n    min_var_ratio = settings.fpca_transform.min_var_ratio\n\n    x = np.arange(data.shape[1]) # assumes uniform spacing\n    try:\n        fcpa_mixture_components = _fpca_base(\n            x=x, \n            y=data, \n            min_var_ratio=min_var_ratio\n        )\n    except FpcaError as e:\n        raise e\n\n    return fcpa_mixture_components\n\n\ndef wavelet_transform(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n) -> np.ndarray:\n    \"\"\"\n    Transforms the data using the wavelet transform settings\n    \"\"\"\n    wavelet_settings = settings.wavelet_transform\n\n    def _dwt_coeffs(data, wavelet=\"db1\", wavelet_mode=\"periodization\", n_levels=None):\n        all_features = []\n        # iterate through rows of numpy array\n        for row in range(len(data)):\n            # get max level of decomposition\n            dwt_max_level = pywt.dwt_max_level(data[row].shape[0], wavelet)\n\n            if n_levels is None: # None could be input into wavedec directly to same effect\n                n_levels = dwt_max_level\n\n            elif n_levels > dwt_max_level:\n                n_levels = dwt_max_level\n            \n            decomp_coeffs = pywt.wavedec(\n                data[row], wavelet=wavelet, mode=wavelet_mode, level=n_levels\n            )\n\n            decomp_coeffs = np.hstack(decomp_coeffs)\n\n            all_features.append(decomp_coeffs)\n\n        return np.vstack(all_features)\n\n    def _pca_coeffs(features, method, min_var_ratio_explained=0.95, n_components=None):\n        if min_var_ratio_explained is not None:\n            n_components = min_var_ratio_explained\n\n        # kernel pca is not fully developed\n        if method == \"kernel_pca\":\n            if n_components ==  \"mle\":\n                pca = PCA(\n                    n_components=n_components,\n                    random_state=settings._seed,\n                )\n                pca_features = pca.fit_transform(features)\n\n            pca = KernelPCA(n_components=None, kernel=\"rbf\")\n            pca_features = pca.fit_transform(features)\n\n            if min_var_ratio_explained is not None:\n                explained_variance_ratio = pca.eigenvalues_ / np.sum(pca.eigenvalues_)\n\n                # get the cumulative explained variance ratio\n                cumulative_explained_variance = np.cumsum(explained_variance_ratio)\n\n                # find number of components that explain pct% of the variance\n                n_components = np.argmax(cumulative_explained_variance > n_components).astype(int)\n\n            if not isinstance(n_components, (int, np.integer)):\n                raise ValueError(\"n_components must be an integer for kernel PCA\")\n\n            # pca = PCA(n_components=n_components)\n            pca = KernelPCA(n_components=n_components, kernel=\"rbf\")\n            pca_features = pca.fit_transform(features)\n\n        else:\n            pca = PCA(\n                n_components=n_components,\n                random_state=settings._seed,\n            )\n            pca_features = pca.fit_transform(features)\n\n        return pca_features\n\n    # calculate wavelet coefficients\n    with warnings.catch_warnings():\n        features = _dwt_coeffs(\n            data, \n            wavelet_settings.wavelet_name, \n            wavelet_settings.wavelet_mode, \n            wavelet_settings.wavelet_n_levels\n        )\n\n    pca_features = _pca_coeffs(\n        features,\n        wavelet_settings.pca_method,\n        wavelet_settings.pca_min_variance_ratio_explained,\n        wavelet_settings.pca_n_components,\n    )\n\n    # normalize pca features\n    if settings.normalize.post_transform:\n        # ignores all other values from normalize settings\n        mean = pca_features.mean()\n        std = pca_features.std()\n        pca_features = _safe_standardize(pca_features, mean, std)\n\n    if wavelet_settings.include_scale_feature:\n        pca_features = np.hstack([pca_features, np.median(data, axis=1)[:, None]])\n\n    return pca_features\n\n\ndef transform_features(\n    data: np.ndarray,\n    settings: _settings.ClusteringSettings\n) -> np.ndarray:\n    \n    # normalize the data\n    if settings.normalize.pre_transform:\n        data = normalize(data, settings.normalize)\n\n    # transform the data\n    if settings.transform_selection == _settings.TransformChoice.FPCA:\n        data = fpca_transform(data, settings)\n\n    elif settings.transform_selection == _settings.TransformChoice.WAVELET:\n        data = wavelet_transform(data, settings)\n\n    return data"
  },
  {
    "path": "opendsm/common/clustering/voting.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2025 OpenDSM contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nfrom scipy.ndimage import gaussian_filter1d\n\n\ndef construct_voting_df(results):\n    \"\"\"\n    Construct a voting DataFrame from the results and score council.\n    \"\"\"\n    # Create a dataframe of all score algorithms and their scores for each number of clusters\n    score_dict = {}\n    for n, label_res in enumerate(results):\n        # n_clusters = label_res.n_clusters\n        score_dict[n] = label_res.score\n\n    res_df = pd.DataFrame.from_dict(score_dict, orient='index')\n\n    # replace non-finite values with inf\n    res_df = res_df.replace([np.nan, -np.inf], np.inf)\n\n    # Drop columns where all values are np.inf\n    res_df = res_df.loc[:, ~(res_df == np.inf).all()]\n\n    # convert from index value to order of cluster number\n    res_df = res_df.apply(lambda col: col.sort_values().index.to_numpy(), axis=0)\n\n    # reset index and delete old index\n    res_df = res_df.reset_index(drop=True)\n\n    # If res_df is a Series, convert it to a DataFrame\n    if isinstance(res_df, pd.Series):\n        res_df = res_df.to_frame()\n    \n    return res_df\n\n\ndef _shulze_pairwise_preference(df, voter_weights=None):\n    \"\"\"\n    Perform pairwise comparison to select the best candidate (row) from a DataFrame.\n    Each column is a 'voter' (score algorithm), and each row index is a candidate (n_clusters).\n    Each column contains a ranking of candidates (row indices), with lower values being better.\n\n    \"\"\"\n\n    if voter_weights is None:\n        voter_weights = {voter: 1.0 for voter in df.columns}\n\n    candidates = np.unique(df.iloc[:, 0])\n\n    # Pre-build rank lookup per voter to avoid repeated pd.Index construction\n    voter_ranks = {}\n    for voter in df.columns:\n        idx = pd.Index(df[voter])\n        voter_ranks[voter] = {candidate: idx.get_loc(candidate) for candidate in candidates}\n\n    Pd = np.zeros((len(candidates), len(candidates), 2))\n    pred = np.zeros((len(candidates), len(candidates)))\n    for i, a in enumerate(candidates):\n        for j, b in enumerate(candidates):\n            if a == b:\n                continue\n\n            votes_a = 0.0\n            votes_b = 0.0\n            for voter in df.columns:\n                w = voter_weights[voter]\n                rank_a = voter_ranks[voter][a]\n                rank_b = voter_ranks[voter][b]\n\n                if rank_a < rank_b:\n                    votes_a += w\n                elif rank_a > rank_b:\n                    votes_b += w\n                else:\n                    votes_a += 0.5 * w\n                    votes_b += 0.5 * w\n\n            Pd[i, j] = [votes_a, votes_b]\n            pred[i, j] = i\n\n    return Pd, pred\n\n\ndef _shulze_path_strength(Pd, pred):\n    \"\"\"\n    Compute strongest path strengths using Floyd-Warshall.\n    Updates Pd so that Pd[j, k][0] holds the strength of the\n    strongest path from candidate j to candidate k.\n    \"\"\"\n    n_candidates = Pd.shape[0]\n\n    for i in range(n_candidates):\n        for j in range(n_candidates):\n            if i == j:\n                continue\n\n            for k in range(n_candidates):\n                if k == i or k == j:\n                    continue\n\n                # Strength of path j→i→k is the bottleneck (min) of two edges\n                strength_ji = Pd[j, i][0]\n                strength_ik = Pd[i, k][0]\n\n                if strength_ji <= strength_ik:\n                    bottleneck = (j, i)\n                    potential_strength = strength_ji\n                else:\n                    bottleneck = (i, k)\n                    potential_strength = strength_ik\n\n                if Pd[j, k][0] < potential_strength:\n                    Pd[j, k] = Pd[bottleneck[0], bottleneck[1], :]\n                    pred[j, k] = pred[i, k]\n\n    return Pd, pred\n\n\ndef _shulze_rank_strength(Pd, pred):\n    \"\"\"\n    Compute the rank strength for each candidate.\n    \"\"\"\n    n_candidates = Pd.shape[0]\n    candidate_wins = np.zeros(n_candidates)\n\n    for i in range(n_candidates):\n        for j in range(n_candidates):\n            if i == j:\n                continue\n\n            if Pd[i, j][0] > Pd[j, i][0]:\n                candidate_wins[i] += 1\n    \n    return candidate_wins\n\n\ndef shulze_voting(df, voter_weights=None, window_size=0, return_preference_df=False):\n    \"\"\"\n    Perform Shulze voting to select the best candidate (row) from a DataFrame.\n    Each column is a 'voter' (score algorithm), and each row index is a candidate (n_clusters).\n    Each column contains a ranking of candidates (row indices), with lower values being better.\n    \n    Based on: A New Monotonic, Clone-Independent, Reversal Symmetric, and Condorcet-Consistent \n              Single-Winner Election Method by Markus Schulze\n              http://www.9mail.de/m-schulze/schulze1.pdf\n    \"\"\"\n\n    if df.shape[0] == 0:\n        raise ValueError(\"Input DataFrame has no rows.\")\n\n    if voter_weights is None:\n        voter_weights = {voter: 1.0 for voter in df.columns}\n    else:\n        # If voter_weights exists but doesn't include all voters, add missing voters with weight 1.0\n        for voter in df.columns:\n            if voter not in voter_weights:\n                voter_weights[voter] = 1.0\n\n    # Normalize voter_weights to sum to the total number of voters\n    n_voters = len(df.columns)\n    total_weight = sum(voter_weights.values())\n    if total_weight != n_voters and total_weight > 0:\n        scale = n_voters / total_weight\n        voter_weights = {k: v * scale for k, v in voter_weights.items()}\n\n    candidates = df.index.to_numpy()\n\n    Pd, pred = _shulze_pairwise_preference(df, voter_weights=voter_weights)\n    Pd, pred = _shulze_path_strength(Pd, pred)\n    candidate_wins = _shulze_rank_strength(Pd, pred)\n\n    df_wins = pd.DataFrame({\n        \"candidate\": candidates,\n        \"wins\": candidate_wins\n    })\n\n    # If df_wins is empty, return 0\n    if df_wins.empty:\n        if not return_preference_df:\n            return 0\n        else:\n            df_pref = df.stack().reset_index()\n            df_pref.columns = [\"preference\", \"score_algo\", \"n_clusters\"]\n            df_pref = df_pref.pivot(index=\"n_clusters\", columns=\"score_algo\", values=\"preference\")\n            \n            return 0, df_pref\n\n    if window_size > 0:\n        df_wins[\"wins\"] = gaussian_filter1d(\n            df_wins[\"wins\"], \n            sigma=window_size,\n            mode=\"nearest\", # constant or nearest?\n            cval=0.0 # for constant mode\n        )\n\n    # this should select the smallest candidate if there is a tie\n    # there is a procedure for this in the paper if we want to improve this later\n    winner_idx = int(np.argmax(df_wins[\"wins\"]))\n\n    if not return_preference_df:\n        return winner_idx\n\n    # Change each voter column in df to preference starting at zero\n    df_pref = df.stack().reset_index()\n    df_pref.columns = [\"preference\", \"score_algo\", \"n_clusters\"]\n    df_pref = df_pref.pivot(index=\"n_clusters\", columns=\"score_algo\", values=\"preference\")\n\n    # invert preferences so that higher is better\n    # df_pref = np.max(df_pref) - df_pref\n\n    # Join df_pref and df_wins on the index (n_clusters/candidate)\n    df_pref = df_pref.merge(df_wins, how=\"left\", left_index=True, right_on=\"candidate\")\n    df_pref = df_pref.set_index(\"candidate\")\n\n    df_pref[\"wins\"] = df_pref[\"wins\"].astype(int)\n\n    return winner_idx, df_pref"
  },
  {
    "path": "opendsm/common/const.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\n\n\ndefault_season_def = {\n    \"options\": [\"summer\", \"shoulder\", \"winter\"],\n    \"January\": \"winter\",\n    \"February\": \"winter\",\n    \"March\": \"shoulder\",\n    \"April\": \"shoulder\",\n    \"May\": \"shoulder\",\n    \"June\": \"summer\",\n    \"July\": \"summer\",\n    \"August\": \"summer\",\n    \"September\": \"summer\",\n    \"October\": \"shoulder\",\n    \"November\": \"winter\",\n    \"December\": \"winter\",\n}\n\n\ndefault_weekday_weekend_def = {\n    \"options\": [\"weekday\", \"weekend\"],\n    \"Monday\": \"weekday\",\n    \"Tuesday\": \"weekday\",\n    \"Wednesday\": \"weekday\",\n    \"Thursday\": \"weekday\",\n    \"Friday\": \"weekday\",\n    \"Saturday\": \"weekend\",\n    \"Sunday\": \"weekend\",\n}\n\n\nclass CAlgoChoice(str, Enum):\n    IQR_LEGACY = \"iqr_legacy\"\n    IQR = \"iqr\"\n    MAD = \"mad\"\n    STDEV = \"stdev\"\n\n\nclass TutorialDataChoice(str, Enum):\n    \"\"\"\n    Options for the tutorial data to load.\n    \"\"\"\n\n    FEATURES = \"features\"\n    SEASONAL_HOUR_DAY_WEEK_LOADSHAPE = \"seasonal_hourly_day_of_week_loadshape\".replace(\n        \"_\", \"\"\n    )\n    SEASONAL_DAY_WEEK_LOADSHAPE = \"seasonal_day_of_week_loadshape\".replace(\"_\", \"\")\n    MONTH_LOADSHAPE = \"month_loadshape\".replace(\"_\", \"\")\n    HOURLY_COMPARISON_GROUP_DATA = \"hourly_comparison_group_data\".replace(\"_\", \"\")\n    HOURLY_TREATMENT_DATA = \"hourly_treatment_data\".replace(\"_\", \"\")\n    DAILY_COMPARISON_GROUP_DATA = \"daily_comparison_group_data\".replace(\"_\", \"\")\n    DAILY_TREATMENT_DATA = \"daily_treatment_data\".replace(\"_\", \"\")\n    MONTHLY_COMPARISON_GROUP_DATA = \"monthly_comparison_group_data\".replace(\"_\", \"\")\n    MONTHLY_TREATMENT_DATA = \"monthly_treatment_data\".replace(\"_\", \"\")"
  },
  {
    "path": "opendsm/common/hourly_interpolation.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport warnings\n\nimport numba\nimport numpy as np\nimport numpy.ma as ma\n\nfrom sklearn.experimental import enable_iterative_imputer\nfrom sklearn.impute import IterativeImputer\nfrom sklearn.linear_model import BayesianRidge\n\nfrom scipy.interpolate import RBFInterpolator\n\nfrom copy import deepcopy as copy\n\n\ndef autocorr_fcn(x, lags, exclude_0=True):\n    \"\"\"manualy compute, non partial\"\"\"\n    x_msk = ma.masked_invalid(x)\n    mean = ma.mean(x_msk)\n    var = ma.var(x_msk)\n    xp = x_msk - mean\n    corr = [1.0 if l == 0 else ma.sum(xp[l:] * xp[:-l]) / len(x) / var for l in lags]\n\n    with warnings.catch_warnings():\n        warnings.filterwarnings(\"ignore\", \"Warning: converting a masked element to nan\")\n        # combine the lags, the correlation values, and mirror to get leads/lags\n        res = np.vstack((lags, corr)).T\n    if exclude_0:  # remove the 0 lag\n        res = res[1:]\n        rev_res = copy(res)[::-1]\n    else:\n        rev_res = copy(res)[::-1][:-1]\n\n    rev_res[:, 0] = -rev_res[:, 0]\n    res = np.vstack((rev_res, res))\n\n    return res\n\n\n# unused\ndef autocorr_fcn2(x, lags):\n    \"\"\"np.correlate, non partial\"\"\"\n    x_msk = ma.masked_invalid(x)\n    mean = ma.mean(x_msk)\n    var = ma.var(x_msk)\n    xp = x_msk - mean\n\n    corr = ma.correlate(xp, xp, \"full\")[len(x) - 1 :] / var / len(x)\n\n    return corr[: len(lags)]\n\n\n# unused\ndef autocorr_fcn3(x, lags):\n    \"\"\"fft, pad 0s, non partial\"\"\"\n    x_msk = ma.masked_invalid(x)\n    n = len(x)\n    # pad 0s to 2n-1\n    ext_size = 2 * n - 1\n    # nearest power of 2\n    fsize = 2 ** np.ceil(np.log2(ext_size)).astype(\"int\")\n\n    mean = ma.mean(x_msk)\n    var = ma.var(x_msk)\n    xp = x - mean\n\n    # do fft and ifft\n    cf = np.fft.fft(xp, fsize)\n    sf = cf.conjugate() * cf\n    corr = np.fft.ifft(sf).real\n    corr = corr / var / n\n\n    return corr[: len(lags)]\n\n\n# unused\ndef multiple_imputation(df, columns=None, **kwargs):\n    # get indices of missing values\n    missing_idx = df[columns].isna().any(axis=1)\n\n    df_imputed = df[columns].reset_index()\n    # convert datetime to hours since earliest datetime\n    df_imputed[\"datetime_elapsed\"] = (\n        df_imputed[\"datetime\"] - df_imputed[\"datetime\"].min()\n    ).dt.total_seconds() / 3600\n    df_imputed[\"hour_of_day\"] = df_imputed[\"datetime\"].dt.hour\n    df_imputed[\"day_of_week\"] = df_imputed[\"datetime\"].dt.dayofweek\n    df_imputed[\"month\"] = df_imputed[\"datetime\"].dt.month\n    df_imputed = df_imputed.set_index(\"datetime\")\n\n    settings_dict = {\n        \"estimator\": BayesianRidge(),  # can use SVR, BayesianRidge, etc.\n        \"max_iter\": 10,\n        \"random_state\": None,\n    }\n    settings_dict.update(kwargs)\n\n    imputer = IterativeImputer(**settings_dict)\n    imputer.fit(df_imputed)\n    df_imputed[:] = imputer.transform(df_imputed)\n\n    # add df_imputed back to df\n    df.loc[missing_idx, columns] = df_imputed.loc[missing_idx, columns]\n\n    # add additional columns to indicate which values were imputed\n    for col in columns:\n        interp_bool_col = f\"interpolated_{col}\"\n\n        df[interp_bool_col] = False\n        df.loc[missing_idx, interp_bool_col] = True\n\n    return df\n\n\n# @numba.njit\ndef shift_array(arr, num, fill_value=np.nan):\n    # Courtesy of https://stackoverflow.com/questions/30399534/shift-elements-in-a-numpy-array\n    # get size of arr\n    arr_size = arr.shape[0]\n\n    if arr_size <= 20000:\n        if num >= 0:\n            return np.concatenate((np.full(num, fill_value), arr[:-num]))\n        else:\n            return np.concatenate((arr[-num:], np.full(-num, fill_value)))\n    else:\n        result = np.empty_like(arr)\n\n        if num > 0:\n            result[:num] = fill_value\n            result[num:] = arr[:-num]\n        elif num < 0:\n            result[num:] = fill_value\n            result[:num] = arr[-num:]\n        else:\n            result[:] = arr\n\n        return result\n\n\ndef _interpolate_col(x, lags):\n    # check that the column has nans\n    if x.isna().sum() == 0:\n        return x\n\n    elif x.isna().sum() == len(x):\n        return x\n\n    # calculate the number of lags and leads to consider\n    if x.name == \"observed\":\n        missing_frac = x.isna().sum() / len(x)\n        n_cor_idx_heuristic = (\n            np.round((4.012 * np.log(missing_frac) + 24.38) / 2, 0) * 2\n        )\n        n_cor_idx = int(np.max([6, n_cor_idx_heuristic]))\n    else:\n        n_cor_idx = 6\n\n    # Calculate the correlation of col with its lags and leads\n    # create lags from -lags to lags\n    lag_array = np.arange(lags + 1)\n    autocorr = autocorr_fcn(x.values, lag_array, exclude_0=True)\n\n    # take the largest n_cor_idx from second column using argpartition\n    idx = np.argpartition(autocorr[:, 1], -n_cor_idx)[-n_cor_idx:]\n    autocorr = autocorr[idx]\n\n    # sort autocorr by the autocorrelation value\n    autocorr = autocorr[np.argsort(autocorr[:, 1])[::-1]]\n    autocorr_idx = autocorr[:, 0]\n\n    # interpolate and update the values\n    max_iter = 10\n    for i, cnt_min in enumerate(np.linspace(n_cor_idx, 1, max_iter).astype(int)):\n        num_rows = x.shape[0]\n        num_cols = len(autocorr_idx)\n\n        autocorr_helpers = np.empty((num_rows, num_cols))\n        for i in range(num_cols):\n            shift = int(autocorr_idx[i])\n            autocorr_helpers[:, i] = shift_array(x.values, shift)\n\n        # get the indices of the missing values\n        nan_series_idx = x.index[x.isna()]\n        nan_idx = x.index.get_indexer(nan_series_idx)\n\n        # nan values where helpers are not nan\n        valid_idx = np.sum(~np.isnan(autocorr_helpers[nan_idx, :]), axis=1) >= cnt_min\n        if valid_idx.sum() == 0:\n            continue\n\n        nan_series_idx = nan_series_idx[valid_idx]\n        nan_idx = x.index.get_indexer(nan_series_idx)\n\n        # for each row, if the value is missing, calculate the mean of the lags and leads\n        # ignore FutureWarning from pandas for now\n        with warnings.catch_warnings():\n            warnings.simplefilter(action=\"ignore\", category=FutureWarning)\n            x.loc[nan_series_idx] = np.nanmean(autocorr_helpers[nan_idx, :], axis=1)\n\n        if x.isna().sum() == 0:\n            break\n\n    return x\n\n\ndef interpolate(df, columns=None):\n    skip_autocorr_interpolation = False\n    if len(df) > 6 * 24 * 7:\n        lags = 24 * 7 * 2 + 1\n    elif (len(df) > 3 * 24 * 7) and (len(df) <= 6 * 24 * 7):\n        lags = 24 * 7 + 1\n    elif (len(df) > 3 * 24) and (len(df) <= 3 * 24 * 7):\n        lags = 24 + 1\n    else:\n        skip_autocorr_interpolation = True\n\n    interp_cols = columns\n    if interp_cols is None:\n        interp_cols = [\"temperature\", \"ghi\", \"observed\"]\n\n    # check if the columns are in the dataframe and modify columns appropriately\n    for col in interp_cols:\n        if col not in df.columns:\n            continue\n\n        interp_bool_col = f\"interpolated_{col}\"\n        if interp_bool_col in df.columns:\n            continue\n\n        # main interpolation method\n        idx_missing = df.loc[df[col].isna()].index\n        if not skip_autocorr_interpolation:\n            df[col] = _interpolate_col(df[col].copy(), lags)\n\n        # backup interpolation methods\n        for method in [\"time\", \"ffill\", \"bfill\"]:\n            na_datetime = df.loc[df[col].isna()].index\n            if len(na_datetime) == 0:\n                break\n\n            if method == \"time\":\n                df[col] = df[col].interpolate(method=\"time\", limit_direction=\"both\")\n\n            elif method == \"ffill\":\n                df[col] = df[col].ffill()\n\n            elif method == \"bfill\":\n                df[col] = df[col].bfill()\n\n        # TODO: we can check if we have similar values multiple times back to back, if yes, raise a warning\n        # where na_datetime_original is True and the col is not na, set the interpolation boolean to True\n        df[interp_bool_col] = False\n        df.loc[df.index.isin(idx_missing) & ~df[col].isna(), interp_bool_col] = True\n\n    return df\n"
  },
  {
    "path": "opendsm/common/metrics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pydantic\nfrom typing import Union, Optional\nfrom enum import Enum\nfrom functools import cached_property\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.stats import pearsonr\n\nfrom opendsm.common.utils import safe_divide\nfrom opendsm.common.stats.basic import (\n    median_absolute_deviation,\n    t_stat,\n)\nfrom opendsm.common.pydantic_utils import (\n    ArbitraryPydanticModel,\n    PydanticDf,\n    PydanticFromDict,\n    computed_field_cached_property,\n)\n\n\n\n_MIN_DENOMINATOR: float = 1e-3\n\n\nclass ColumnMetrics(ArbitraryPydanticModel):\n    \"\"\"Statistical metrics for a single pandas Series.\n\n    Computes various statistical measures including mean, variance, standard\n    deviation, median, and distribution characteristics for a data series.\n\n    Parameters\n    ----------\n    series : pd.Series\n        Input data series for metric calculations.\n    \"\"\"\n    series: pd.Series = pydantic.Field(\n        exclude=True,\n        repr=False,\n    )\n\n    @computed_field_cached_property()\n    def sum(self) -> float:\n        \"\"\"Calculate the sum of all values in the series.\n\n        Returns\n        -------\n        float\n            Sum of series values.\n        \"\"\"\n        return self.series.sum()\n\n    @computed_field_cached_property()\n    def mean(self) -> float:\n        \"\"\"Calculate the arithmetic mean of the series.\n\n        Returns\n        -------\n        float\n            Mean value, or 0.0 if series is empty.\n        \"\"\"\n        n = len(self.series)\n        if n == 0:\n            return 0.0\n        return self.sum / n\n\n    @computed_field_cached_property()\n    def variance(self) -> float:\n        \"\"\"Calculate the variance of the series.\n\n        Uses population variance (ddof=0).\n\n        Returns\n        -------\n        float\n            Variance value.\n        \"\"\"\n        return self.series.var(ddof=0)\n\n    @computed_field_cached_property()\n    def std(self) -> float:\n        \"\"\"Calculate the standard deviation of the series.\n\n        Returns\n        -------\n        float\n            Standard deviation value.\n        \"\"\"\n        return self.variance**0.5\n\n    @computed_field_cached_property()\n    def cvstd(self) -> float:\n        \"\"\"Calculate coefficient of variation of standard deviation.\n\n        Ratio of standard deviation to mean, providing a normalized\n        measure of dispersion. Useful for comparing variability across\n        datasets with different scales.\n\n        Returns\n        -------\n        float\n            Coefficient of variation.\n        \"\"\"\n        return safe_divide(self.std, self.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def sum_squared(self) -> float:\n        \"\"\"Calculate the sum of squared values.\n\n        Used in various statistical calculations including variance and SSE.\n\n        Returns\n        -------\n        float\n            Sum of squared values.\n        \"\"\"\n        return (self.series**2).sum()\n\n    @computed_field_cached_property()\n    def median(self) -> float:\n        \"\"\"Calculate the median of the series.\n\n        The middle value that separates the higher half from the lower half.\n\n        Returns\n        -------\n        float\n            Median value.\n        \"\"\"\n        return self.series.median()\n\n    @computed_field_cached_property()\n    def MAD_scaled(self) -> float:\n        \"\"\"Calculate scaled Median Absolute Deviation (MAD).\n\n        A robust measure of statistical dispersion that is less sensitive\n        to outliers than standard deviation. Scaled to be consistent with\n        the standard deviation for normally distributed data.\n\n        Returns\n        -------\n        float\n            Scaled MAD value.\n        \"\"\"\n        return median_absolute_deviation(self.series, self.median)\n\n    @computed_field_cached_property()\n    def iqr(self) -> float:\n        \"\"\"Calculate the interquartile range (IQR).\n\n        The difference between the 75th and 25th percentiles, providing a robust\n        measure of spread that is resistant to outliers.\n\n        Returns\n        -------\n        float\n            Interquartile range.\n        \"\"\"\n        return np.diff(np.quantile(self.series, [0.25, 0.75]))[0]\n\n    @computed_field_cached_property()\n    def skew(self) -> float:\n        \"\"\"Calculate the skewness of the distribution.\n\n        Measures the asymmetry of the distribution. Positive values indicate\n        right skew, negative values indicate left skew.\n\n        Returns\n        -------\n        float\n            Skewness value.\n        \"\"\"\n        return self.series.skew()\n\n    @computed_field_cached_property()\n    def kurtosis(self) -> float:\n        \"\"\"Calculate the kurtosis of the distribution.\n\n        Measures the \"tailedness\" of the distribution. Higher values indicate\n        heavier tails and more outliers.\n\n        Returns\n        -------\n        float\n            Kurtosis value (excess kurtosis, where normal distribution = 0).\n        \"\"\"\n        return self.series.kurtosis()\n\n\ndef A_n(x: np.ndarray, n: float) -> float:\n    \"\"\"Calculate the proportion of values in x that are less than or equal to n.\n\n    Parameters\n    ----------\n    x : np.ndarray\n        Input array\n    n : float\n        Threshold value\n\n    Returns\n    -------\n    float\n        Proportion of values <= n\n    \"\"\"\n    return np.mean(x <= n)\n\nclass BaselineMetrics(ArbitraryPydanticModel):\n    \"\"\"Comprehensive baseline model evaluation metrics.\n\n    Calculates a wide range of statistical measures and goodness-of-fit metrics\n    for evaluating baseline energy model performance. Includes error metrics,\n    normalized metrics, percentage error metrics, efficiency metrics, and\n    autocorrelation-adjusted variants following ASHRAE Guideline 14 methodology.\n\n    Parameters\n    ----------\n    df : pd.DataFrame\n        DataFrame with 'observed' and 'predicted' columns containing model\n        baseline period data.\n    num_model_params : int\n        Number of parameters in the baseline model (used for degrees of freedom\n        adjustments). Must be >= 1.\n\n    Attributes\n    ----------\n    n : float\n        Number of valid observations\n    n_prime : float\n        Autocorrelation-adjusted sample size\n    ddof : float\n        Delta degrees of freedom\n    ddof_autocorr : float\n        Autocorrelation-adjusted degrees of freedom\n    observed : ColumnMetrics\n        Statistical metrics for observed data\n    predicted : ColumnMetrics\n        Statistical metrics for predicted data\n    residuals : ColumnMetrics\n        Statistical metrics for residuals\n    max_error : float\n        Maximum absolute error\n    mae : float\n        Mean absolute error\n    nmae : float\n        Normalized mean absolute error (by mean)\n    pnmae : float\n        Percentile normalized MAE (by IQR)\n    medae : float\n        Median absolute error\n    mbe : float\n        Mean bias error\n    nmbe : float\n        Normalized mean bias error (by mean)\n    pnmbe : float\n        Percentile normalized MBE (by IQR)\n    sse : float\n        Sum of squared errors\n    mse : float\n        Mean squared error\n    rmse : float\n        Root mean squared error\n    rmse_autocorr : float\n        Autocorrelation-corrected RMSE\n    rmse_adj : float\n        Adjusted RMSE (by ddof)\n    rmse_autocorr_adj : float\n        Fully adjusted RMSE\n    cvrmse : float\n        Coefficient of variation RMSE\n    cvrmse_autocorr : float\n        Autocorrelation-corrected CVRMSE\n    cvrmse_adj : float\n        Adjusted CVRMSE\n    cvrmse_autocorr_adj : float\n        Fully adjusted CVRMSE\n    pnrmse : float\n        Percentile normalized RMSE\n    pnrmse_autocorr : float\n        Autocorrelation-corrected PNRMSE\n    pnrmse_adj : float\n        Adjusted PNRMSE\n    pnrmse_autocorr_adj : float\n        Fully adjusted PNRMSE\n    r_squared : float\n        Coefficient of determination\n    r_squared_adj : float\n        Adjusted R-squared\n    mape : float\n        Mean absolute percentage error\n    smape : float\n        Symmetric MAPE\n    wape : float\n        Weighted absolute percentage error\n    swape : float\n        Symmetric WAPE\n    maape : float\n        Mean arctangent absolute percentage error\n    nse : float\n        Nash-Sutcliffe efficiency\n    nnse : float\n        Normalized Nash-Sutcliffe efficiency\n    kge : Optional[float]\n        Kling-Gupta efficiency\n    a10 : float\n        Proportion within 10% accuracy\n    a20 : float\n        Proportion within 20% accuracy\n    a30 : float\n        Proportion within 30% accuracy\n    wi : float\n        Willmott index\n    index_of_agreement : float\n        Refined Willmott index\n    pearson_r : float\n        Pearson correlation coefficient\n    pi : float\n        Performance index\n    pi_rating : str\n        Performance rating (excellent/very good/good/satisfactory/poor/bad/very bad)\n    explained_variance_score : float\n        Explained variance score\n\n    Notes\n    -----\n    All metrics are computed on valid (finite) observations only. Autocorrelation\n    adjustments follow ASHRAE Guideline 14 methodology for M&V applications.\n    \"\"\"\n\n    df: pd.DataFrame = pydantic.Field(\n        exclude=True,\n        repr=False,\n        description=\"Input dataframe with 'observed' and 'predicted' columns\",\n    )\n\n    num_model_params: int = pydantic.Field(\n        ge=1,\n        validate_default=True,\n        description=\"Number of parameters in the baseline model\",\n    )\n\n    @cached_property\n    def _df(self) -> pd.DataFrame:\n        \"\"\"Prepare and validate the input dataframe.\n\n        Validates column types, filters non-finite values, and computes residuals.\n\n        Returns\n        -------\n        pd.DataFrame\n            Processed dataframe with 'observed', 'predicted', and 'residuals' columns.\n\n        Raises\n        ------\n        ValueError\n            If input dataframe is empty.\n        \"\"\"\n        _df = self.df[[\"observed\", \"predicted\"]].copy()\n\n        if len(_df) < 1:\n            raise ValueError(\"Input dataframe must have at least one row\")\n\n        # Check dataframe\n        expected_columns = {\"observed\": \"float\", \"predicted\": \"float\"}\n        _df = PydanticDf(df=_df, column_types=expected_columns).df\n\n        # drop non finite values from df\n        _df = _df[np.isfinite(_df[\"observed\"]) & np.isfinite(_df[\"predicted\"])]\n\n        # get residuals\n        _df[\"residuals\"] = _df[\"observed\"] - _df[\"predicted\"]\n\n        return _df\n\n    @computed_field_cached_property()\n    def n(self) -> float:\n        \"\"\"Calculate the number of observations.\n\n        Returns the count of valid observations after filtering non-finite values.\n\n        Returns\n        -------\n        float\n            Number of observations.\n        \"\"\"\n        return len(self._df)\n\n    @computed_field_cached_property()\n    def n_prime(self) -> float:\n        \"\"\"Calculate effective sample size corrected for autocorrelation.\n\n        Adjusts the sample size to account for autocorrelation in residuals,\n        following ASHRAE Guideline 14 methodology. Uses lag-1 autocorrelation\n        as recommended in LBNL technical report.\n\n        Reference: https://www.osti.gov/servlets/purl/1366449\n\n        Returns\n        -------\n        float\n            Effective sample size (minimum value of 1).\n        \"\"\"\n        # lag should be 1 according to LBNL guidance\n        autocorr = acf(self._df[\"residuals\"].values, lag_n=1, ac_type=\"moving_stats\")[1]\n\n        numerator = self.n * (1 - autocorr)\n        denominator = 1 + autocorr\n        _n_prime = safe_divide(numerator, denominator, _MIN_DENOMINATOR)\n\n        # Ensure valid result\n        if not np.isfinite(_n_prime) or _n_prime < 1:\n            _n_prime = 1\n\n        return _n_prime\n\n    @computed_field_cached_property()\n    def ddof(self) -> float:\n        \"\"\"Calculate delta degrees of freedom (ddof).\n\n        The number of independent observations minus the number of model\n        parameters. Used in adjusted statistical calculations like\n        adjusted RMSE and R-squared.\n\n        Returns\n        -------\n        float\n            Delta degrees of freedom (minimum value of 1).\n        \"\"\"\n        _ddof = self.n - self.num_model_params\n\n        if _ddof < 1:\n            _ddof = 1\n\n        return _ddof\n\n    @computed_field_cached_property()\n    def ddof_autocorr(self) -> float:\n        \"\"\"Calculate autocorrelation-adjusted delta degrees of freedom.\n\n        Similar to ddof but uses the effective sample size (n_prime) that\n        accounts for autocorrelation in residuals. Used in ASHRAE Guideline 14\n        uncertainty calculations.\n\n        Returns\n        -------\n        float\n            Autocorrelation-adjusted ddof (minimum value of 1).\n        \"\"\"\n        _ddof_autocorr = self.n_prime - self.num_model_params\n\n        if _ddof_autocorr < 1:\n            _ddof_autocorr = 1\n\n        return _ddof_autocorr\n\n    @computed_field_cached_property()\n    def observed(self) -> ColumnMetrics:\n        \"\"\"Calculate statistical metrics for observed values.\n\n        Returns\n        -------\n        ColumnMetrics\n            Statistical metrics for the observed data column.\n        \"\"\"\n        return ColumnMetrics(series=self._df[\"observed\"])\n\n    @computed_field_cached_property()\n    def predicted(self) -> ColumnMetrics:\n        \"\"\"Calculate statistical metrics for predicted values.\n\n        Returns\n        -------\n        ColumnMetrics\n            Statistical metrics for the predicted data column.\n        \"\"\"\n        return ColumnMetrics(series=self._df[\"predicted\"])\n\n    @computed_field_cached_property()\n    def residuals(self) -> ColumnMetrics:\n        \"\"\"Calculate statistical metrics for residuals.\n\n        Returns\n        -------\n        ColumnMetrics\n            Statistical metrics for the residuals (observed - predicted).\n        \"\"\"\n        return ColumnMetrics(series=self._df[\"residuals\"])\n\n    @computed_field_cached_property()\n    def max_error(self) -> float:\n        \"\"\"Calculate maximum absolute error.\n\n        The largest absolute difference between predicted and observed values.\n        Useful for understanding worst-case prediction errors.\n\n        Returns\n        -------\n        float\n            Maximum absolute error.\n        \"\"\"\n        return np.max(np.abs(self._df[\"residuals\"].values))\n\n    @computed_field_cached_property()\n    def mae(self) -> float:\n        \"\"\"Calculate Mean Absolute Error (MAE).\n\n        The average of absolute differences between predicted and observed values.\n        Provides a straightforward measure of prediction accuracy.\n\n        Returns\n        -------\n        float\n            Mean absolute error.\n        \"\"\"\n        return np.mean(np.abs(self._df[\"residuals\"].values))\n\n    @computed_field_cached_property()\n    def nmae(self) -> float:\n        \"\"\"Calculate Normalized Mean Absolute Error (NMAE).\n\n        Normalizes MAE by the mean of observed values. Commonly used in\n        ASHRAE Guideline 14 for model validation. Lower values indicate\n        better performance.\n\n        Returns\n        -------\n        float\n            NMAE value.\n        \"\"\"\n        return safe_divide(self.mae, self.observed.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def pnmae(self) -> float:\n        \"\"\"Calculate Percentile Normalized Mean Absolute Error (PNMAE).\n\n        Normalizes MAE by the interquartile range (IQR) of observed values\n        instead of the mean, making it more robust to outliers and extreme values.\n\n        Returns\n        -------\n        float\n            PNMAE value.\n        \"\"\"\n        return safe_divide(self.mae, self.observed.iqr, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def medae(self) -> float:\n        \"\"\"Calculate Median Absolute Error (MedAE).\n\n        The median of absolute errors, providing a robust central measure of\n        prediction error that is less sensitive to outliers than MAE.\n\n        Returns\n        -------\n        float\n            Median absolute error.\n        \"\"\"\n        return np.median(np.abs(self._df[\"residuals\"].values))\n\n    @computed_field_cached_property()\n    def mbe(self) -> float:\n        \"\"\"Calculate Mean Bias Error (MBE).\n\n        The average of residuals (observed - predicted), indicating systematic\n        bias in predictions. Positive values indicate under-prediction, negative\n        values indicate over-prediction.\n\n        Returns\n        -------\n        float\n            Mean bias error.\n        \"\"\"\n        return self.residuals.mean\n\n    @computed_field_cached_property()\n    def nmbe(self) -> float:\n        \"\"\"Calculate Normalized Mean Bias Error (NMBE).\n\n        Normalizes MBE by the mean of observed values. Measures systematic\n        bias in predictions (over- or under-prediction). Used in ASHRAE\n        Guideline 14 for model validation. Values near zero indicate\n        unbiased predictions.\n\n        Returns\n        -------\n        float\n            NMBE value.\n        \"\"\"\n        return safe_divide(self.mbe, self.observed.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def pnmbe(self) -> float:\n        \"\"\"Calculate Percentile Normalized Mean Bias Error (PNMBE).\n\n        Normalizes MBE by the interquartile range (IQR) of observed values\n        instead of the mean, providing a robust measure of bias that is less\n        sensitive to outliers.\n\n        Returns\n        -------\n        float\n            PNMBE value.\n        \"\"\"\n        return safe_divide(self.mbe, self.observed.iqr, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def sse(self) -> float:\n        \"\"\"Calculate Sum of Squared Errors (SSE).\n\n        The sum of squared residuals, a fundamental measure used in many\n        statistical calculations and goodness-of-fit metrics.\n\n        Returns\n        -------\n        float\n            Sum of squared errors.\n        \"\"\"\n        return self.residuals.sum_squared\n\n    @computed_field_cached_property()\n    def mse(self) -> float:\n        \"\"\"Calculate Mean Squared Error (MSE).\n\n        The average of squared residuals, penalizing larger errors more than\n        smaller ones. Square root of MSE gives RMSE.\n\n        Returns\n        -------\n        float\n            Mean squared error.\n        \"\"\"\n        return self.sse / self.n\n\n    @computed_field_cached_property()\n    def rmse(self) -> float:\n        \"\"\"Calculate Root Mean Squared Error (RMSE).\n\n        The square root of MSE, providing an error metric in the same units\n        as the original data. Commonly used for model evaluation.\n\n        Returns\n        -------\n        float\n            Root mean squared error.\n        \"\"\"\n        return self.mse**0.5\n\n    @computed_field_cached_property()\n    def rmse_autocorr(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected RMSE.\n\n        RMSE adjusted for autocorrelation in residuals using the effective\n        sample size (n_prime). More accurate for time-series data.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected RMSE.\n        \"\"\"\n        return (self.sse / self.n_prime) ** 0.5\n\n    @computed_field_cached_property()\n    def rmse_adj(self) -> float:\n        \"\"\"Calculate adjusted RMSE.\n\n        RMSE adjusted for degrees of freedom to account for model complexity.\n        Penalizes models with more parameters.\n\n        Returns\n        -------\n        float\n            Adjusted RMSE.\n        \"\"\"\n        return (self.sse / self.ddof) ** 0.5\n\n    @computed_field_cached_property()\n    def rmse_autocorr_adj(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected and adjusted RMSE.\n\n        RMSE with both autocorrelation and degrees-of-freedom adjustments,\n        providing the most robust error metric for time-series modeling.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected and adjusted RMSE.\n        \"\"\"\n        return (self.sse / self.ddof_autocorr) ** 0.5\n\n    @computed_field_cached_property()\n    def cvrmse(self) -> float:\n        \"\"\"Calculate Coefficient of Variation of Root Mean Squared Error (CVRMSE).\n\n        Normalizes RMSE by the mean of observed values, making it a\n        dimensionless measure of model fit quality. Commonly used in\n        ASHRAE Guideline 14 for M&V applications. Lower values indicate\n        better performance.\n\n        Returns\n        -------\n        float\n            CVRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse, self.observed.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def cvrmse_autocorr(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected CVRMSE.\n\n        CVRMSE using autocorrelation-adjusted RMSE for better handling of\n        time-series data with correlated residuals.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected CVRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse_autocorr, self.observed.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def cvrmse_adj(self) -> float:\n        \"\"\"Calculate adjusted CVRMSE.\n\n        CVRMSE using degrees-of-freedom adjusted RMSE to account for\n        model complexity. Used in ASHRAE Guideline 14 uncertainty calculations.\n\n        Returns\n        -------\n        float\n            Adjusted CVRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse_adj, self.observed.mean, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def cvrmse_autocorr_adj(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected and adjusted CVRMSE.\n\n        CVRMSE using both autocorrelation and degrees-of-freedom adjustments\n        for the most robust normalized error metric in time-series modeling.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected and adjusted CVRMSE value.\n        \"\"\"\n        return safe_divide(\n            self.rmse_autocorr_adj, self.observed.mean, _MIN_DENOMINATOR\n        )\n\n    @computed_field_cached_property()\n    def pnrmse(self) -> float:\n        \"\"\"Calculate Percentile Normalized Root Mean Squared Error (PNRMSE).\n\n        Normalizes RMSE by the interquartile range (IQR) instead of the mean,\n        providing a robust dimensionless error metric that is less sensitive\n        to outliers.\n\n        Returns\n        -------\n        float\n            PNRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse, self.observed.iqr, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def pnrmse_autocorr(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected PNRMSE.\n\n        PNRMSE using autocorrelation-adjusted RMSE for better handling of\n        time-series data with correlated residuals.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected PNRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse_autocorr, self.observed.iqr, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def pnrmse_adj(self) -> float:\n        \"\"\"Calculate adjusted PNRMSE.\n\n        PNRMSE using degrees-of-freedom adjusted RMSE to account for\n        model complexity.\n\n        Returns\n        -------\n        float\n            Adjusted PNRMSE value.\n        \"\"\"\n        return safe_divide(self.rmse_adj, self.observed.iqr, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def pnrmse_autocorr_adj(self) -> float:\n        \"\"\"Calculate autocorrelation-corrected and adjusted PNRMSE.\n\n        PNRMSE using both autocorrelation and degrees-of-freedom adjustments\n        for the most robust error metric in time-series with model complexity.\n\n        Returns\n        -------\n        float\n            Autocorrelation-corrected and adjusted PNRMSE value.\n        \"\"\"\n        return safe_divide(\n            self.rmse_autocorr_adj, self.observed.iqr, _MIN_DENOMINATOR\n        )\n\n    @computed_field_cached_property()\n    def r_squared(self) -> float:\n        \"\"\"Calculate coefficient of determination (R²).\n\n        Represents the proportion of variance in the observed data that is\n        predictable from the model. Ranges from 0 to 1, with 1 indicating\n        perfect prediction.\n\n        Returns\n        -------\n        float\n            R-squared value.\n        \"\"\"\n        return self._df[[\"predicted\", \"observed\"]].corr().iloc[0, 1] ** 2\n\n    @computed_field_cached_property()\n    def r_squared_adj(self) -> float:\n        \"\"\"Calculate adjusted R-squared.\n\n        Adjusts R-squared for the number of model parameters, penalizing\n        model complexity. More appropriate than R-squared when comparing\n        models with different numbers of parameters.\n\n        Returns\n        -------\n        float\n            Adjusted R-squared value.\n        \"\"\"\n        n = self.n\n        n_adj = self.ddof\n\n        num = (1 - self.r_squared) * (n - 1)\n        den = n_adj - 1\n\n        res = safe_divide(num, den, _MIN_DENOMINATOR)\n\n        return 1 - res\n\n    @computed_field_cached_property()\n    def mape(self) -> float:\n        \"\"\"Calculate Mean Absolute Percentage Error (MAPE).\n\n        Expresses prediction accuracy as a percentage of the observed values.\n        Lower values indicate better performance. Can be problematic when\n        observed values are close to zero.\n\n        Returns\n        -------\n        float\n            Mean absolute percentage error.\n        \"\"\"\n        df = self._df\n\n        num = np.abs(df[\"residuals\"].values)\n        den = np.abs(df[\"observed\"].values)\n\n        inner = safe_divide(num, den, _MIN_DENOMINATOR)\n\n        return np.mean(inner)\n\n    @computed_field_cached_property()\n    def smape(self) -> float:\n        \"\"\"Calculate Symmetric Mean Absolute Percentage Error (SMAPE).\n\n        A symmetric alternative to MAPE that treats over- and under-predictions\n        equally by using the average of observed and predicted values in the\n        denominator. More robust when values approach zero.\n\n        Returns\n        -------\n        float\n            Symmetric mean absolute percentage error.\n        \"\"\"\n        df = self._df\n\n        num = np.abs(df[\"residuals\"].values)\n        obs = np.abs(df[\"observed\"].values)\n        pred = np.abs(df[\"predicted\"].values)\n        den = (obs + pred) / 2\n\n        inner = safe_divide(num, den, _MIN_DENOMINATOR)\n\n        return np.mean(inner)\n\n    @computed_field_cached_property()\n    def wape(self) -> float:\n        \"\"\"Calculate Weighted Absolute Percentage Error (WAPE).\n\n        Also known as MAD/Mean ratio. Weights errors by the magnitude of\n        observations, making it more robust to outliers than MAPE.\n\n        Returns\n        -------\n        float\n            Weighted absolute percentage error.\n        \"\"\"\n        df = self._df\n\n        num = self.mae * self.n\n        den = np.sum(np.abs(df[\"observed\"].values))\n\n        return safe_divide(num, den, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def swape(self) -> float:\n        \"\"\"Calculate Symmetric Weighted Absolute Percentage Error (SWAPE).\n\n        Combines the symmetry of SMAPE with the weighting approach of WAPE,\n        providing a balanced metric that is robust to both outliers and\n        near-zero values.\n\n        Returns\n        -------\n        float\n            Symmetric weighted absolute percentage error.\n        \"\"\"\n        df = self._df\n\n        num = self.mae * self.n\n        obs = np.abs(df[\"observed\"].values)\n        pred = np.abs(df[\"predicted\"].values)\n        den = np.sum((obs + pred) / 2)\n\n        return safe_divide(num, den, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def maape(self) -> float:\n        \"\"\"Calculate Mean Arctangent Absolute Percentage Error (MAAPE).\n\n        Uses arctangent transformation to bound percentage errors, making it\n        highly robust to outliers and extreme values. Returns values in the\n        range [0, π/2].\n\n        Returns\n        -------\n        float\n            Mean arctangent absolute percentage error.\n        \"\"\"\n        df = self._df\n\n        num = df[\"residuals\"].values\n        den = df[\"observed\"].values\n\n        inner = safe_divide(num, den, _MIN_DENOMINATOR)\n        inner = np.arctan(np.abs(inner))\n\n        return np.mean(inner)\n\n    @computed_field_cached_property()\n    def nse(self) -> float:\n        \"\"\"Calculate Nash-Sutcliffe Efficiency (NSE).\n\n        Measures how well predictions match observations relative to using the\n        mean as a predictor. Ranges from -∞ to 1, with 1 being perfect match,\n        0 meaning the model is no better than the mean, and negative values\n        indicating worse performance than using the mean.\n\n        Returns\n        -------\n        float\n            Nash-Sutcliffe Efficiency value.\n        \"\"\"\n        df = self._df\n\n        num = self.sse\n        den = np.sum((df[\"observed\"].values - self.observed.mean)**2)\n\n        return 1 - safe_divide(num, den, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def nnse(self) -> float:\n        \"\"\"Calculate Normalized Nash-Sutcliffe Efficiency (NNSE).\n\n        A normalized version of NSE that transforms the range to [0, 1],\n        making it easier to interpret. Values closer to 1 indicate better\n        model performance.\n\n        Returns\n        -------\n        float\n            Normalized Nash-Sutcliffe Efficiency value.\n        \"\"\"\n        return safe_divide(1.0, 2 - self.nse, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def kge(self) -> Optional[float]:\n        \"\"\"Calculate Kling-Gupta Efficiency (KGE).\n\n        A comprehensive goodness-of-fit measure that decomposes into\n        correlation, bias, and variability components. Ranges from -∞ to 1,\n        with 1 being perfect agreement.\n\n        Returns\n        -------\n        Optional[float]\n            Kling-Gupta Efficiency value, or None if calculation fails.\n        \"\"\"\n        r = self.pearson_r\n        bias_ratio = safe_divide(self.predicted.mean, self.observed.mean, _MIN_DENOMINATOR)\n        variability_ratio = safe_divide(self.predicted.cvstd, self.observed.cvstd, _MIN_DENOMINATOR)\n\n        # Check if all components are finite\n        if not np.isfinite(r) or not np.isfinite(bias_ratio) or not np.isfinite(variability_ratio):\n            return None\n\n        result = 1 - np.sqrt((r - 1)**2 + (bias_ratio - 1)**2 + (variability_ratio - 1)**2)\n\n        if not np.isfinite(result):\n            return None\n\n        return result\n\n    @cached_property\n    def _relative_errors(self) -> np.ndarray:\n        \"\"\"Cache the relative error calculation used by a10, a20, a30 metrics.\"\"\"\n        numerator = np.abs(self._df[\"residuals\"].values)\n        denominator = np.abs(self._df[\"observed\"].values)\n\n        return safe_divide(numerator, denominator, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def a10(self) -> float:\n        \"\"\"Calculate A10 metric (proportion of predictions within 10% of observed).\n\n        Returns the fraction of predictions where the absolute percentage error\n        is less than or equal to 10%. Higher values indicate better performance.\n\n        Returns\n        -------\n        float\n            Proportion of predictions within 10% accuracy.\n        \"\"\"\n        return A_n(self._relative_errors, 0.1)\n\n    @computed_field_cached_property()\n    def a20(self) -> float:\n        \"\"\"Calculate A20 metric (proportion of predictions within 20% of observed).\n\n        Returns the fraction of predictions where the absolute percentage error\n        is less than or equal to 20%. Higher values indicate better performance.\n\n        Returns\n        -------\n        float\n            Proportion of predictions within 20% accuracy.\n        \"\"\"\n        return A_n(self._relative_errors, 0.2)\n\n    @computed_field_cached_property()\n    def a30(self) -> float:\n        \"\"\"Calculate A30 metric (proportion of predictions within 30% of observed).\n\n        Returns the fraction of predictions where the absolute percentage error\n        is less than or equal to 30%. Higher values indicate better performance.\n\n        Returns\n        -------\n        float\n            Proportion of predictions within 30% accuracy.\n        \"\"\"\n        return A_n(self._relative_errors, 0.3)\n\n    @computed_field_cached_property()\n    def wi(self) -> float:\n        \"\"\"Calculate the Willmott Index of Agreement.\n\n        Measures the degree of model prediction error relative to potential error.\n        Ranges from 0 to 1, with 1 indicating perfect agreement.\n\n        Returns\n        -------\n        float\n            Willmott Index value.\n        \"\"\"\n        df = self._df\n\n        num = self.sse\n\n        mean_obs = self.observed.mean\n        pred_shifted = df[\"predicted\"].values - mean_obs\n        obs_shifted = df[\"observed\"].values - mean_obs\n        den = np.sum((np.abs(pred_shifted) + np.abs(obs_shifted))**2)\n\n        return 1 - safe_divide(num, den, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def index_of_agreement(self) -> float:\n        \"\"\"Calculate the refined Index of Agreement (d_r).\n\n        A refinement of the Willmott Index that is more sensitive to systematic\n        over- or under-prediction. Ranges from -1 to 1, with 1 indicating\n        perfect agreement.\n\n        Reference: Willmott et al. (2012), https://rmets.onlinelibrary.wiley.com/doi/10.1002/joc.2419\n\n        Returns\n        -------\n        float\n            Refined index of agreement value.\n        \"\"\"\n        df = self._df\n\n        num = self.mae * self.n\n        den = 2 * np.sum(np.abs(df[\"observed\"].values - self.observed.mean))\n\n        if num <= den:\n            return 1 - safe_divide(num, den, _MIN_DENOMINATOR)\n\n        return safe_divide(den, num, _MIN_DENOMINATOR) - 1\n    \n    @computed_field_cached_property()\n    def pearson_r(self) -> float:\n        \"\"\"Calculate Pearson correlation coefficient.\n\n        Measures the linear correlation between observed and predicted values.\n        Ranges from -1 to 1, with 1 indicating perfect positive correlation,\n        -1 perfect negative correlation, and 0 no linear correlation.\n\n        Returns\n        -------\n        float\n            Pearson correlation coefficient.\n        \"\"\"\n        return pearsonr(self._df[\"observed\"].values, self._df[\"predicted\"].values)[0]\n\n    @computed_field_cached_property()\n    def pi(self) -> float:\n        \"\"\"Calculate Performance Index (PI).\n\n        Combines Pearson correlation and Willmott Index to provide a\n        comprehensive model performance metric. Ranges from -1 to 1,\n        with higher values indicating better performance.\n\n        Reference: https://doi.org/10.1016/j.asoc.2021.107282\n\n        Returns\n        -------\n        float\n            Performance Index value.\n        \"\"\"\n        return self.pearson_r * self.wi\n\n    @computed_field_cached_property()\n    def pi_rating(self) -> str:\n        \"\"\"Classify model performance based on Performance Index (PI).\n\n        Returns a qualitative rating of the model performance based on\n        the Performance Index value according to established thresholds.\n\n        Returns\n        -------\n        str\n            Performance rating: 'excellent', 'very good', 'good', 'satisfactory',\n            'poor', 'bad', or 'very bad'.\n        \"\"\"\n        pi = self.pi\n\n        if pi >= 0.85:\n            return \"excellent\"\n        elif pi >= 0.75:\n            return \"very good\"\n        elif pi >= 0.65:\n            return \"good\"\n        elif pi >= 0.60:\n            return \"satisfactory\"\n        elif pi >= 0.50:\n            return \"poor\"\n        elif pi >= 0.40:\n            return \"bad\"\n        else:\n            return \"very bad\"\n        \n    @computed_field_cached_property()\n    def explained_variance_score(self) -> float:\n        \"\"\"Calculate the explained variance score.\n\n        Measures the proportion of variance in the observed data that is\n        explained by the model predictions. Ranges from -∞ to 1, with 1\n        indicating perfect prediction and 0 indicating no explanatory power.\n\n        Returns\n        -------\n        float\n            Explained variance score.\n        \"\"\"\n        num = self.residuals.variance\n        den = self.observed.variance\n\n        return 1 - safe_divide(num, den, _MIN_DENOMINATOR)\n    \n\ndef BaselineMetricsFromDict(input_dict: dict) -> BaselineMetrics:\n    \"\"\"Construct a BaselineMetrics instance from a dictionary.\n\n    Parameters\n    ----------\n    input_dict : dict\n        Dictionary containing BaselineMetrics data, with optional nested\n        ColumnMetrics data for 'observed', 'predicted', and 'residuals' keys.\n\n    Returns\n    -------\n    BaselineMetrics\n        Constructed BaselineMetrics instance.\n    \"\"\"\n    for k in [\"observed\", \"predicted\", \"residuals\"]:\n        if k in input_dict:\n            input_dict[k] = PydanticFromDict(input_dict[k], name=\"ColumnMetrics\")\n\n    return PydanticFromDict(input_dict, name=\"BaselineMetrics\")\n\n\nclass ModelChoice(str, Enum):\n    \"\"\"Data frequency choices for baseline models.\n\n    Determines the time granularity of the baseline model, which affects\n    uncertainty calculations in ASHRAE Guideline 14 methodology.\n\n    Attributes\n    ----------\n    HOURLY : str\n        Hourly data frequency.\n    HOURLYSOLAR : str\n        Hourly solar data frequency (mapped to \"hourly\").\n    DAILY : str\n        Daily data frequency.\n    BILLING : str\n        Billing period data frequency.\n    \"\"\"\n    HOURLY = \"hourly\"\n    HOURLYSOLAR = \"hourly\"\n    DAILY = \"daily\"\n    BILLING = \"billing\"\n\n\nclass ReportingMetrics(pydantic.BaseModel):\n    \"\"\"Reporting period metrics for energy savings calculations.\n\n    Calculates savings, uncertainty, and fractional savings uncertainty (FSU)\n    for a reporting period based on baseline model metrics and reporting data.\n    Follows ASHRAE Guideline 14 methodology.\n    \"\"\"\n    model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)\n\n    baseline_metrics: Union[BaselineMetrics, pydantic.BaseModel] = pydantic.Field(\n        exclude=True,\n        description=\"Baseline model metrics instance\",\n    )\n\n    reporting_df: pd.DataFrame = pydantic.Field(\n        exclude=True,\n        description=\"Reporting period dataframe with 'observed' and 'predicted' columns\",\n    )\n\n    data_frequency: ModelChoice = pydantic.Field(\n        exclude=False,\n        description=\"Data frequency of the model (hourly, daily, or billing)\",\n    )\n\n    confidence_level: float = pydantic.Field(\n        ge=0.0,\n        le=1.0,\n        default=0.90,\n        validate_default=True,\n        description=\"Confidence level for uncertainty calculations\",\n    )\n\n    t_tail: int = pydantic.Field(\n        ge=1,\n        le=2,\n        default=2,\n        validate_default=True,\n        description=\"Number of tails for hypothesis testing (1 or 2)\",\n    )\n\n    @property\n    def _baseline(self) -> BaselineMetrics:\n        \"\"\"Convenience property to access baseline metrics.\"\"\"\n        return self.baseline_metrics\n\n    @cached_property\n    def _df(self) -> pd.DataFrame:\n        \"\"\"Prepare and validate the reporting period dataframe.\n\n        Validates column types and filters non-finite values.\n\n        Returns\n        -------\n        pd.DataFrame\n            Processed dataframe with 'observed' and 'predicted' columns.\n\n        Raises\n        ------\n        ValueError\n            If reporting dataframe is empty.\n        \"\"\"\n        _df = self.reporting_df[[\"observed\", \"predicted\"]].copy()\n\n        if len(_df) < 1:\n            raise ValueError(\"Input dataframe must have at least one row\")\n\n        # Check dataframe\n        expected_columns = {\"observed\": \"float\", \"predicted\": \"float\"}\n        _df = PydanticDf(df=_df, column_types=expected_columns).df\n\n        # drop non finite values from df\n        _df = _df[np.isfinite(_df[\"observed\"]) & np.isfinite(_df[\"predicted\"])]\n\n        return _df\n\n    @computed_field_cached_property()\n    def n(self) -> float:\n        \"\"\"Calculate the number of observations in the reporting period.\n\n        Returns the count of valid observations after filtering non-finite values.\n\n        Returns\n        -------\n        float\n            Number of observations.\n        \"\"\"\n        return len(self._df)\n\n    @computed_field_cached_property()\n    def observed_sum(self) -> float:\n        \"\"\"Calculate total observed energy consumption.\n\n        Sum of all observed values in the reporting period.\n\n        Returns\n        -------\n        float\n            Total observed energy.\n        \"\"\"\n        return self._df[\"observed\"].sum()\n\n    @computed_field_cached_property()\n    def predicted_sum(self) -> float:\n        \"\"\"Calculate total predicted energy consumption.\n\n        Sum of all predicted values in the reporting period (baseline forecast).\n\n        Returns\n        -------\n        float\n            Total predicted energy.\n        \"\"\"\n        return self._df[\"predicted\"].sum()\n\n    @computed_field_cached_property()\n    def t_stat(self) -> float:\n        \"\"\"Calculate t-statistic for uncertainty calculations.\n\n        Returns the t-statistic value based on confidence level, degrees of\n        freedom, and number of tails for hypothesis testing.\n\n        Returns\n        -------\n        float\n            t-statistic value.\n        \"\"\"\n        return t_stat(1 - self.confidence_level, self._baseline.ddof, tail=self.t_tail)\n\n    @computed_field_cached_property()\n    def savings(self) -> float:\n        \"\"\"Calculate energy savings.\n\n        The difference between predicted (baseline) and observed energy\n        consumption. Positive values indicate energy savings.\n\n        Returns\n        -------\n        float\n            Energy savings.\n        \"\"\"\n        return self.predicted_sum - self.observed_sum\n\n    @computed_field_cached_property()\n    def total_savings_uncertainty(self) -> Optional[float]:\n        \"\"\"Calculate total savings uncertainty following ASHRAE Guideline 14.\n\n        Computes uncertainty in energy savings predictions accounting for\n        autocorrelation, sample size, and data frequency effects.\n\n        Returns\n        -------\n        Optional[float]\n            Total savings uncertainty, or None if calculation fails.\n        \"\"\"\n        E_reporting = self.predicted_sum\n        n = self._baseline.n\n        n_prime = self._baseline.n_prime\n        m = self.n\n        t = self.t_stat\n        cvrmse_adj = self._baseline.cvrmse_adj\n\n        # Approximation factor from ASHRAE Guideline 14\n        n_ratio = safe_divide(n, n_prime, _MIN_DENOMINATOR)\n        n_prime_term = safe_divide(2.0, n_prime, _MIN_DENOMINATOR)\n        approx_factor = np.sqrt(n_ratio * (1 + n_prime_term) * m)\n\n        try:\n            e_per_m = safe_divide(E_reporting, m, _MIN_DENOMINATOR)\n            s_unc_base = np.abs(e_per_m * cvrmse_adj) * t * approx_factor\n        except (ZeroDivisionError, FloatingPointError, ValueError):\n            return None\n\n        if self.data_frequency == \"hourly\":\n            # ASHRAE 14 hourly data correction factor\n            s_unc = 1.26 * s_unc_base\n\n        elif self.data_frequency in [\"daily\", \"billing\"]:\n            M = len(self._df.index.month.unique())\n\n            # Sun & Baltazar 2013 polynomial corrections\n            if self.data_frequency == \"daily\":\n                coefs = [-0.00024, 0.03535, 1.00286]\n            else:\n                coefs = [-0.00022, 0.03306, 0.94054]\n\n            s_unc = np.polyval(coefs, M) * s_unc_base\n\n        else:\n            raise ValueError(\"model_type must be 'hourly', 'daily', or 'billing'\")\n\n        return s_unc\n\n    @computed_field_cached_property()\n    def fsu(self) -> float:\n        \"\"\"Calculate Fractional Savings Uncertainty (FSU).\n\n        The ratio of total savings uncertainty to actual savings, expressed\n        as a fraction. Used to assess the reliability of savings estimates.\n\n        Returns\n        -------\n        float\n            Fractional savings uncertainty.\n        \"\"\"\n        return safe_divide(self.total_savings_uncertainty, self.savings, _MIN_DENOMINATOR)\n\n    @computed_field_cached_property()\n    def predicted_data_point_unc(self) -> Optional[float]:\n        \"\"\"Calculate uncertainty per predicted data point.\n\n        Normalizes total savings uncertainty by the square root of the number\n        of reporting period observations.\n\n        Returns\n        -------\n        Optional[float]\n            Per-point uncertainty, or None if total uncertainty cannot be calculated.\n        \"\"\"\n        if self.total_savings_uncertainty is None:\n            return None\n\n        return self.total_savings_uncertainty / np.sqrt(self.n)\n    \n\nclass AutocorrelationMethod(Enum):\n    \"\"\"Methods for computing autocorrelation function.\n\n    Attributes\n    ----------\n    MOVING_STATS : str\n        Compute mean and standard deviation in a rolling window.\n    STATIONARY_CORRELATE : str\n        Compute over entire series using correlate.\n    STATIONARY_STATS_FFT : str\n        Compute over entire series using FFT for efficiency.\n    \"\"\"\n    MOVING_STATS = \"moving_stats\"\n    STATIONARY_CORRELATE = \"stationary_correlate\"\n    STATIONARY_STATS_FFT = \"stationary_stats_fft\"\n\n\ndef acf(\n    x: np.ndarray,\n    lag_n: Optional[int] = None,\n    ac_type: AutocorrelationMethod = AutocorrelationMethod.MOVING_STATS\n) -> np.ndarray:\n    \"\"\"Compute the autocorrelation function (ACF) of a time series.\n\n    The ACF measures the correlation of a signal with a delayed copy of itself\n    as a function of delay. It helps identify repeating patterns, periodic signals\n    obscured by noise, or missing fundamental frequencies implied by harmonics.\n\n    Parameters\n    ----------\n    x : np.ndarray\n        The time series data.\n    lag_n : int, optional\n        The number of lags to compute the ACF for. If None, computes the ACF\n        for all possible lags.\n    ac_type : AutocorrelationMethod, optional\n        Method to compute the ACF. Default is MOVING_STATS.\n\n    Returns\n    -------\n    np.ndarray\n        The autocorrelation function values for the given time series and lags.\n    \"\"\"\n    if isinstance(ac_type, AutocorrelationMethod):\n        ac_type = ac_type.value\n\n    if lag_n is None:\n        lags = range(len(x) - 1)\n    else:\n        lags = range(lag_n + 1)\n\n    if ac_type == AutocorrelationMethod.MOVING_STATS.value:\n        # mean and std are computed in a rolling window\n        corr = [1.0 if l == 0 else np.corrcoef(x[l:], x[:-l])[0][1] for l in lags]\n        corr = np.array(corr)\n\n    elif \"stationary\" in ac_type:\n        # mean and std are computed over the entire series\n        n = len(x)\n        mean = x.mean()\n        var = np.var(x)\n        xc = x - mean\n\n        if ac_type == AutocorrelationMethod.STATIONARY_CORRELATE.value:\n            corr = np.correlate(xc, xc, \"full\")[(n - 1):] / var / n\n\n        elif ac_type == AutocorrelationMethod.STATIONARY_STATS_FFT.value:\n            cf = np.fft.fft(xc)\n            sf = cf.conjugate() * cf\n            corr = np.fft.ifft(sf).real / var / len(x)\n\n        corr = corr[:len(lags)]\n\n    return corr"
  },
  {
    "path": "opendsm/common/pydantic_utils.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport pydantic\nimport math\n\nfrom typing import Any, Optional\n\nfrom functools import cached_property  # TODO: This requires Python 3.8\n\n\nclass PydanticDf(pydantic.BaseModel):\n    model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)\n\n    df: pd.DataFrame\n\n    \"\"\"list of required column types\"\"\"\n    column_types: Optional[dict[str, Any]] = None\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_columns(self):\n        if self.column_types is not None:\n            expected_columns = list(self.column_types.keys())\n            if not set(self.df.columns) == set(expected_columns):\n                raise ValueError(\n                    f\"Expected columns {expected_columns} but got {self.df.columns}\"\n                )\n\n            for col, col_type in self.column_types.items():\n                if col_type is None or col_type is Any:\n                    continue\n\n                if self.df[col].dtype != col_type:\n                    # attempt to coerce numeric columns\n                    if np.issubdtype(col_type, np.number) and np.issubdtype(\n                        self.df[col].dtype, np.number\n                    ):\n                        self.df[col] = self.df[col].astype(col_type)\n                    else:\n                        raise ValueError(\n                            f\"Expected column {col} to be of type {col_type} but got {self.df[col].dtype}\"\n                        )\n        return self\n\n\nclass ArbitraryPydanticModel(pydantic.BaseModel):\n    model_config = pydantic.ConfigDict(arbitrary_types_allowed=True, extra=\"allow\")\n\n    @pydantic.model_serializer(mode=\"wrap\")\n    def _serialize_special_floats(self, serializer, info):\n        \"\"\"Custom serializer to handle nan, inf, -inf values.\"\"\"\n        data = serializer(self)\n        # Only convert to strings when serializing to JSON (mode='json')\n        # For Python dicts (mode='python'), keep native float('nan') values\n        if info.mode == 'json':\n            return self._convert_special_floats_to_str(data)\n        return data\n\n    @staticmethod\n    def _convert_special_floats_to_str(obj):\n        \"\"\"Recursively convert nan, inf, -inf to string representations.\"\"\"\n        if isinstance(obj, float):\n            if math.isnan(obj):\n                return \"nan\"\n            elif math.isinf(obj):\n                return \"inf\" if obj > 0 else \"-inf\"\n        elif isinstance(obj, dict):\n            return {k: ArbitraryPydanticModel._convert_special_floats_to_str(v) for k, v in obj.items()}\n        elif isinstance(obj, (list, tuple)):\n            return type(obj)(ArbitraryPydanticModel._convert_special_floats_to_str(item) for item in obj)\n        return obj\n\n    @pydantic.model_validator(mode=\"before\")\n    @classmethod\n    def _parse_special_floats(cls, data):\n        \"\"\"Custom validator to parse string representations back to nan, inf, -inf.\"\"\"\n        if isinstance(data, dict):\n            return cls._convert_str_to_special_floats(data)\n        elif isinstance(data, (list, tuple)):\n            return cls._convert_str_to_special_floats(data)\n        return data\n\n    @staticmethod\n    def _convert_str_to_special_floats(obj):\n        \"\"\"Recursively convert string representations to nan, inf, -inf.\"\"\"\n        if isinstance(obj, str):\n            if obj == \"nan\":\n                return float(\"nan\")\n            elif obj == \"inf\":\n                return float(\"inf\")\n            elif obj == \"-inf\":\n                return float(\"-inf\")\n        elif isinstance(obj, dict):\n            return {k: ArbitraryPydanticModel._convert_str_to_special_floats(v) for k, v in obj.items()}\n        elif isinstance(obj, list):\n            return [ArbitraryPydanticModel._convert_str_to_special_floats(item) for item in obj]\n        elif isinstance(obj, tuple):\n            return tuple(ArbitraryPydanticModel._convert_str_to_special_floats(item) for item in obj)\n        return obj\n\n\ndef PydanticFromDict(input_dict, name=\"PydanticModel\"):\n    \"\"\"Creates a Pydantic model from a dictionary.\n\n    Args:\n        input_dict (dictionary): Dictionary and values to be used to create the Pydantic model.\n        name (str, optional): Name of the Pydantic model. Defaults to \"PydanticModel\".\n\n    Returns:\n        Pydantic.BaseModel: Instantiated Pydantic model from input dictionary.\n    \"\"\"\n\n    model = pydantic.create_model(\n        name,\n        **{name: (type(value), ...) for name, value in input_dict.items()},\n        __base__=ArbitraryPydanticModel,\n    )\n\n    return model(**input_dict)\n\n\ndef computed_field_cached_property():\n    decs = [pydantic.computed_field, cached_property]\n\n    def deco(f):\n        for dec in reversed(decs):\n            f = dec(f)\n        return f\n\n    return deco"
  },
  {
    "path": "opendsm/common/stats/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License."
  },
  {
    "path": "opendsm/common/stats/adaptive_loss.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom typing import Optional, Tuple, Union\n\nimport numba\nimport numpy as np\nfrom scipy.optimize import minimize_scalar\n\nfrom opendsm.common.stats.adaptive_loss_Z import ln_Z\nfrom opendsm.common.stats.outliers import (\n    remove_outliers,\n    _IQR_outlier,\n)\nfrom opendsm.common.stats.basic import _median_absolute_deviation\nfrom opendsm.common.utils import OoM_numba\n\n# Loss function constants\nLOSS_ALPHA_MIN = -100.0\nLOSS_ALPHA_MAX = 100.0\n\n\n@numba.jit(nopython=True, cache=True)\ndef sliding_window(\n    arr: np.ndarray, \n    window_size: int, \n    step: int = 0,\n) -> np.ndarray:\n    \"\"\"Create sliding windows over a time series array.\n\n    Reference: https://giov.dev/2018/05/a-window-on-numpy-s-views.html\n\n    Args:\n        arr: Input array with time advancing along dimension 0\n        window_size: Size of sliding window\n        step: Step size for sliding window. If 0, uses non-overlapping\n            contiguous windows (step=window_size)\n\n    Returns:\n        Array with windowed views of the input data\n\n    Raises:\n        ValueError: If window_size > array size or step < 0\n    \"\"\"\n    n_obs = arr.shape[0]\n\n    # validate arguments\n    if window_size > n_obs:\n        raise ValueError(\n            \"Window size must be less than or equal \"\n            \"the size of array in first dimension.\"\n        )\n    if step < 0:\n        raise ValueError(\"Step must be positive.\")\n\n    n_windows = 1 + int(np.floor((n_obs - window_size) / step))\n\n    obs_stride = arr.strides[0]\n    windowed_row_stride = obs_stride * step\n\n    new_shape = (n_windows, window_size) + arr.shape[1:]\n    new_strides = (windowed_row_stride,) + arr.strides\n\n    strided = np.lib.stride_tricks.as_strided(\n        arr,\n        shape=new_shape,\n        strides=new_strides,\n    )\n    return strided\n\n\ndef rolling_IQR_outlier(\n    x: np.ndarray,\n    y: np.ndarray,\n    sigma_threshold: float = 3.0,\n    quantile: float = 0.25,\n    window: Union[float, int] = 0.05,\n    step: float = 1.0,\n) -> np.ndarray:\n    \"\"\"Calculate rolling outlier thresholds using IQR method.\n\n    Args:\n        x: X-values of the data (e.g., time)\n        y: Y-values of the data (residuals or observations)\n        sigma_threshold: Sigma threshold for IQR outlier detection\n        quantile: Quantile for IQR calculation (0.25 for standard IQR)\n        window: Window size. If <= 1, treated as proportion of data length\n        step: Step size for rolling calculation (as proportion of window if < 1)\n\n    Returns:\n        2D array with shape (2, len(x)) where row 0 is lower threshold\n        and row 1 is upper threshold\n    \"\"\"\n\n    if window <= 1.0:\n        window = int(np.floor(len(y) * window))\n    else:\n        window = int(window)\n\n    step = int(np.floor(window * step))\n    if step < 1:\n        step = 1\n\n    y = np.abs(y)\n\n    x_windows = sliding_window(x, window, step=step)\n    y_windows = sliding_window(y, window, step=step)\n\n    # Vectorized computation of means and quantiles\n    x_interp = np.mean(x_windows, axis=1)\n    q13 = np.quantile(y_windows, [quantile, 1 - quantile], axis=1)\n    q1 = q13[0]\n    q3 = q13[1]\n\n    # Empirical scaling factor to convert sigma threshold to IQR multiplier\n    q13_scalar = 0.7413 * sigma_threshold - 0.5\n    iqr = (q3 - q1) * q13_scalar\n    outlier_bnds = [q1 - iqr, q3 + iqr]\n\n    outlier_threshold = np.zeros((2, len(x)))\n    outlier_threshold[0] = np.interp(x, x_interp, outlier_bnds[0])\n    outlier_threshold[1] = np.interp(x, x_interp, outlier_bnds[1])\n\n    # x_interp = np.arange(0, len(outlier_bnds[0]))\n    # x_orig = np.linspace(0, len(outlier_bnds[0]), len(x))\n\n    # outlier_threshold = np.zeros((2, len(x)))\n    # outlier_threshold[0] = np.interp(x_orig, x_interp, outlier_bnds[0])\n    # outlier_threshold[1] = np.interp(x_orig, x_interp, outlier_bnds[1])\n\n    return outlier_threshold\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef get_C(\n    resid: np.ndarray,\n    mu: float,\n    sigma: float,\n    quantile: float = 0.25,\n    algo: str = \"iqr_legacy\",\n) -> float:\n    \"\"\"Calculate scale parameter C for adaptive loss weighting.\n\n    Computes a robust scale estimate using various methods to normalize\n    residuals for adaptive loss functions.\n\n    Args:\n        resid: Residuals from model fit\n        mu: Location parameter (typically median of residuals)\n        sigma: Scale factor (typically sigma threshold for outliers)\n        quantile: Quantile for IQR calculation (0.25 for standard IQR)\n        algo: Algorithm to use - 'iqr_legacy', 'iqr', 'mad', or 'stdev'\n\n    Returns:\n        Scale parameter C for normalizing residuals\n    \"\"\"\n    # remove non-finite values\n    resid = resid[np.isfinite(resid)]\n\n    if algo == \"iqr_legacy\":\n        # TODO: uncertain if these C functions should use np.min, np.mean, or np.max\n        # suspect we can switch to IQR below, but need to test\n        bounds = _IQR_outlier(\n            resid - mu, weights=None, sigma_threshold=sigma, quantile=quantile\n        )\n        C = np.max(np.abs(bounds))\n\n    elif algo == \"iqr\":\n        resid = np.abs(resid)\n\n        bounds = _IQR_outlier(\n            resid - mu, weights=None, sigma_threshold=sigma, quantile=quantile\n        )\n        C = np.max(np.abs(bounds))\n\n    elif algo == \"mad\":\n        C = sigma * _median_absolute_deviation(resid, median=None, weights=None)\n\n    elif algo == \"stdev\":\n        C = sigma * np.std(resid)\n\n    if C == 0:\n        C = OoM_numba(np.array([C]), method=\"floor\")[0]\n\n    return C\n\n\ndef rolling_C(\n    T: np.ndarray,\n    resid: np.ndarray,\n    mu: float,\n    sigma: float = 3.0,\n    quantile: float = 0.25,\n    window: Union[float, int] = 0.2,\n    step: float = 1.0,\n) -> np.ndarray:\n    \"\"\"Calculate rolling scale parameter C for adaptive loss weighting.\n\n    Args:\n        T: Time or x-axis values\n        resid: Residuals from model fit\n        mu: Location parameter (typically median)\n        sigma: Sigma threshold for outlier detection\n        quantile: Quantile for IQR calculation (0.25 for standard IQR)\n        window: Window size (proportion if <= 1, absolute if > 1)\n        step: Step size for rolling calculation\n\n    Returns:\n        Array of rolling C values\n    \"\"\"\n    q13 = rolling_IQR_outlier(T, resid - mu, sigma, quantile, window, step)\n    C = np.max(np.abs(q13), axis=0)\n\n    return C\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef generalized_loss_fcn(\n    x: Union[float, np.ndarray], \n    alpha: float = 2.0, \n    alpha_min: float = LOSS_ALPHA_MIN,\n) -> Union[float, np.ndarray]:\n    \"\"\"Calculate generalized loss function value.\n\n    Implements a family of robust loss functions parameterized by alpha.\n    Different alpha values correspond to different well-known loss functions.\n\n    Args:\n        x: Input value(s) - typically normalized residuals\n        alpha: Shape parameter determining loss function type\n        alpha_min: Minimum alpha value for Welsch/Leclerc loss\n\n    Returns:\n        Loss function value(s)\n\n    Loss function types by alpha value:\n        - alpha = 2.0: L2 (squared error) loss\n        - alpha = 1.0: Smoothed L1 (Pseudo-Huber) loss\n        - alpha = 0.0: Charbonnier loss\n        - alpha = -2.0: Cauchy/Lorentzian loss\n        - alpha <= alpha_min: Welsch/Leclerc loss\n        - other: Generalized Charbonnier loss\n    \"\"\"\n\n    # Defaults to sum of squared error\n    x_2 = x**2\n\n    if alpha == 2.0:  # L2\n        loss = 0.5 * x_2\n    elif alpha == 1.0:  # smoothed L1\n        loss = np.sqrt(x_2 + 1) - 1\n    elif alpha == 0.0:  # Charbonnier loss\n        loss = np.log(0.5 * x_2 + 1)\n    elif alpha == -2.0:  # Cauchy/Lorentzian loss\n        loss = 2 * x_2 / (x_2 + 4)\n    elif alpha <= alpha_min:  # at -infinity, Welsch/Leclerc loss\n        loss = 1 - np.exp(-0.5 * x_2)\n    else:\n        loss = np.abs(alpha - 2) / alpha * ((x_2 / np.abs(alpha - 2) + 1) ** (alpha / 2) - 1)\n\n    return loss\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef generalized_loss_derivative(\n    x: Union[float, np.ndarray], \n    scale: float = 1.0, \n    alpha: float = 2.0,\n) -> Union[float, np.ndarray]:\n    \"\"\"Calculate derivative of generalized loss function.\n\n    Computes the gradient of the loss function with respect to the input,\n    accounting for the scale parameter.\n\n    Args:\n        x: Input value(s) - typically residuals\n        scale: Scale parameter for normalization\n        alpha: Shape parameter determining loss function type\n\n    Returns:\n        Derivative of loss function with respect to x\n\n    Loss function types by alpha value:\n        - alpha = 2.0: L2 loss\n        - alpha = 1.0: Smoothed L1 (Pseudo-Huber) loss\n        - alpha = 0.0: Charbonnier loss\n        - alpha = -2.0: Cauchy/Lorentzian loss\n        - alpha <= LOSS_ALPHA_MIN: Welsch/Leclerc loss\n        - other: Generalized loss\n    \"\"\"\n    if alpha == 2.0:  # L2\n        dloss_dx = x / scale**2\n    elif alpha == 1.0:  # smoothed L1\n        dloss_dx = x / scale**2 / np.sqrt((x / scale) ** 2 + 1)\n    elif alpha == 0.0:  # Charbonnier loss\n        dloss_dx = 2 * x / (x**2 + 2 * scale**2)\n    elif alpha == -2.0:  # Cauchy/Lorentzian loss\n        dloss_dx = 16 * scale**2 * x / (4 * scale**2 + x**2) ** 2\n    elif alpha <= LOSS_ALPHA_MIN:  # at -infinity, Welsch/Leclerc loss\n        dloss_dx = x / scale**2 * np.exp(-0.5 * (x / scale) ** 2)\n    else:\n        dloss_dx = x / scale**2 * ((x / scale) ** 2 / np.abs(alpha - 2) + 1)\n\n    return dloss_dx\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef generalized_loss_weights(\n    x: np.ndarray, \n    alpha: float = 2.0, \n    min_weight: float = 0.0,\n) -> np.ndarray:\n    \"\"\"Calculate adaptive weights based on generalized loss function.\n\n    Computes observation weights that downweight outliers according to\n    the loss function shape parameter alpha.\n\n    Args:\n        x: Normalized residuals (typically (residuals - mu) / scale)\n        alpha: Shape parameter determining weight behavior\n        min_weight: Minimum weight value (prevents complete downweighting)\n\n    Returns:\n        Array of weights in range [min_weight, 1.0]\n    \"\"\"\n\n    dtype = numba.float64\n    if numba.config.DISABLE_JIT:\n        dtype = np.float64\n\n    # Vectorized computation\n    x_sq = x**2\n    w = np.ones(len(x), dtype=dtype)\n\n    if alpha == 2.0:\n        # L2 loss: all weights are 1.0\n        pass\n    elif alpha == 0.0:\n        # Charbonnier loss\n        w = np.where(x > 0, 1.0 / (0.5 * x_sq + 1.0), 1.0)\n    elif alpha <= LOSS_ALPHA_MIN:\n        # Welsch/Leclerc loss\n        w = np.where(x > 0, np.exp(-0.5 * x_sq), 1.0)\n    else:\n        # Generalized loss\n        w = np.where(x > 0, (x_sq / np.abs(alpha - 2) + 1) ** (0.5 * alpha - 1), 1.0)\n\n    return w * (1.0 - min_weight) + min_weight\n\n\ndef penalized_loss_fcn(\n    x: np.ndarray, \n    alpha: float = 2.0, \n    use_penalty: bool = True,\n) -> np.ndarray:\n    \"\"\"Calculate penalized loss function with partition function penalty.\n\n    Adds a penalty term based on the approximate partition function to\n    penalize more complex loss functions (lower alpha values).\n\n    Args:\n        x: Normalized input values (typically residuals)\n        alpha: Shape parameter for loss function\n        use_penalty: Whether to include partition function penalty\n\n    Returns:\n        Penalized loss values\n\n    Raises:\n        Exception: If non-finite values are found in calculated loss\n    \"\"\"\n    loss = generalized_loss_fcn(x, alpha=alpha)\n\n    if use_penalty:\n        # Approximate partition function penalty for C=1, tau=10\n        penalty = ln_Z(alpha, LOSS_ALPHA_MIN)\n        loss += penalty\n\n        if not np.isfinite(loss).all():\n            # print(\"alpha: \", alpha)\n            # print(\"x: \", x)\n            # print(\"penalty: \", penalty)\n            raise Exception(\"non-finite values in 'penalized_loss_fcn'\")\n\n    return loss\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef alpha_scaled(\n    s: float, \n    alpha_max: float = 2.0,\n) -> float:\n    \"\"\"Convert scaled parameter s to alpha value.\n\n    Transforms a bounded input s to the alpha parameter space using\n    nonlinear scaling to provide smooth optimization behavior.\n\n    Args:\n        s: Scaled input value (typically in [0, 1] for optimization)\n        alpha_max: Maximum alpha value (determines scaling method)\n\n    Returns:\n        Alpha value in range [LOSS_ALPHA_MIN, alpha_max] (approximately)\n    \"\"\"\n    if alpha_max == 2.0:\n        a = 3\n        b = 0.25\n\n        # Clip s to valid range\n        if s < 0:\n            s = 0\n        if s > 1:\n            s = 1\n\n        # Nonlinear scaling using power law\n        s_max = 1 - 2 / (1 + 10**a)\n        s = (1 - 2 / (1 + 10 ** (a * s**b))) / s_max\n\n        alpha = LOSS_ALPHA_MIN + (2.0 - LOSS_ALPHA_MIN) * s\n\n    else:\n        # Alternative scaling using logistic function\n        x0 = 1.0\n        k = 1.5\n\n        if s >= 1:\n            return LOSS_ALPHA_MAX\n        elif s <= 0:\n            return LOSS_ALPHA_MIN\n\n        A = (np.exp((LOSS_ALPHA_MAX - x0) / k) + 1) / (\n            1 - np.exp(2 * LOSS_ALPHA_MAX / k)\n        )\n        K = (1 - A) * np.exp((x0 - LOSS_ALPHA_MAX) / k) + 1\n\n        alpha = x0 - k * np.log((K - A) / (s - A) - 1)\n\n    return alpha\n\n\ndef adaptive_loss_fcn(\n    x: np.ndarray,\n    mu: float = 0.0,\n    scale: float = 1.0,\n    alpha: Union[str, float] = \"adaptive\",\n    replace_nonfinite: bool = True,\n) -> Tuple[float, float]:\n    \"\"\"Calculate adaptive loss function and optimal alpha parameter.\n\n    Computes the total loss and optionally optimizes the alpha parameter\n    to minimize the penalized loss function.\n\n    Args:\n        x: Input residuals\n        mu: Location parameter for normalization\n        scale: Scale parameter for normalization\n        alpha: Shape parameter ('adaptive' for optimization, or fixed value)\n        replace_nonfinite: Replace non-finite values with max finite value\n\n    Returns:\n        Tuple of (total loss value, alpha parameter used)\n    \"\"\"\n    # Standardize residuals if needed\n    if np.all(mu != 0.0) or np.all(scale != 1.0):\n        x = (x - mu) / scale\n\n    if replace_nonfinite:\n        x[~np.isfinite(x)] = np.max(x[np.isfinite(x)])\n\n    def _loss_for_alpha(alpha_val: float) -> float:\n        \"\"\"Compute total penalized loss for given alpha.\"\"\"\n        return penalized_loss_fcn(x, alpha=alpha_val, use_penalty=True).sum()\n\n    if alpha == \"adaptive\":\n        # Optimize alpha parameter over scaled space\n        res = minimize_scalar(\n            lambda s: _loss_for_alpha(alpha_scaled(s)),\n            bounds=[-1e-5, 1 + 1e-5],\n            method=\"Bounded\",\n            options={\"xatol\": 1e-5},\n        )\n        loss_alpha = alpha_scaled(res.x)\n        # res = minimize(lambda s: _loss_for_alpha(alpha_scaled(s[0])), x0=[0.7], bounds=[[0, 1]], method=\"L-BFGS-B\")\n        # loss_alpha = alpha_scaled(res.x[0])\n        loss_fcn_val = res.fun\n    else:\n        loss_alpha = alpha\n        loss_fcn_val = _loss_for_alpha(alpha)\n\n    return loss_fcn_val, loss_alpha\n\n\ndef adaptive_weights(\n    x: np.ndarray,\n    alpha: Union[str, float] = \"adaptive\",\n    sigma: float = 3.0,\n    quantile: float = 0.25,\n    min_weight: float = 0.0,\n    C_algo: str = \"iqr_legacy\",\n    replace_nonfinite: bool = True,\n) -> Tuple[np.ndarray, float, float]:\n    \"\"\"Calculate adaptive weights for robust regression.\n\n    Computes observation weights that downweight outliers based on\n    the adaptive loss function. The scale and alpha parameters are\n    automatically determined from the data.\n\n    Args:\n        x: Input residuals (not standardized)\n        alpha: Shape parameter ('adaptive' for optimization, or fixed value)\n        sigma: Sigma threshold for outlier detection\n        quantile: Quantile for IQR calculation (0.25 for standard IQR)\n        min_weight: Minimum weight value (prevents complete downweighting)\n        C_algo: Algorithm for scale estimation ('iqr_legacy', 'iqr', 'mad', 'stdev')\n        replace_nonfinite: Replace non-finite values with max finite value\n\n    Returns:\n        Tuple of (weights array, scale parameter C, alpha parameter)\n    \"\"\"\n    x_no_outlier, _ = remove_outliers(x, sigma_threshold=sigma, quantile=0.25)\n\n    # TODO: Should x be abs or not?\n    # likely should be abs\n    # mu = np.median(np.abs(x_no_outlier))\n    mu = np.median(x_no_outlier)\n\n    C = get_C(x, mu, sigma, quantile, C_algo)\n    x_normalized = (x - mu) / C\n\n    if alpha == \"adaptive\":\n        _, alpha = adaptive_loss_fcn(\n            x_normalized, alpha=alpha, replace_nonfinite=replace_nonfinite\n        )\n\n    return generalized_loss_weights(x_normalized, alpha=alpha, min_weight=min_weight), C, alpha"
  },
  {
    "path": "opendsm/common/stats/adaptive_loss_Z.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nfrom scipy.interpolate import BSpline\n\n# fmt: off\nTCK = (\n    np.array(\n        [-9.99999800e+01, -9.99999800e+01, -9.99999800e+01, -9.99999800e+01,\n         -9.99999800e+01, -9.99999800e+01, -9.91619068e+01, -9.83266402e+01,\n         -9.74941846e+01, -9.66645407e+01, -9.58377122e+01, -9.50137126e+01,\n         -9.41925408e+01, -9.33741970e+01, -9.25586971e+01, -9.17460351e+01,\n         -9.09362253e+01, -9.01292653e+01, -8.93251652e+01, -8.85239253e+01,\n         -8.77255539e+01, -8.69300586e+01, -8.61374414e+01, -8.53477099e+01,\n         -8.45608629e+01, -8.37769094e+01, -8.29958600e+01, -8.22177146e+01,\n         -8.14424793e+01, -8.06701612e+01, -7.99007612e+01, -7.91342906e+01,\n         -7.83707533e+01, -7.76101558e+01, -7.68524971e+01, -7.60977931e+01,\n         -7.53460405e+01, -7.45972544e+01, -7.38514297e+01, -7.31085828e+01,\n         -7.23687129e+01, -7.16318305e+01, -7.08979361e+01, -7.01670372e+01,\n         -6.94391472e+01, -6.87142636e+01, -6.79923997e+01, -6.72735535e+01,\n         -6.65577345e+01, -6.58449533e+01, -6.51352131e+01, -6.44285222e+01,\n         -6.37248873e+01, -6.30243066e+01, -6.23268027e+01, -6.16323670e+01,\n         -6.09410147e+01, -6.02527504e+01, -5.95675831e+01, -5.88855131e+01,\n         -5.82065575e+01, -5.75307138e+01, -5.68579983e+01, -5.61884124e+01,\n         -5.55219639e+01, -5.48586609e+01, -5.41985122e+01, -5.35415210e+01,\n         -5.28877003e+01, -5.22370562e+01, -5.15895993e+01, -5.09453269e+01,\n         -5.03042593e+01, -4.96663975e+01, -4.90317486e+01, -4.84003254e+01,\n         -4.77721368e+01, -4.71471841e+01, -4.65254823e+01, -4.59070388e+01,\n         -4.52918565e+01, -4.46799529e+01, -4.40713312e+01, -4.34660006e+01,\n         -4.28639755e+01, -4.22652588e+01, -4.16698569e+01, -4.10777887e+01,\n         -4.04890524e+01, -3.99036691e+01, -3.93216401e+01, -3.87429783e+01,\n         -3.81676957e+01, -3.75957956e+01, -3.70272924e+01, -3.64621965e+01,\n         -3.59005176e+01, -3.53422672e+01, -3.47874533e+01, -3.42360858e+01,\n         -3.36881813e+01, -3.31437432e+01, -3.26027860e+01, -3.20653230e+01,\n         -3.15313647e+01, -3.10009221e+01, -3.04740010e+01, -2.99506229e+01,\n         -2.94307969e+01, -2.89145283e+01, -2.84018381e+01, -2.78927378e+01,\n         -2.73872312e+01, -2.68853430e+01, -2.63870773e+01, -2.58924510e+01,\n         -2.54014782e+01, -2.49141706e+01, -2.44305415e+01, -2.39506055e+01,\n         -2.34743794e+01, -2.30018754e+01, -2.25331069e+01, -2.20680896e+01,\n         -2.16068408e+01, -2.11493713e+01, -2.06956992e+01, -2.02458406e+01,\n         -1.97998137e+01, -1.93576289e+01, -1.89193063e+01, -1.84848656e+01,\n         -1.80543172e+01, -1.76276847e+01, -1.72049815e+01, -1.67862257e+01,\n         -1.63714383e+01, -1.59606383e+01, -1.55538454e+01, -1.51510716e+01,\n         -1.47523449e+01, -1.43576817e+01, -1.39671004e+01, -1.35806287e+01,\n         -1.31982778e+01, -1.28200789e+01, -1.24460463e+01, -1.20762057e+01,\n         -1.17105795e+01, -1.13491892e+01, -1.09920619e+01, -1.06392173e+01,\n         -1.02906780e+01, -9.94647806e+00, -9.60663360e+00, -9.27117058e+00,\n         -8.94011550e+00, -8.61349914e+00, -8.29134188e+00, -7.97367254e+00,\n         -7.66051449e+00, -7.35189906e+00, -7.04784859e+00, -6.74839097e+00,\n         -6.45355449e+00, -6.16336078e+00, -5.87783314e+00, -5.59700021e+00,\n         -5.32087490e+00, -5.04948077e+00, -4.78283065e+00, -4.52093336e+00,\n         -4.26378900e+00, -4.01139575e+00, -3.76373659e+00, -3.52077339e+00,\n         -3.28246606e+00, -3.04874699e+00, -2.81953592e+00, -2.59473887e+00,\n         -2.37425513e+00, -2.15802261e+00, -1.94608169e+00, -1.73866290e+00,\n         -1.53611957e+00, -1.33898610e+00, -1.14766835e+00, -9.62055927e-01,\n         -7.81266320e-01, -6.03322183e-01, -4.25221239e-01, -2.46824794e-01,\n         -7.76763660e-02,  7.54617469e-02,  2.17486985e-01,  3.53826824e-01,\n          4.89933140e-01,  6.34761751e-01,  7.89741708e-01,  9.38636540e-01,\n          1.09388055e+00,  1.23000133e+00,  1.33898962e+00,  1.43317887e+00,\n          1.51596478e+00,  1.58762073e+00,  1.65063047e+00,  1.70532890e+00,\n          1.75257216e+00,  1.79327671e+00,  1.82855195e+00,  1.85849916e+00,\n          1.88400303e+00,  1.90000960e+00,  1.90172739e+00,  1.90440391e+00,\n          1.92213219e+00,  1.93770949e+00,  1.95064293e+00,  1.96130851e+00,\n          1.97005891e+00,  1.97715673e+00,  1.98278630e+00,  1.98727574e+00,\n          1.99079451e+00,  1.99357901e+00,  1.99556189e+00,  1.99694659e+00,\n          1.99746232e+00,  1.99773606e+00,  1.99797487e+00,  1.99829915e+00,\n          1.99855288e+00,  1.99876116e+00,  1.99892488e+00,  1.99913056e+00,\n          1.99922343e+00,  1.99930634e+00,  1.99941449e+00,  1.99949065e+00,\n          1.99960643e+00,  1.99966948e+00,  1.99972133e+00,  1.99976616e+00,\n          1.99980879e+00,  1.99985830e+00,  1.99990106e+00,  1.99993097e+00,\n          1.99995254e+00,  1.99997109e+00,  1.99998582e+00,  2.00000352e+00,\n          2.00002031e+00,  2.00003968e+00,  2.00007094e+00,  2.00010168e+00,\n          2.00013595e+00,  2.00017135e+00,  2.00020381e+00,  2.00024097e+00,\n          2.00028403e+00,  2.00032807e+00,  2.00039608e+00,  2.00045393e+00,\n          2.00049877e+00,  2.00054776e+00,  2.00059183e+00,  2.00064085e+00,\n          2.00068659e+00,  2.00073395e+00,  2.00079058e+00,  2.00089693e+00,\n          2.00099802e+00,  2.00114617e+00,  2.00125120e+00,  2.00134046e+00,\n          2.00142858e+00,  2.00150875e+00,  2.00160096e+00,  2.00172427e+00,\n          2.00183041e+00,  2.00196187e+00,  2.00223323e+00,  2.00247477e+00,\n          2.00278542e+00,  2.00306566e+00,  2.00337256e+00,  2.00371468e+00,\n          2.00490935e+00,  2.00708025e+00,  2.00991650e+00,  2.01359578e+00,\n          2.01773492e+00,  2.02056213e+00,  2.02730471e+00,  2.03546465e+00,\n          2.04531408e+00,  2.05732220e+00,  2.07177821e+00,  2.08892863e+00,\n          2.10935079e+00,  2.13331863e+00,  2.16131658e+00,  2.19378209e+00,\n          2.23156110e+00,  2.27481045e+00,  2.32452319e+00,  2.38125418e+00,\n          2.44570900e+00,  2.51949658e+00,  2.60261083e+00,  2.69662353e+00,\n          2.80288640e+00,  2.92225114e+00,  3.05781839e+00,  3.20958028e+00,\n          3.38098345e+00,  3.57319716e+00,  3.78871648e+00,  4.02987694e+00,\n          4.29691296e+00,  4.59331070e+00,  4.91487791e+00,  5.26115258e+00,\n          5.62945458e+00,  6.01729356e+00,  6.42298855e+00,  6.84526884e+00,\n          7.28323899e+00,  7.73626562e+00,  8.20386272e+00,  8.68564408e+00,\n          9.18127472e+00,  9.69048564e+00,  1.02130375e+01,  1.07487232e+01,\n          1.12973390e+01,  1.18587095e+01,  1.24326704e+01,  1.30190710e+01,\n          1.36177612e+01,  1.42286006e+01,  1.48514663e+01,  1.54862274e+01,\n          1.61327737e+01,  1.67909888e+01,  1.74607586e+01,  1.81419907e+01,\n          1.88345746e+01,  1.95384173e+01,  2.02534327e+01,  2.09795231e+01,\n          2.17166067e+01,  2.24645942e+01,  2.32234140e+01,  2.39929758e+01,\n          2.47732099e+01,  2.55640466e+01,  2.63654087e+01,  2.71772207e+01,\n          2.79994267e+01,  2.88319499e+01,  2.96747324e+01,  3.05277064e+01,\n          3.13908146e+01,  3.22639955e+01,  3.31471892e+01,  3.40403413e+01,\n          3.49433956e+01,  3.58562966e+01,  3.67789887e+01,  3.77114269e+01,\n          3.86535543e+01,  3.96053168e+01,  4.05666749e+01,  4.15375779e+01,\n          4.25179747e+01,  4.35078202e+01,  4.45070773e+01,  4.55156882e+01,\n          4.65336222e+01,  4.75608253e+01,  4.85972646e+01,  4.96428974e+01,\n          5.06976770e+01,  5.17615737e+01,  5.28345391e+01,  5.39165383e+01,\n          5.50075334e+01,  5.61074933e+01,  5.72163692e+01,  5.83341361e+01,\n          5.94607534e+01,  6.05961909e+01,  6.17404082e+01,  6.28933719e+01,\n          6.40550589e+01,  6.52254252e+01,  6.64044422e+01,  6.75920782e+01,\n          6.87883009e+01,  6.99930845e+01,  7.12063926e+01,  7.24282003e+01,\n          7.36584741e+01,  7.48971867e+01,  7.61443064e+01,  7.73998087e+01,\n          7.86636653e+01,  7.99358482e+01,  8.12163330e+01,  8.25050862e+01,\n          8.38020842e+01,  8.51073021e+01,  8.64207117e+01,  8.77422900e+01,\n          8.90720150e+01,  9.04098555e+01,  9.17557859e+01,  9.31097879e+01,\n          9.44718340e+01,  9.58419042e+01,  9.72199674e+01,  9.86060051e+01,\n          1.00000000e+02,  1.00000000e+02,  1.00000000e+02,  1.00000000e+02,\n          1.00000000e+02,  1.00000000e+02]),\n    np.array(\n        [2.15241886, 2.15239732, 2.15235414, 2.15228893, 2.15220104,\n         2.15208949, 2.15197643, 2.15186182, 2.15174563, 2.15162783,\n         2.15150839, 2.15138728, 2.15126447, 2.15113993, 2.15101361,\n         2.15088549, 2.15075552, 2.15062368, 2.15048993, 2.15035422,\n         2.15021653, 2.15007679, 2.14993499, 2.14979107, 2.14964499,\n         2.1494967 , 2.14934616, 2.14919332, 2.14903814, 2.14888056,\n         2.14872053, 2.148558  , 2.14839292, 2.14822523, 2.14805487,\n         2.14788178, 2.14770591, 2.14752719, 2.14734556, 2.14716095,\n         2.14697329, 2.14678252, 2.14658857, 2.14639136, 2.14619081,\n         2.14598685, 2.14577939, 2.14556836, 2.14535366, 2.14513522,\n         2.14491294, 2.14468672, 2.14445648, 2.1442221 , 2.14398349,\n         2.14374055, 2.14349316, 2.14324121, 2.14298459, 2.14272317,\n         2.14245683, 2.14218544, 2.14190888, 2.14162699, 2.14133965,\n         2.14104671, 2.14074801, 2.1404434 , 2.14013271, 2.13981578,\n         2.13949244, 2.1391625 , 2.13882579, 2.13848209, 2.13813122,\n         2.13777298, 2.13740713, 2.13703346, 2.13665175, 2.13626174,\n         2.13586319, 2.13545585, 2.13503943, 2.13461366, 2.13417826,\n         2.13373291, 2.13327731, 2.13281113, 2.13233402, 2.13184564,\n         2.1313456 , 2.13083354, 2.13030904, 2.12977169, 2.12922105,\n         2.12865667, 2.12807807, 2.12748476, 2.12687621, 2.12625189,\n         2.12561122, 2.12495362, 2.12427846, 2.1235851 , 2.12287286,\n         2.12214101, 2.12138882, 2.1206155 , 2.11982023, 2.11900214,\n         2.11816032, 2.11729383, 2.11640166, 2.11548276, 2.11453601,\n         2.11356024, 2.11255423, 2.11151667, 2.11044619, 2.10934135,\n         2.10820061, 2.10702235, 2.10580488, 2.10454638, 2.10324493,\n         2.1018985 , 2.10050495, 2.09906198, 2.09756717, 2.09601793,\n         2.09441153, 2.09274504, 2.09101536, 2.08921916, 2.0873529 ,\n         2.08541282, 2.08339487, 2.08129473, 2.07910776, 2.07682903,\n         2.0744532 , 2.07197458, 2.06938702, 2.06668393, 2.06385821,\n         2.06090219, 2.05780762, 2.05456557, 2.05116642, 2.04759972,\n         2.04385416, 2.0399175 , 2.03577645, 2.03141654, 2.02682207,\n         2.02197592, 2.01685948, 2.0114524 , 2.00573252, 1.99967561,\n         1.99325521, 1.98644235, 1.97920538, 1.97150962, 1.96331709,\n         1.95458623, 1.94527151, 1.93532302, 1.92468616, 1.91330112,\n         1.90110237, 1.88801823, 1.87397035, 1.85887315, 1.84263358,\n         1.82515123, 1.80631939, 1.7860286 , 1.76417417, 1.7406663 ,\n         1.71544581, 1.6884964 , 1.65984458, 1.62954065, 1.59761539,\n         1.56401609, 1.52869221, 1.4920363 , 1.45492882, 1.41830507,\n         1.38304801, 1.34971675, 1.31789431, 1.28673462, 1.25630383,\n         1.22626369, 1.19731608, 1.17042042, 1.14592617, 1.12356836,\n         1.10382229, 1.08627322, 1.0701912 , 1.05533321, 1.04158601,\n         1.02879729, 1.01693774, 1.00594766, 0.99614615, 0.98836752,\n         0.98257838, 0.97750256, 0.97305027, 0.96868045, 0.96322834,\n         0.95670166, 0.95074867, 0.94543953, 0.94074184, 0.93662438,\n         0.93304778, 0.929999  , 0.92744291, 0.92543981, 0.92396184,\n         0.92296245, 0.92230994, 0.92191552, 0.92158792, 0.92127977,\n         0.92097067, 0.92071658, 0.9205028 , 0.92031392, 0.92014471,\n         0.91999943, 0.91986096, 0.91972707, 0.91961127, 0.91950272,\n         0.91941389, 0.919332  , 0.91925326, 0.91918257, 0.91911623,\n         0.91906362, 0.91901607, 0.91898093, 0.91892304, 0.91887442,\n         0.91882514, 0.91876538, 0.91870849, 0.91864118, 0.91858104,\n         0.91851137, 0.91844507, 0.91836555, 0.9182813 , 0.91819518,\n         0.91810947, 0.91802511, 0.91794894, 0.91787628, 0.91780379,\n         0.91773116, 0.91763942, 0.91753445, 0.91740104, 0.91725416,\n         0.91710059, 0.91695562, 0.91681812, 0.91669648, 0.91657225,\n         0.9164454 , 0.91630778, 0.91612458, 0.91590644, 0.91564697,\n         0.91535115, 0.91502114, 0.91468199, 0.91413672, 0.91320789,\n         0.91180931, 0.90987257, 0.90743783, 0.90494806, 0.90196083,\n         0.89848543, 0.89453194, 0.89003571, 0.8847903 , 0.87911514,\n         0.87301681, 0.86653323, 0.85972191, 0.85264744, 0.84535685,\n         0.83793598, 0.83044481, 0.82295242, 0.81552485, 0.80820985,\n         0.80106912, 0.79414917, 0.78748437, 0.78110965, 0.77503897,\n         0.76929405, 0.76387843, 0.75880196, 0.75406415, 0.7496679 ,\n         0.74561249, 0.74189059, 0.73850467, 0.73544876, 0.73271275,\n         0.73027848, 0.7281251 , 0.72622343, 0.7245446 , 0.72306012,\n         0.72174368, 0.72057213, 0.71952555, 0.71858701, 0.71774217,\n         0.71697891, 0.71628698, 0.71565763, 0.71508345, 0.71455805,\n         0.71407596, 0.71363246, 0.71322346, 0.71284538, 0.71249512,\n         0.71216995, 0.71186748, 0.71158559, 0.71132242, 0.71107629,\n         0.71084574, 0.71062943, 0.71042621, 0.71023499, 0.71005483,\n         0.70988487, 0.70972434, 0.70957253, 0.7094288 , 0.70929258,\n         0.70916333, 0.70904058, 0.70892389, 0.70881285, 0.70870709,\n         0.70860627, 0.70851009, 0.70841826, 0.7083305 , 0.70824658,\n         0.70816627, 0.70808935, 0.70801564, 0.70794495, 0.70787712,\n         0.70781199, 0.70774941, 0.70768926, 0.70763139, 0.70757569,\n         0.70752206, 0.70747039, 0.70742058, 0.70737255, 0.70732619,\n         0.70728145, 0.70723823, 0.70719648, 0.70715611, 0.70711708,\n         0.70707931, 0.70704276, 0.70700737, 0.70697309, 0.70693987,\n         0.70690767, 0.70687645, 0.70684616, 0.70681678, 0.70678825,\n         0.70676055, 0.70673364, 0.7067075 , 0.70668209, 0.70665739,\n         0.70663337, 0.70661   , 0.70658726, 0.70656513, 0.70654359,\n         0.7065226 , 0.70650216, 0.70648225, 0.70646284, 0.70644392,\n         0.70642547, 0.70640747, 0.70638992, 0.70637279, 0.70635608,\n         0.70633976, 0.70632383, 0.70630827, 0.70629307, 0.70627822,\n         0.70626665, 0.70625812, 0.7062525 , 0.70624971, 0.        ,\n         0.        , 0.        , 0.        , 0.        , 0.        ]),\n 5)\n# fmt: on\n\n# approximate partition function for C=1, tau = 10 from r=-100 to 100\n# error < 4E-7\nln_Z_fit = BSpline.construct_fast(*TCK)\nln_Z_inf = 2.1653591123321405\n\n\ndef ln_Z(alpha, alpha_min=-100):\n    \"\"\"\n    Function to fit a spline onto the data points. Since some points may have higher changes in their local neighborhood,\n    we need to fit more points in that region via the spline. The spline is fit on the data points for alpha >= alpha_min.\n\n    Parameters:\n    alpha (float): The alpha value for which the spline of Z is to be calculated.\n    alpha_min (float, optional): The minimum value of alpha. Defaults to -100.\n\n    Returns:\n    float: The spline fit on Z for the given alpha. If alpha is less than or equal to alpha_min,\n    the function returns the value at infinity, i.e. 11.2.\n    \"\"\"\n\n    if alpha <= alpha_min:\n        return ln_Z_inf\n\n    return ln_Z_fit(alpha)"
  },
  {
    "path": "opendsm/common/stats/basic.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2024 OpenEEmeter contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\n\nfrom typing import Literal, Optional, Union\n\nimport numba\nimport numpy as np\n\nfrom scipy.special import (\n    stdtrit,  # faster than using t.ppf\n    erfinv,  # faster than using norm.ppf\n)\n\nfrom opendsm.common.utils import to_np_array\n\n\n\n# Constant to convert MAD to std deviation for normal distribution\n# Equivalent to 1 / norm_dist.ppf(0.75)\nMAD_k = 1 / (erfinv(2 * 0.75 - 1) * np.sqrt(2))\n\n\ndef t_stat(alpha: float, n: int, tail: Union[int, str] = 2) -> float:\n    \"\"\"Calculate the t-statistic for hypothesis testing.\n\n    Args:\n        alpha: Significance level\n        n: Sample size\n        tail: Type of tail test - 1/\"one\" for one-tailed, 2/\"two\" for two-tailed\n\n    Returns:\n        Calculated t-statistic value\n\n    Raises:\n        ValueError: If tail parameter is invalid\n    \"\"\"\n    degrees_of_freedom = n - 1\n    if (tail == \"one\") or (tail == 1):\n        perc = np.asarray(1 - alpha)\n    elif (tail == \"two\") or (tail == 2):\n        perc = np.asarray(1 - alpha / 2)\n    else:\n        raise ValueError(f\"Invalid tail parameter: {tail}. Must be 1/'one' or 2/'two'\")\n\n    return stdtrit(degrees_of_freedom, perc)\n\n\ndef z_stat(alpha: float, tail: Union[int, str] = 2) -> float:\n    \"\"\"Calculate the z-statistic for hypothesis testing.\n\n    Args:\n        alpha: Significance level\n        tail: Type of tail test - 1/\"one\" for one-tailed, 2/\"two\" for two-tailed\n\n    Returns:\n        Calculated z-statistic value\n\n    Raises:\n        ValueError: If tail parameter is invalid\n    \"\"\"\n    if (tail == \"one\") or (tail == 1):\n        perc = np.asarray(1 - alpha)\n    elif (tail == \"two\") or (tail == 2):\n        perc = np.asarray(1 - alpha / 2)\n    else:\n        raise ValueError(f\"Invalid tail parameter: {tail}. Must be 1/'one' or 2/'two'\")\n\n    return erfinv(2 * perc - 1) * np.sqrt(2)\n\n\ndef unc_factor(\n    n: int, interval: Literal[\"PI\", \"CI\"] = \"PI\", alpha: float = 0.10\n) -> float:\n    \"\"\"Calculate uncertainty factor for confidence or prediction intervals.\n\n    Args:\n        n: Sample size\n        interval: Interval type - \"CI\" for Confidence Interval or \"PI\" for Prediction Interval\n        alpha: Significance level\n\n    Returns:\n        Uncertainty factor value\n\n    Raises:\n        ValueError: If interval type is invalid\n    \"\"\"\n    if interval == \"CI\":\n        return t_stat(alpha, n) / np.sqrt(n)\n    elif interval == \"PI\":\n        return t_stat(alpha, n) * (1 + 1 / np.sqrt(n))\n    else:\n        raise ValueError(f\"Invalid interval: {interval}. Must be 'CI' or 'PI'\")\n\n\n@numba.jit(nopython=True, cache=True)\ndef weighted_std(\n    x: np.ndarray,\n    w: np.ndarray,\n    mean: Optional[float] = None,\n    w_sum_err: float = 1e-6,\n) -> float:\n    \"\"\"Calculate weighted standard deviation with optional normalization.\n\n    Args:\n        x: Input data array\n        w: Weights for each data point\n        mean: Pre-computed mean (if None, calculated from weighted data)\n        w_sum_err: Tolerance for weight normalization check\n\n    Returns:\n        Weighted standard deviation\n    \"\"\"\n    n = float(len(x))\n\n    w_sum = np.sum(w)\n    if w_sum < 1 - w_sum_err or w_sum > 1 + w_sum_err:\n        w /= w_sum\n\n    if mean is None:\n        mean = np.sum(w * x)\n\n    var = np.sum(w * np.power((x - mean), 2)) / (1 - 1 / n)\n\n    return np.sqrt(var)\n\n\ndef fast_std(\n    x: np.ndarray,\n    weights: Optional[Union[np.ndarray, float, int]] = None,\n    mean: Optional[float] = None,\n) -> float:\n    \"\"\"Calculate standard deviation (weighted or unweighted) efficiently.\n\n    Automatically determines whether to use weighted or unweighted calculation\n    based on the weights parameter.\n\n    Args:\n        x: Input data array\n        weights: Optional weights (array, scalar, or None for unweighted)\n        mean: Pre-computed mean (if None, calculated from data)\n\n    Returns:\n        Standard deviation value\n    \"\"\"\n    if isinstance(weights, (int, float)):\n        weights = np.array([weights])\n\n    if weights is None or len(weights) == 1 or np.allclose(weights - weights[0], 0):\n        if mean is None:\n            return np.std(x)\n        else:\n            n = float(len(x))\n            var = np.sum(np.power((x - mean), 2)) / n\n            return np.sqrt(var)\n    else:\n        if mean is None:\n            mean = np.average(x, weights=weights)\n\n        return weighted_std(x, weights, mean)\n\n\n@numba.jit(nopython=True, cache=True)\ndef _weighted_quantile(\n    values: np.ndarray,\n    quantiles: np.ndarray,\n    weights: Optional[np.ndarray] = None,\n    values_presorted: bool = False,\n    old_style: bool = False,\n) -> np.ndarray:\n    \"\"\"Calculate weighted quantiles (numba-optimized internal implementation).\n\n    Similar to numpy.percentile but supports weighted observations.\n    Reference: https://stackoverflow.com/questions/21844024/weighted-percentile-using-numpy\n\n    Args:\n        values: Input data array\n        quantiles: Array of quantiles to compute (must be in [0, 1])\n        weights: Optional weights for each value (same length as values)\n        values_presorted: If True, assumes values are already sorted\n        old_style: If True, uses numpy.quantile-compatible output\n\n    Returns:\n        Array of computed quantiles\n\n    Raises:\n        ValueError: If quantiles are not in [0, 1]\n    \"\"\"\n    for q in quantiles:\n        if not 0 <= q <= 1:\n            raise ValueError(\"quantiles should be in [0, 1]\")\n\n    finite_idx = np.where(np.isfinite(values))\n    values = values[finite_idx]\n\n    if weights is None:\n        weights = np.ones_like(values)\n    else:\n        weights = weights[finite_idx]\n\n    if not values_presorted:\n        sorted_idx = np.argsort(values)\n        values = values[sorted_idx]\n        weights = weights[sorted_idx]\n\n    res = np.cumsum(weights) - 0.5 * weights\n    if old_style:  # To be convenient with numpy.quantile\n        res -= res[0]\n        res /= res[-1]\n    else:\n        res /= np.sum(weights)\n\n    return np.interp(quantiles, res, values)\n\n\ndef weighted_quantile(\n    values: Union[np.ndarray, list],\n    quantiles: Union[np.ndarray, list, float],\n    weights: Optional[Union[np.ndarray, list]] = None,\n    values_presorted: bool = False,\n    old_style: bool = False,\n) -> np.ndarray:\n    \"\"\"Calculate weighted quantiles with input validation.\n\n    Public wrapper for _weighted_quantile that handles input conversion\n    and provides better error messages.\n\n    Args:\n        values: Input data (array-like)\n        quantiles: Quantiles to compute (array-like or scalar, in [0, 1])\n        weights: Optional weights (array-like)\n        values_presorted: If True, assumes values are already sorted\n        old_style: If True, uses numpy.quantile-compatible output\n\n    Returns:\n        Array of computed quantiles\n\n    Raises:\n        Exception: If weighted quantile calculation fails\n    \"\"\"\n    values = to_np_array(values)\n    quantiles = to_np_array(quantiles)\n\n    if weights is None:\n        weights = np.ones_like(values)\n    else:\n        weights = to_np_array(weights)\n\n    try:\n        res = _weighted_quantile(values, quantiles, weights, values_presorted, old_style)\n    except Exception as e:\n        print(\"Error in weighted_quantile:\")\n        print(f\"  values shape: {values.shape}, dtype: {values.dtype}\")\n        print(f\"  quantiles: {quantiles}\")\n        print(f\"  weights shape: {weights.shape}, dtype: {weights.dtype}\")\n        raise Exception(f\"Error in weighted_quantile: {str(e)}\") from e\n\n    return res\n\n\n@numba.jit(nopython=True, cache=True)\ndef _median_absolute_deviation(\n    x: np.ndarray,\n    median: Optional[float] = None,\n    weights: Optional[np.ndarray] = None,\n) -> float:\n    \"\"\"Calculate Median Absolute Deviation (numba-optimized internal implementation).\n\n    Computes MAD scaled to match standard deviation of normal distribution.\n    Supports both weighted and unweighted calculations. Only handles 1D arrays.\n\n    Args:\n        x: Input data array (1D)\n        median: Pre-computed median (if None, calculated from data)\n        weights: Optional weights for weighted MAD calculation\n\n    Returns:\n        MAD value scaled to match standard deviation units\n    \"\"\"\n    mu = median\n    if weights is None:\n        if mu is None:\n            mu = np.median(x)\n\n        sigma = np.median(np.abs(x - mu))\n\n    else:\n        if mu is None:\n            mu = _weighted_quantile(x, np.array([0.5]), weights=weights, values_presorted=False)[0]\n\n        sigma = _weighted_quantile(\n            np.abs(x - mu), np.array([0.5]), weights=weights, values_presorted=False\n        )[0]\n\n    return sigma * MAD_k\n\n\ndef median_absolute_deviation(\n    x: Union[np.ndarray, list],\n    median: Optional[float] = None,\n    weights: Optional[Union[np.ndarray, list]] = None,\n    axis: Optional[int] = None,\n) -> Union[float, np.ndarray]:\n    \"\"\"Calculate Median Absolute Deviation (MAD) scaled to standard deviation.\n\n    Public wrapper that handles input conversion. Supports both weighted\n    and unweighted calculations.\n\n    Args:\n        x: Input data (array-like)\n        median: Pre-computed median (if None, calculated from data)\n        weights: Optional weights for weighted MAD calculation\n        axis: Axis along which to compute MAD (None for flattened array)\n\n    Returns:\n        MAD value scaled to match standard deviation units\n    \"\"\"\n    x = to_np_array(x)\n\n    if weights is not None:\n        weights = to_np_array(weights)\n\n    if axis is None:\n        # Flatten array for 1D calculation\n        x_flat = x.ravel()\n        weights_flat = weights.ravel() if weights is not None else None\n        return _median_absolute_deviation(x_flat, median=median, weights=weights_flat)\n    else:\n        # Apply along specified axis\n        def mad_1d(x_slice):\n            return _median_absolute_deviation(x_slice, median=None, weights=None)\n\n        return np.apply_along_axis(mad_1d, axis, x)"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .standardize import robust_standardize\nfrom .bisymlog import bisymlog\nfrom .scipy_yeo_johnson import scipy_YJ, robust_scipy_YJ\nfrom .raymaekers_robust_yeo_johnson import raymaekers_robust_YJ\n\n\n__all__ = (\n    \"robust_standardize\",\n    \"bisymlog\",\n    \"scipy_YJ\",\n    \"robust_scipy_YJ\"\n    \"raymaekers_robust_YJ\",\n)"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/bisymlog.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\n\nfrom scipy.optimize import minimize_scalar\nfrom scipy.stats import skew\nfrom statsmodels.stats.stattools import robust_skewness as robust_skew\n\nfrom opendsm.common.stats.distribution_transform.standardize import robust_standardize\nfrom opendsm.common.stats.outliers import IQR_outlier\n\nfrom opendsm.common.utils import (\n    OoM,\n    RoundToSigFigs,\n)\n\n\nC_base = 1/np.log(10)\n\nclass Bisymlog:\n    def __init__(self, C=C_base, heuristic_scaling_factor=0.5, base=10, rescale_quantile=None):\n        self.C = C\n        self._heuristic_scaling_factor = heuristic_scaling_factor\n        self._base = base\n        self.rescale_quantile = rescale_quantile\n        self.inv_rescale_fcn = None\n\n        self.scaling_factor_bnds = [-1.0, 6.0] # Hardcoded, but not necessary to do so\n\n        if rescale_quantile is not None and (rescale_quantile < 0.0 or rescale_quantile > 0.5):\n            raise ValueError(\"Bisymlog 'rescale_quantile' must be 0 < x < 0.5\")\n\n    def set_C_heuristically(self, y, scaling_factor=None): # scaling factor: 0 looks loglike, 1 linear like\n        if scaling_factor is None:\n            scaling_factor = self._heuristic_scaling_factor\n        else:\n            self._heuristic_scaling_factor = scaling_factor\n\n        min_y = y.min()\n        max_y = y.max()\n\n        if min_y == max_y:\n            self.C = None\n            return 1/np.log(1000)\n\n        elif np.sign(max_y) != np.sign(min_y): # if zero is within total range, find largest pos or neg range\n            processed_data = [y[y >= 0], y[y <= 0]]\n            C = 0\n            for data in processed_data:\n                range = np.abs(data.max() - data.min())\n                if range > C:\n                    C = range\n                    max_y = data.max()\n\n        else:\n            C = np.abs(max_y-min_y)\n\n        s_fcn = lambda x: np.power(10, np.power(x, 2))\n        s_fcn_range = s_fcn([0, 1])\n        scaling_factor = s_fcn(self._heuristic_scaling_factor)\n\n        s_bnds = self.scaling_factor_bnds\n\n        s = (scaling_factor - s_fcn_range[0])/np.diff(s_fcn_range)*np.diff(s_bnds) + s_bnds[0]\n\n        C *= 10**(OoM(max_y) + s[0])\n        # TODO: round or not?\n        # C = RoundToSigFigs(C, 1)    # round to 1 significant figure\n\n        self.C = C\n\n        return C\n\n    def transform(self, y):\n        if self.C is None:\n            self.C = self.set_C_heuristically(y)\n\n        if self.C is None:\n            return y\n\n        else:\n            idx = np.isfinite(y)   # only perform transformation on finite values\n            res = np.empty_like(y)\n            res[~idx] = np.nan\n\n            if self.rescale_quantile is None:\n                res[idx] = np.sign(y[idx])*np.log10(np.abs(y[idx]/self.C) + 1)/np.log10(self._base)\n\n            else:\n                # get prior quantiles for rescaling\n                pq = np.quantile(y[idx], [self.rescale_quantile, 1 - self.rescale_quantile])\n\n                res[idx] = np.sign(y[idx])*np.log10(np.abs(y[idx]/self.C) + 1)/np.log10(self._base)\n\n                # get current quantiles for rescaling and set rescaling functions\n                cq = np.quantile(res[idx], [self.rescale_quantile, 1 - self.rescale_quantile])\n                rescale_fcn = lambda x: (x - cq[0])/np.diff(cq)*np.diff(pq) + pq[0]\n                self.inv_rescale_fcn = lambda x: (x - pq[0])/np.diff(pq)*np.diff(cq) + cq[0]\n\n                # rescale to prior quantiles\n                res[idx] = rescale_fcn(res[idx])\n\n            return res\n\n    def invTransform(self, y):\n        if self.C is None:\n            raise Exception('C is unspecified in Bisymlog')\n\n        idx = np.isfinite(y)   # only perform transformation on finite values\n\n        if self.inv_rescale_fcn is not None:\n            y[idx] = self.inv_rescale_fcn(y[idx])\n\n        res = np.empty_like(y)\n        res[~idx] = np.nan\n        res[idx] = np.sign(y[idx])*self.C*(np.power(self._base, np.abs(y[idx])) - 1)\n\n        return res\n\n\ndef bisymlog(x, rescale_quantile=None):\n    def obj_fcn(X):\n        C = 10**X\n\n        xt = Bisymlog(C=C, rescale_quantile=rescale_quantile).transform(x)\n        xt = robust_standardize(xt, robust_type=\"adaptive_weighted\", use_mean=False, rel_err=1E-4, abs_err=1E-4)\n\n        xt_outliers = IQR_outlier(xt, sigma_threshold=3, quantile=0.05)\n        xt = xt[(xt_outliers[0] < xt) & (xt < xt_outliers[1])]\n\n        abs_skew = np.abs(skew(xt))\n\n        return abs_skew\n\n    bnds = [-14, 6]\n\n    res = minimize_scalar(obj_fcn, bounds=bnds, method='bounded')\n    C = 10**res.x\n\n    xt = Bisymlog(C=C, rescale_quantile=rescale_quantile).transform(x)\n    xt = robust_standardize(xt, robust_type=\"adaptive_weighted\", use_mean=False, rel_err=1E-4, abs_err=1E-4)\n\n    return xt"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/mu_sigma.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom copy import deepcopy as copy\nimport numpy as np\nfrom statsmodels.robust.scale import Huber as huber_m_estimate\n\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.common.stats.basic import (\n    MAD_k, \n    weighted_quantile,\n    median_absolute_deviation,\n)\n\n\ndef adaptive_weighted_mu_sigma(x, use_mean=False, rel_err=1E-4, abs_err=1E-4):\n    mu = np.median(x)\n    sigma = median_absolute_deviation(x, mu=mu)\n\n    for n in range(10):\n        mu_prior = copy(mu)\n        sigma_prior = copy(sigma)\n        weight = adaptive_weights(x, mu=mu_prior, sigma=sigma_prior)[0]\n        if use_mean:\n            mu = np.sum(weight*x)/np.sum(weight)\n            sigma = np.sum(weight*(x - mu)**2)/np.sum(weight)\n\n        else:\n            mu = weighted_quantile(x, 0.5, weights=weight)\n            sigma = median_absolute_deviation(x, mu=mu, weights=weight)\n\n        max_abs_err = np.max(np.abs([(mu - mu_prior), (sigma - sigma_prior)]))\n        max_rel_err = np.max(np.abs([(mu - mu_prior)/mu_prior, (sigma - sigma_prior)/sigma_prior]))\n\n        if (max_rel_err < rel_err) | (max_abs_err < abs_err):\n            break\n\n    if sigma == 0:\n        sigma = 1\n\n    return mu, sigma\n\n\ndef ransac_mu_sigma(x, n_iter=100, n_sample=100, seed=None):\n    mu = np.median(x)\n    sigma = median_absolute_deviation(x, mu=mu)\n\n    for _ in range(n_iter):\n        np.random.seed(seed)\n        idx = np.random.choice(x.size, n_sample)\n        x_sample = x[idx]\n\n        mu_sample = np.median(x_sample)\n        sigma_sample = median_absolute_deviation(x_sample, mu=mu_sample)\n\n        if sigma_sample < sigma:\n            mu = mu_sample\n            sigma = sigma_sample\n\n    return mu, sigma\n\n\ndef robust_mu_sigma(x, robust_type=\"huber_m_estimate\", **kwargs):\n    if (len(x) <= 3) and (robust_type != \"iqr\"):\n        robust_type = \"iqr\"\n\n    if robust_type == \"iqr\":\n        mu = weighted_quantile(x, 0.5)\n        sigma = weighted_quantile(np.abs(x - mu), 0.5)*MAD_k\n\n    elif robust_type == \"huber_m_estimate\":\n        try:\n            if \"maxiter\" not in kwargs:\n                kwargs[\"maxiter\"] = 50\n\n            # raise RuntimeWarning to error\n            with np.seterr(all='raise'):\n                mu, sigma = huber_m_estimate(**kwargs)(x)\n\n        except Exception as e:\n            mu, sigma = robust_mu_sigma(x, robust_type=\"iqr\")\n\n    elif robust_type == \"adaptive_weighted\": # slow\n        mu, sigma = adaptive_weighted_mu_sigma(x, **kwargs)\n\n    elif robust_type == \"ransac\":\n        mu, sigma = ransac_mu_sigma(x, **kwargs)\n\n    return mu, sigma"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/raymaekers_robust_yeo_johnson.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport numba\nfrom numba import float64, boolean\n\nfrom scipy.stats import norm\nfrom scipy.optimize import minimize_scalar\n\nfrom opendsm.common.stats.distribution_transform.standardize import (\n    robust_standardize,\n)\nfrom opendsm.common.stats.distribution_transform.mu_sigma import adaptive_weighted_mu_sigma, robust_mu_sigma\nfrom opendsm.common.stats.adaptive_loss import adaptive_loss_fcn\n\n\n# Work based on RayMaekers 2021 paper titled \"Transforming variables to central normality\"\n# https://doi.org/10.1007/s10994-021-05960-5\n\n# TODO: interesting article: https://link.springer.com/article/10.1007/s10260-022-00640-7#Sec17\n#                            https://github.com/UniprJRC/FSDA/tree/master/toolbox/regression\n#                            https://github.com/UniprJRC/FSDA/blob/master/toolbox/regression/FSRfan.m\n#                            https://github.com/UniprJRC/FSDA/blob/master/toolbox/regression/fanBIC.m\n\n_NO_DERIV = False\n_DERIV = True\n\n\n@numba.jit((float64)(float64, float64, boolean), nopython=True, error_model=\"numpy\", cache=True)\ndef _yeo_johnson_base(x, lam, deriv):\n    if not deriv:\n        if   (lam != 0) and (x >= 0):\n            return ((1 + x)**lam - 1)/lam\n        elif (lam == 0) and (x >= 0):\n            return np.log(1 + x)\n        elif (lam != 2) and (x < 0):\n            return -((1 - x)**(2 - lam) - 1)/(2 - lam)\n        elif (lam == 2) and (x < 0):\n            return -np.log(1 - x)\n        else:\n            return np.nan\n\n    else:\n        if   (lam != 0) and (x >= 0):\n            return (x + 1)**(lam - 1)\n        elif (lam == 0) and (x >= 0):\n            return 1/(1 + x)\n        elif (lam != 2) and (x < 0):\n            return (1 - x)**(1 - lam)\n        elif (lam == 2) and (x < 0):\n            return 1/(1 - x)\n        else:\n            return np.nan\n\n\n@numba.jit((float64)(float64, float64, boolean), nopython=True, error_model=\"numpy\", cache=True)\ndef _box_cox_base(x, lam, deriv):\n    if not deriv:\n        if   (lam != 0):\n            return (x**lam - 1)/lam\n        elif (lam == 0):\n            return np.log(x)\n        else:\n            return np.nan\n\n    else:\n        if   (lam != 0):\n            return x**(lam - 1)\n        elif (lam == 0):\n            return 1/x\n        else:\n            return np.nan\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef rectified_transform(x, lam, Q, tr_type=\"Yeo-Johnson\"):\n    if tr_type == \"Yeo-Johnson\":\n        tr = _yeo_johnson_base\n    elif tr_type == \"Box-Cox\":\n        tr = _box_cox_base\n\n    [q1, q3] = Q\n\n    h = np.empty_like(x)\n    for i, xi in enumerate(x):\n        if (q1 <= xi) and (xi < q3):\n            h[i] = tr(xi, lam, _NO_DERIV)\n\n        elif q3 < xi:\n            h[i] = tr(q3, lam, _NO_DERIV) + (xi - q3)*tr(q3, lam, _DERIV)\n\n        elif xi < q1:\n            h[i] = tr(q1, lam, _NO_DERIV) + (xi - q1)*tr(q1, lam, _DERIV)\n\n    return h\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef unrectified_transform(x, lam, tr_type=\"Yeo-Johnson\"):\n    if tr_type == \"Yeo-Johnson\":\n        tr = _yeo_johnson_base\n    elif tr_type == \"Box-Cox\":\n        tr = _box_cox_base\n\n    h = np.empty_like(x)\n    for i, xi in enumerate(x):\n        h[i] = tr(xi, lam, _NO_DERIV)\n\n    return h\n\n\ndef loss_fcn(x, mu=0, c=1, loss_type=\"adaptive\"):\n    if loss_type == \"adaptive\":\n        loss, _ = adaptive_loss_fcn(x, mu=mu, c=c, alpha=\"adaptive\", replace_nonfinite=True)\n        return loss\n    \n    elif loss_type == \"tukey_bisquare\":\n        return np.piecewise(x, [np.abs(x) <= c, np.abs(x) > c], [lambda x: 1 - (1 - (x/c)**2)**3, 1])\n\n    else:\n        raise NotImplementedError(f\"loss_type: {loss_type} not implemented\")\n\n\ndef _robust_standardize(x, robust_type, c_huber):\n    if robust_type == \"huber_m_estimate\":\n        return robust_standardize(x, robust_type=robust_type, c=c_huber, tol=1e-08)\n    else:\n        return robust_standardize(x, robust_type=robust_type)\n\n\ndef initial_lam_obj_fcn_dec(x, Q, transform_type=\"Yeo-Johnson\", c=0.5, robust_type=\"huber_m_estimate\", c_huber=1.5):\n    phi = norm.ppf((np.arange(0, len(x)) + 2/3)/(len(x) + 1/3))\n\n    if robust_type == \"huber_m_estimate\":\n        mu, sigma = robust_mu_sigma(x, robust_type, c=c_huber, tol=1e-08)\n    else:\n        mu, sigma = robust_mu_sigma(x, robust_type)\n\n    def lam_obj_fcn(lam):\n        h = rectified_transform(x, lam, Q, tr_type=transform_type)\n        \n        loss = loss_fcn((h - mu)/sigma - phi, mu=0, c=c, loss_type=\"tukey_bisquare\")\n        # loss = loss_fcn((h - mu)/sigma - phi, mu=0, c=c, loss_type=\"adaptive\")\n\n        return np.sum(loss)\n\n    return lam_obj_fcn\n\n\ndef lam_obj_fcn_dec(\n    x, \n    lam_0, \n    transform_type=\"Yeo-Johnson\",\n    robust_type=\"huber_m_estimate\",\n    c_huber=1.5, \n    outlier_alpha=0.005, \n):\n    h_0 = unrectified_transform(x, lam_0, tr_type=transform_type)\n    h_0_standardized = np.abs(_robust_standardize(h_0, robust_type, c_huber))\n\n    phi = norm.ppf(1 - outlier_alpha)\n\n    weight = np.zeros_like(x)\n    weight[h_0_standardized <= phi] = 1\n\n    def lam_obj_fcn(lam):\n        h = unrectified_transform(x, lam, tr_type=transform_type)\n        if not np.any(weight):\n            weighted_var = 1\n        else:\n            weighted_mu = np.sum(weight*h)/np.sum(weight)\n            weighted_var = np.sum(weight*(h - weighted_mu)**2)/np.sum(weight)\n\n        if transform_type == \"Yeo-Johnson\":\n            ML = -0.5*np.log(weighted_var) + (lam - 1)*np.sign(x)*np.log(1 + np.abs(x))\n\n        elif transform_type == \"Box-Cox\":\n            ML = -0.5*np.log(weighted_var) + (lam - 1)*np.log(x)\n\n        return -np.sum(weight*ML)\n\n    return lam_obj_fcn\n\n\ndef normal_transformation(\n    x, \n    Q_perc=0.25, \n    transform_type=\"Yeo-Johnson\", \n    c=0.5,\n    outlier_alpha=0.005, \n    c_huber=1.5, \n    robust_type=\"huber_m_estimate\",\n    pre_standardize=True,\n    post_standardize=True,\n    ):\n\n    # bounds = np.array([-1, 3]) + np.array([-10, 10])\n    # bracket = np.array([-1, 1, 3])\n    lmbda_bnds = np.array([-10, 10])\n\n    if pre_standardize:\n        if transform_type == \"Yeo-Johnson\":\n            x = _robust_standardize(x, robust_type, c_huber)\n        elif transform_type == \"Box-Cox\":\n            x = np.exp(_robust_standardize(np.log(x), robust_type, c_huber))\n\n    x = np.sort(x)\n    Q = np.quantile(x, [Q_perc, 1 - Q_perc])\n    for n in range(3):\n        if n == 0:\n            lam_loss_0 = initial_lam_obj_fcn_dec(x, Q, transform_type, c, robust_type, c_huber)\n            # res = minimize_scalar(lam_loss_0, bounds=lmbda_bnds, method=\"bounded\")\n            res = minimize_scalar(lam_loss_0, bracket=lmbda_bnds, method=\"brent\")\n            lam = res.x\n\n        else:\n            lam_loss = lam_obj_fcn_dec(x, lam, transform_type, robust_type, c_huber, outlier_alpha=outlier_alpha)\n            # res = minimize_scalar(lam_loss, bounds=lmbda_bnds, method=\"bounded\")\n            res = minimize_scalar(lam_loss, bracket=lmbda_bnds, method=\"brent\")\n            lam = res.x\n\n    xt = rectified_transform(x, lam, Q=Q, tr_type=transform_type)\n\n    if post_standardize:\n        xt = _robust_standardize(xt, robust_type, c_huber)\n\n    return xt, lam\n\n\ndef raymaekers_robust_YJ(x, Q_perc=0.25, c=0.5, outlier_alpha=0.005, c_huber=1.5, robust_type=\"huber_m_estimate\"):\n    # outlier_alpha should be between 0.005 and 0.025 (0.005 is higher efficiency, less robust)\n\n    if np.all(x == x[0]): # if all values are the same, do not transform, return\n        return np.zeros_like(x)\n\n    idx_finite = np.argwhere(np.isfinite(x)).flatten()\n    idx_nonfinite = np.array([i for i in np.arange(len(x)) if i not in idx_finite])\n\n    xt_yj_out = np.empty_like(x)    \n    if len(idx_finite) > 3:\n        xt_yj, _ = normal_transformation(\n            x[idx_finite], \n            Q_perc=Q_perc, \n            transform_type=\"Yeo-Johnson\", \n            c=c, \n            c_huber=c_huber, \n            outlier_alpha=outlier_alpha,\n            robust_type=robust_type,\n        )\n\n        xt_yj_out[idx_finite] = xt_yj\n\n    if len(idx_nonfinite) > 0:\n        xt_yj_out[idx_nonfinite] = x[idx_nonfinite]\n    \n    return xt_yj_out"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/scipy_yeo_johnson.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\n\nfrom scipy.optimize import minimize_scalar\nfrom scipy.stats import yeojohnson\nfrom statsmodels.stats.stattools import robust_skewness as robust_skew\n\nfrom opendsm.common.stats.distribution_transform import robust_standardize\n\n\ndef scipy_YJ(x, robust_type=\"huber_m_estimate\"):\n    x_std, _ = yeojohnson(x, lmbda=None)\n    x_std = robust_standardize(x_std, robust_type)\n\n    return x_std\n\n\ndef obj_fcn_dec(x):\n    def obj_fcn(X):\n        xt = yeojohnson(x, lmbda=X)\n\n        n = 1 # [0: standard_skew, 1: quartile skew, 2: mean-median difference, standardized by abs deviation, 3: mean-median diff, standardized by std dev]\n        abs_skew = np.abs(robust_skew(xt))[n]\n\n        return abs_skew\n    return obj_fcn\n\n\ndef robust_scipy_YJ(x, robust_type=\"huber_m_estimate\", method=\"trim\", **kwargs):\n    idx_finite = np.argwhere(np.isfinite(x)).flatten()\n\n    if len(idx_finite) < 3:\n        return x\n\n    # pre standardize x\n    # x_finite = x[idx_finite]\n    x_finite = robust_standardize(x[idx_finite], robust_type)\n\n    if method == \"trim\":\n        trim_quantile = 0.1\n        if \"trim_quantile\" in kwargs:\n            trim_quantile = kwargs[\"trim_quantile\"]\n\n        x_bnds = np.quantile(x_finite, [trim_quantile, 1 - trim_quantile])\n\n        # get idx of x that is within the bounds\n        idx_trim = np.argwhere((x_finite >= x_bnds[0]) & (x_finite <= x_bnds[1])).flatten()\n\n        if len(idx_trim) >= 3:\n            _, lmbda = yeojohnson(x_finite[idx_trim], lmbda=None)\n        else:\n            lmbda = None\n\n    elif method == \"skew\":\n        bnds = [-1, 1]\n\n        obj_fcn = obj_fcn_dec(x_finite)\n        res = minimize_scalar(obj_fcn, bracket=bnds, method='brent')\n        lmbda = res.x\n\n    if lmbda is not None:\n        try:\n            x[idx_finite] = yeojohnson(x_finite, lmbda=lmbda)\n        except:\n            pass # if yeojohnson fails, return x as is\n    \n    # post standardize x\n    x[idx_finite] = robust_standardize(x[idx_finite], robust_type)\n\n    return x"
  },
  {
    "path": "opendsm/common/stats/distribution_transform/standardize.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.common.stats.distribution_transform.mu_sigma import robust_mu_sigma\n\n\ndef robust_standardize(x, robust_type=\"iqr\", **kwargs):\n    mu, sigma = robust_mu_sigma(x, robust_type, **kwargs)\n    x_std = (x - mu)/sigma\n\n    return x_std"
  },
  {
    "path": "opendsm/common/stats/outliers.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2024 OpenEEmeter contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\n\nimport numba\nimport numpy as np\n\nfrom opendsm.common.stats.basic import _weighted_quantile\nfrom opendsm.common.utils import to_np_array\n\n\ndef IQR_outlier(data, weights=None, sigma_threshold=3, quantile=0.25):\n    data = to_np_array(data)\n\n    if weights is not None:\n        weights = to_np_array(weights)\n\n    return _IQR_outlier(data, weights, sigma_threshold, quantile)\n\n\n@numba.jit(nopython=True, cache=True)\ndef _IQR_outlier(data, weights=None, sigma_threshold=3, quantile=0.25):\n    # only use finite data\n    if weights is None:\n        q13 = np.nanquantile(data[np.isfinite(data)], [quantile, 1 - quantile])\n    else:  # weighted_quantile could be used always, don't know speed\n        q13 = _weighted_quantile(\n            data[np.isfinite(data)], np.array([quantile, 1 - quantile]), weights=weights\n        )\n\n    q13_scalar = (\n        0.7413 * sigma_threshold - 0.5\n    )  # this is a pretty good fit to get the scalar for any sigma\n    iqr = np.diff(q13)[0] * q13_scalar\n    outlier_threshold = np.array([q13[0] - iqr, q13[1] + iqr])\n\n    return outlier_threshold\n\n\ndef remove_outliers(x, weights=None, sigma_threshold=3, quantile=0.25):\n    # if all values are the same return back all indices\n    if len(np.unique(x)) == 1:\n        return x, np.arange(len(x))\n\n    # prevent x_no_outliers from being empty\n    for sigma_added in range(10):\n        outlier_bnds = _IQR_outlier(x, weights, sigma_threshold + sigma_added, quantile)\n        idx_no_outliers = np.argwhere((x >= outlier_bnds[0]) & (x <= outlier_bnds[1])).flatten()\n\n        if idx_no_outliers.size > 0:\n            break\n\n    # if idx_no_outliers is empty, keep the closest meter to the outlier bounds\n    if len(idx_no_outliers) == 0:\n        # distance between x and outlier bounds\n        dist = -np.minimum(x - outlier_bnds[0], outlier_bnds[1] - x)\n\n        # sort by distance\n        # idx_no_outliers = np.argsort(dist)\n\n        # select closest\n        idx_no_outliers = np.array([np.argmin(dist)])\n\n    x_no_outliers = x[idx_no_outliers]\n\n    return x_no_outliers, idx_no_outliers"
  },
  {
    "path": "opendsm/common/stats/outliers_transformed.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2024 OpenEEmeter contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\n\nfrom opendsm.common.stats.distribution_transform import (\n    bisymlog,\n    scipy_YJ,\n    robust_scipy_YJ,\n    raymaekers_robust_YJ,\n    robust_standardize,\n)\nfrom opendsm.common.stats.outliers import remove_outliers as basic_remove_outliers\n\n\ndef remove_outliers(x, weights=None, sigma_threshold=3, quantile=0.25, transform=None):\n    if transform is None:\n        xt = x\n    elif transform == \"standardize\":\n        xt = robust_standardize(x)\n    elif transform == \"bisymlog\":\n        xt = bisymlog(x)\n    elif transform == \"scipy_YJ\":\n        xt = scipy_YJ(x)\n    elif transform == \"robust_scipy_YJ\":\n        xt = robust_scipy_YJ(x)\n    elif transform == \"robust_YJ\":\n        xt = raymaekers_robust_YJ(x)\n\n    _, idx_no_outliers = basic_remove_outliers(xt, weights, sigma_threshold, quantile)\n\n    if len(idx_no_outliers) == 0:\n        return x, []\n    \n    x_no_outliers = x[idx_no_outliers]\n\n    return x_no_outliers, idx_no_outliers"
  },
  {
    "path": "opendsm/common/test_data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom pathlib import Path\nfrom io import BytesIO\n\nimport pandas as pd\nimport pyarrow.parquet as pq\n\nimport requests\n\nfrom opendsm import __file__ as opendsm_file_path\nfrom opendsm.common.const import TutorialDataChoice\n\n# Define the current directory\ncurrent_dir = Path(opendsm_file_path).resolve().parent\ndata_dir = current_dir.parent / \"data\"\n\n# Set download information\nrepo_full_name = \"opendsm/opendsm\"\nbranch = \"master\"\npath = \"data\"\n\n\ncomparison_group_time_series = [\n    TutorialDataChoice.HOURLY_COMPARISON_GROUP_DATA,\n    TutorialDataChoice.DAILY_COMPARISON_GROUP_DATA,\n    TutorialDataChoice.MONTHLY_COMPARISON_GROUP_DATA,\n]\n\ntreatment_time_series = [\n    TutorialDataChoice.HOURLY_TREATMENT_DATA,\n    TutorialDataChoice.DAILY_TREATMENT_DATA,\n    TutorialDataChoice.MONTHLY_TREATMENT_DATA,\n]\n\n\ndef load_test_data(data_type: str):\n    \"\"\"Returns back tutorial data of the given data type as a dataframe\n\n    Args:\n        data_type (str): Must be one of the following:\n            - \"features\"\n            - \"seasonal_hourly_day_of_week_loadshape\"\n            - \"seasonal_day_of_week_loadshape\"\n            - \"month_loadshape\"\n            - \"hourly_data\"\n            - \"daily_treatment_data\"\n            - \"monthly_treatment_data\"\n\n    Returns:\n        (dataframe): Returns a dataframe\n    \"\"\"\n\n    # remove all \"_\" and \" \" from string and convert to lowercase\n    data_type = data_type.lower()\n    data_type = data_type.replace(\"_\", \"\").replace(\" \", \"\")\n\n    valid_list = [k.value for k in TutorialDataChoice]\n    keys = [k.lower() for k in TutorialDataChoice.__members__.keys()]\n\n    if data_type not in valid_list:\n        raise ValueError(\n            f\"Data type {data_type} not recognized. \\nMust be one of {keys}.\"\n        )\n\n    if data_type in [*comparison_group_time_series, *treatment_time_series]:\n        return _load_time_series_data(data_type)\n\n    else:\n        return _load_other_data(data_type)\n\n\ndef _load_time_series_data(data_type):\n    if data_type in comparison_group_time_series:\n        df = pd.concat(\n            [_load_file(\"hourly_data_0.parquet\"), _load_file(\"hourly_data_1.parquet\")],\n            axis=0,\n        )\n\n    elif data_type in treatment_time_series:\n        df = _load_file(\"hourly_data_2.parquet\")\n\n    # localize datetime and convert to CST\n    df = df.reset_index()\n    df[\"datetime\"] = df[\"datetime\"].dt.tz_localize(\"UTC\")\n    df[\"datetime\"] = df[\"datetime\"] + pd.Timedelta(hours=5)\n    df[\"datetime\"] = df[\"datetime\"].dt.tz_convert(\"America/Chicago\")\n    df = df.set_index([\"id\", \"datetime\"])\n\n    df_baseline = df[[\"temperature\", \"ghi_baseline\", \"observed_baseline\"]]\n    df_baseline = df_baseline.rename(columns={\"observed_baseline\": \"observed\", \"ghi_baseline\": \"ghi\"})\n\n    df_reporting = df[[\"temperature\", \"ghi_reporting\", \"observed_reporting\"]]\n    df_reporting = df_reporting.rename(columns={\"observed_reporting\": \"observed\", \"ghi_reporting\": \"ghi\"})\n\n    df_reporting = df_reporting.reset_index()\n    df_reporting[\"datetime\"] = df_reporting[\"datetime\"] + pd.Timedelta(days=365)\n    df_reporting = df_reporting.set_index([\"id\", \"datetime\"])\n\n    if \"daily\" in data_type:\n        df_baseline = _aggregate_hourly_data(df_baseline, \"D\")\n        df_reporting = _aggregate_hourly_data(df_reporting, \"D\")\n\n    elif \"monthly\" in data_type:\n        df_baseline = _aggregate_hourly_data(df_baseline, \"MS\")\n        df_reporting = _aggregate_hourly_data(df_reporting, \"MS\")\n\n    return df_baseline, df_reporting\n\n\ndef _aggregate_hourly_data(df, agg):\n    df_agg = df.reset_index().set_index(\"datetime\").groupby(\"id\")\n    df_agg_temperature = df_agg[\"temperature\"].resample(\"D\").mean()\n    df_agg_observed = df_agg[\"observed\"].resample(agg).sum()\n\n    if agg == \"MS\":\n        df_agg_observed = df_agg_observed.reindex(df_agg_temperature.index)\n\n    df = pd.concat([df_agg_temperature, df_agg_observed], axis=1)\n    df = df.reset_index().set_index([\"id\", \"datetime\"])\n\n    return df\n\n\ndef _load_other_data(data_type):\n    if data_type == TutorialDataChoice.FEATURES:\n        df = _load_file(\"features.csv\")\n\n    elif data_type == TutorialDataChoice.SEASONAL_HOUR_DAY_WEEK_LOADSHAPE:\n        df = _load_file(\"seasonal_hourly_day_of_week_loadshape.csv\")\n\n    elif data_type == TutorialDataChoice.SEASONAL_DAY_WEEK_LOADSHAPE:\n        df = _load_file(\"seasonal_day_of_week_loadshape.csv\")\n\n    elif data_type == TutorialDataChoice.MONTH_LOADSHAPE:\n        df = _load_file(\"month_loadshape.csv\")\n\n    df = df.set_index(\"id\")\n\n    return df\n\n\ndef _load_file(file: Path | str):\n    if isinstance(file, str):\n        file = data_dir / file\n\n    file_type = None\n    if file.suffix == \".csv\":\n        file_type = \"csv\"\n    elif file.suffix == \".parquet\":\n        file_type = \"parquet\"\n    \n    url = f\"https://raw.githubusercontent.com/{repo_full_name}/{branch}/{path}/{file.name}\"\n\n    if file.exists():\n        data = file\n\n    else:\n        response = requests.get(url)\n        response.raise_for_status()\n        try:\n            with open(file, \"wb\") as f:\n                f.write(response.content)\n\n            data = file\n            raise Exception(\"I dunno\")\n\n        except:\n            data = BytesIO(response.content)\n            print(f\"Warning: Could not write file {file}. Ensure the directory exists and you have write permissions.\")\n\n    try:\n        if file_type == \"csv\":\n            df = pd.read_csv(data)\n\n        elif file_type == \"parquet\":\n            df = pd.read_parquet(data, engine=\"pyarrow\")\n\n            # Read the Parquet file into a PyArrow Table\n            # table = pq.read_table(file)\n            # df = table.to_pandas()\n\n    except Exception as e:\n        print(f\"Error loading file {file}: {e}\")\n        raise e\n\n    return df\n\n\nif __name__ == \"__main__\":\n    df = load_test_data(\"hourly_treatment_data\")\n    print(df.index.get_level_values(0).nunique())\n    print(df.head())\n"
  },
  {
    "path": "opendsm/common/utils.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\n\nimport numba\nimport numpy as np\nimport pandas as pd\nfrom numba.extending import overload\n\n\n\nMIN_POS_SYSTEM_VALUE = (np.finfo(float).tiny * (1e20)) ** (1 / 2)\nMAX_POS_SYSTEM_VALUE = (np.finfo(float).max * (1e-20)) ** (1 / 2)\nLN_MIN_POS_SYSTEM_VALUE = np.log(MIN_POS_SYSTEM_VALUE)\nLN_MAX_POS_SYSTEM_VALUE = np.log(MAX_POS_SYSTEM_VALUE)\n\n\n@overload(np.clip)\ndef np_clip(a, a_min, a_max):\n    \"\"\"\n    This function applies a clip operation on the input array 'a' using the provided minimum and maximum values.\n    The clip operation ensures that all elements in 'a' are within the range [a_min, a_max].\n    If an element in 'a' is less than 'a_min', it is replaced with 'a_min'.\n    If an element in 'a' is greater than 'a_max', it is replaced with 'a_max'.\n    NaN values in 'a' are preserved as NaN.\n\n    Parameters:\n    a (numpy array): The input array to be clipped.\n    a_min (float): The minimum value for the clip operation.\n    a_max (float): The maximum value for the clip operation.\n\n    Returns:\n    numpy array: The clipped array.\n    \"\"\"\n\n    @numba.vectorize\n    def _clip(a, a_min, a_max):\n        \"\"\"\n        This is a vectorized implementation of the clip function.\n        It applies the clip operation on each element of the input array 'a'.\n\n        Parameters:\n        a (float): The input value to be clipped.\n        a_min (float): The minimum value for the clip operation.\n        a_max (float): The maximum value for the clip operation.\n\n        Returns:\n        float: The clipped value.\n        \"\"\"\n\n        if np.isnan(a):\n            return np.nan\n        elif a < a_min:\n            return a_min\n        elif a > a_max:\n            return a_max\n        else:\n            return a\n\n    def clip_impl(a, a_min, a_max):\n        \"\"\"\n        This is a numba implementation of the clip function.\n        It applies the clip operation on the input array 'a' using the provided minimum and maximum values.\n\n        Parameters:\n        a (numpy array): The input array to be clipped.\n        a_min (float): The minimum value for the clip operation.\n        a_max (float): The maximum value for the clip operation.\n\n        Returns:\n        numpy array: The clipped array.\n        \"\"\"\n\n        return _clip(a, a_min, a_max)\n\n    return clip_impl\n\n\ndef to_np_array(x):\n    \"\"\"\n    This function converts the input value 'x' to a numpy array.\n\n    Parameters:\n    x [int, float, array]: The input value to be converted to a numpy array.\n\n    Returns:\n    numpy array: The converted numpy array.\n    \"\"\"\n    if x is None:\n        return None\n\n    if not hasattr(x, \"__len__\"):\n        x = [x]\n\n    if not isinstance(x, np.ndarray):\n        x = np.array(x)\n\n    # if ndim is 0 then convert to 1D array\n    if x.ndim == 0:\n        x = np.array([x])\n\n    return np.array(x)\n\n\ndef safe_divide(num, den, min_denominator=1e-3, return_all=True):\n    \"\"\"\n    Safely divide numerator by denominator, returning np.nan for invalid cases.\n    Works on scalars or numpy arrays.\n\n    Parameters:\n    numerator: scalar or array-like\n    denominator: scalar or array-like\n    min_denominator: float, minimum allowed denominator\n\n    Returns:\n    result: scalar or numpy array, or np.nan where division is unsafe\n    \"\"\"\n    min_den = min_denominator\n\n    input_num_type = type(num)\n    input_den_type = type(den)\n\n    num = np.asarray(num)\n    den = np.asarray(den)\n\n    # Create mask for invalid denominators\n    invalid_mask = (den == 0) | ((den <= min_den) & (num > 10 * min_den))\n\n    # Prepare result array\n    # Determine the maximum shape that can broadcast num and den\n    result_shape = np.broadcast(num, den).shape\n    result = np.empty(result_shape, dtype=np.float64)\n\n    # Where valid, perform division\n    valid_mask = ~invalid_mask\n    # Use numpy errstate to suppress divide by zero warnings and replace with nan\n    with np.errstate(divide='ignore', invalid='ignore'):\n        if num.ndim > 0 and den.ndim > 0:\n            temp_result = np.divide(num[valid_mask], den[valid_mask])\n            temp_result[np.isinf(temp_result)] = np.nan\n            result[valid_mask] = temp_result\n        elif num.ndim > 0 and den.ndim == 0:\n            temp_result = np.divide(num[valid_mask], den)\n            temp_result[np.isinf(temp_result)] = np.nan\n            result[valid_mask] = temp_result\n        elif num.ndim == 0 and den.ndim > 0:\n            temp_result = np.divide(num, den[valid_mask])\n            temp_result[np.isinf(temp_result)] = np.nan\n            result[valid_mask] = temp_result\n        else:\n            temp_result = np.divide(num, den)\n            if np.isinf(temp_result):\n                temp_result = np.nan\n            result[valid_mask] = temp_result\n\n    # Where invalid, set to np.nan\n    result[invalid_mask] = np.nan\n\n    # replace any non-finite values with np.nan\n    result = np.where(np.isfinite(result), result, np.nan)\n\n    # If input was scalar, return scalar\n    if input_num_type not in (np.ndarray, list, pd.Series) and input_den_type not in (np.ndarray, list, pd.Series):\n        if invalid_mask:\n            return np.nan\n        else:\n            return float(result)\n\n    if return_all:\n        return result\n    else:\n        return result[~invalid_mask]\n\n\ndef OoM(x, method=\"round\"):\n    if not isinstance(x, np.ndarray):\n        x = np.array(x)\n\n    return OoM_numba(x, method=method)\n\n\n@numba.jit(nopython=True, cache=True)\ndef OoM_numba(x, method=\"round\"):\n    \"\"\"\n    This function calculates the order of magnitude (OoM) of each element in the input array 'x' using the specified method.\n\n    Parameters:\n    x (numpy array): The input array for which the OoM is to be calculated.\n    method (str): The method to be used for calculating the OoM. It can be one of the following:\n                  \"round\" - round to the nearest integer (default)\n                  \"floor\" - round down to the nearest integer\n                  \"ceil\" - round up to the nearest integer\n                  \"exact\" - return the exact OoM without rounding\n\n    Returns:\n    x_OoM (numpy array): The array of the same shape as 'x' containing the OoM of each element in 'x'.\n    \"\"\"\n\n    x_OoM = np.empty_like(x)\n    for i, xi in enumerate(x):\n        if xi == 0.0:\n            x_OoM[i] = 1.0\n\n        elif method.lower() == \"floor\":\n            x_OoM[i] = np.floor(np.log10(np.abs(xi)))\n\n        elif method.lower() == \"ceil\":\n            x_OoM[i] = np.ceil(np.log10(np.abs(xi)))\n\n        elif method.lower() == \"round\":\n            x_OoM[i] = np.round(np.log10(np.abs(xi)))\n\n        else:  # \"exact\"\n            x_OoM[i] = np.log10(np.abs(xi))\n\n    return x_OoM\n\n\ndef RoundToSigFigs(x, p):\n    \"\"\"\n    This function rounds the input array 'x' to 'p' significant figures.\n\n    Parameters:\n    x (numpy.ndarray): The input array to be rounded.\n    p (int): The number of significant figures to round to.\n\n    Returns:\n    numpy.ndarray: The rounded array.\n    \"\"\"\n\n    x = np.asarray(x)\n    x_positive = np.where(np.isfinite(x) & (x != 0), np.abs(x), 10 ** (p - 1))\n    mags = 10 ** (p - 1 - OoM(x_positive))\n    return np.round(x * mags) / mags\n\n\ndef sigmoid(x, x0=0, k=1):\n    # https://stackoverflow.com/questions/51976461/optimal-way-of-defining-a-numerically-stable-sigmoid-function-for-a-list-in-pyth\n    \n    def _positive_sigmoid(x):\n        return 1 / (1 + np.exp(-x))\n\n    def _negative_sigmoid(x):\n        # Cache exp so you won't have to calculate it twice\n        exp = np.exp(x)\n\n        return exp / (exp + 1)\n\n    x = np.asarray(x, dtype=float)\n\n    if callable(k):\n        k = k(x, x0)\n\n        if np.any(k <= 0):\n            raise ValueError(\"k parameter must be non-negative and non-zero\")\n\n    x = (x - x0) / k\n\n    positive = x >= 0\n    # Boolean array inversion is faster than another comparison\n    negative = ~positive\n\n    # empty contains junk hence will be faster to allocate\n    # Zeros has to zero-out the array after allocation, no need for that\n    # See comment to the answer when it comes to dtype\n    res = np.empty_like(x, dtype=float)\n    res[positive] = _positive_sigmoid(x[positive])\n    res[negative] = _negative_sigmoid(x[negative])\n\n    return res\n\ndef log_cosh(x):\n    # log(cosh(x)) = log(e^x + e^-x) - log(2).\n    # For x > 0, we can rewrite this as x + log(1 + e^(-2 * x)) - log(2).\n    # The second term will be small when x is large, so we don't get any large\n    # cancellations.\n    # Similarly for x < 0, we can rewrite the expression as -x + log(1 + e^(2 *\n    # x)) - log(2)\n    # This gives us abs(x) + softplus(-2 * abs(x)) - log(2)\n\n    # For x close to zero, we can write the taylor series of softplus(\n    # -2 * abs(x)) to see that we get;\n    # log(2) - abs(x) + x**2 / 2. - x**4 / 12 + x**6 / 45. + O(x**8)\n    # We can cancel out terms to get:\n    # x ** 2 / 2.  * (1. - x ** 2 / 6) + x ** 6 / 45. + O(x**8)\n    # For x < 45 * sixthroot(smallest normal), all higher level terms\n    # disappear and we can use the above expression.\n    #\n    # to calculate taylor series coefficients, we can use the formula:\n    # from scipy.special import zeta\n    # n = 1\n    # 1/((-1)**(n-1) * (2**(2*n) - 1)*np.abs(zeta(2*n)) / (n * np.pi**(2*n)))\n    \n    # Handle scalar inputs\n    isscalar = False\n    if np.isscalar(x):\n        isscalar = True\n        x = np.array([x])\n\n    # Convert integer types to float types\n    if np.issubdtype(x.dtype, np.integer):\n        precision = np.iinfo(x.dtype).bits\n        x = x.astype(np.dtype(f'float{precision}'))\n    \n    # Set bounds for taylor series approximation based on data type\n    if x.dtype == np.float16:\n        bound = 5.5E-2\n    elif x.dtype == np.float32:\n        bound = 1E-1\n    elif x.dtype == np.float64:\n        bound = 8E-9\n    elif x.dtype == np.float128:\n        bound = 1E-8\n    else:\n        bound = 45 * np.power(np.finfo(x.dtype).tiny, 1 / 6.)\n\n    abs_x = np.abs(x)\n\n    idx_taylor = abs_x <= bound\n    idx_logcosh = ~idx_taylor\n\n    res = np.empty_like(x)\n\n    # For small x, log(cosh(x)) = x**2 / 2 - x**4 / 12 + x**6 / 45 - ...\n    x_t = x[idx_taylor]\n    res[idx_taylor] = x_t**2 / 2. - x_t**4 / 12. + x_t**6 / 45. - x_t**8 / 148.23529411764702 + x_t**10 / 457.2580645161289\n\n    # for large x, use logcosh\n    _abs_x = abs_x[idx_logcosh]\n    res[idx_logcosh] = _abs_x + np.log1p(np.exp(-2 * _abs_x)) - np.log(2)\n\n    if isscalar:\n        return res[0]\n\n    return res"
  },
  {
    "path": "opendsm/comparison_groups/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.cg_clustering import *\nfrom opendsm.comparison_groups.individual_meter_matching import *\n\nfrom opendsm.comparison_groups.stratified_sampling import (\n    Stratified_Sampling,\n    SS_Settings, \n    DSS_Settings\n)\n\nfrom opendsm.comparison_groups.random_sampling import (\n    Random_Sampling,\n    RS_Settings,\n)\n\nfrom opendsm.comparison_groups.common import (\n    Data,\n    Data_Settings,\n    load_tutorial_data,\n)\n"
  },
  {
    "path": "opendsm/comparison_groups/archived_gridmeter_changelog.md",
    "content": "Changelog\n=========\n\nDevelopment\n-----------\n\n* Placeholder\n\n1.1.0\n-----\n\n* Add usage-pattern distance calculation as option for comparison group methods selection.\n\n1.0.1\n-----\n\n* Add registered trademark\n\n1.0.0\n-----\n\n* Official release as GRIDmeter\n* Complete renaming\n\n\n0.10.1\n------\n\n* Update description\n\n0.10.0\n------\n\n* Rename to 'gridmeter' -- final release as eesampling\n\n0.9.1\n-----\n\n* Add comparison pool equivalence to results_as_json().\n\n0.9.0\n-----\n\n* Refactor equivalence calculation for bin selection (much faster)\n* Change input format for equivalence in bin selection\n\n0.8.0\n-----\n\n* Add synthetic data generation for testing and tutorials\n* Add tutorial Jupyter notebook\n* Rename Diagnostics --> StratifiedSamplingDiagnostics\n* Expose all classes for top-level imports.\n* Made adjustment to how n_samples_approx is calculated. It now works where the minimum sampled:treatment ratio can be violated if n_samples_approx is used as an upper bound and that upper bound is reached.\n\n0.7.0\n-----\n\n* Rename train --> treatment, test --> pool \n\n0.6.1\n-----\n\n* Fix Github URL\n\n0.6.0\n-----\n\n* First public release \n* Update default params for bin_selection.StratifiedSamplingBinSelector(...) so n_samples_approx = 5000 and relax_n_samples_approx_constraint=False and min_n_sampled_to_n_train_ratio = 0.25, which means that we aim for 5000 comparison group meters but if we can't reach it, we need at least 0.25 sample to train ratio or else it fails. \n* Add relax_n_samples_approx constraint so that you can use n_samples_approx as upper bound rather than a target.\n* Refactor results_as_json a bit so selected sample output is cleaner.\n\n0.5.5\n-----\n\n* Update results serialization.\n\n0.5.4\n-----\n\n* Add kwargs and results serialization.\n\n0.5.3\n-----\n\n* Separate bin selection into a different class.\n\n0.5.2\n-----\n\n* Fix issue with naming during equivalence chisquare checking of diagnostics (this needs to be refactored later).\n\n0.5.1\n-----\n\n* Renamed `min_bin_size` to `min_n_train_per_bin`.\n* Move BinnedData to bins.py.\n* Added chisquared equivalence option.\n* Add equivalence via a separate dataframe.\n\n0.5.0\n-----\n\n* Added some unit tests for modelling and some test framework.\n* Generalized Diagnostics so that .plot_equivalence(...) can also plot the comparison pool.\n* Changed automatic n_samples_approx to use the maximum number of samples available (based on how many test values are in the \"worst\" bin) rather than use binary search.\n* Renamed n_outputs to n_samples_approx\n\n0.4.2\n-----\n\n* Fix random seed so that numpy random seeding for pertubation is happening in the right place.\n* Make a copy of the dataframe in the `_perturb()` function.\n\n0.4.1\n-----\n\n* Add random seed option.\n\n0.4.0\n-----\n\n* Support fixed-width or variable-with bins\n* Auto-choose number of outputs via binary search\n\n0.3.3\n-----\n\n* Scatter plot has fixed y scales and correct size\n\n\n0.3.2\n-----\n\n* Fix bug if not using auto-bin\n\n0.3.1\n-----\n\n* Remove plotly dependency\n\n0.3.0\n-----\n\n* Simplify plotting\n* Add auto_bin option\n\n\n0.2.0\n-----\n\n* Big refactor, add plotting diagnostics.\n* Add plotly support\n\n0.1.0\n-----\n\n* Initial create of model.\n\n0.0.1\n-----\n\n* Initial creation of library.\n"
  },
  {
    "path": "opendsm/comparison_groups/cg_clustering/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.cg_clustering.create_comparison_groups import CG_Clustering\nfrom opendsm.comparison_groups.cg_clustering.settings import CG_Clustering_Settings\n"
  },
  {
    "path": "opendsm/comparison_groups/cg_clustering/bounds.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\n\n\ndef _get_num_cluster_min(\n    data_size: int, \n    min_cluster_size: int, \n    num_cluster_bound_lower: int\n):\n    \"\"\"\n    returns lower bounds using data_size which is number models in cluster\n    \"\"\"\n\n    linear = False\n\n    # assume we want 8 clusters of min size 15 meters with a pool of 1000 meters\n    base_pool = 1000\n    base_cluster_size = 15\n    base_min_clusters = 8\n\n    if linear:\n        k = (base_cluster_size*base_min_clusters)/base_pool\n        num_cluster_min = k*data_size/min_cluster_size\n    \n    else:\n        k = 30 + 4.58*np.exp(data_size/335)\n        num_cluster_min = k/min_cluster_size\n\n    n = max(int(np.floor(num_cluster_min)), 2)\n    n = min(num_cluster_bound_lower, n)\n\n    return n\n\n\ndef _get_num_cluster_max(\n    data_size: int, \n    min_cluster_size: int, \n    num_cluster_bound_upper: int\n):\n    \"\"\"\n    returns upper bounds using data_size which is number models in cluster\n    \"\"\"\n    n_min = min_cluster_size\n\n    min_clusters = 1\n    max_clusters = num_cluster_bound_upper\n\n    # assume we want 250 with a size of 1000\n    n_set = 1000\n    n_max_set = 250\n\n    k = (n_set - n_min) * (\n        np.log(\n            (\n                ((n_max_set - min_clusters) / (2 * max_clusters - min_clusters) + 0.5)\n                ** -1\n                - 1\n            )\n            ** -1\n        )\n    ) ** -1\n\n    if not np.isfinite(k):\n        \"\"\"\n        TODO: Figure out better way to handle this.\n        Currently occurs when num_cluster_bound_upper is less than n_max_set\n        \"\"\"\n        return min(data_size, num_cluster_bound_upper)\n\n    num_cluster_max = (2 * max_clusters - min_clusters) * (\n        1 / (1 + np.exp(-(data_size - n_min) / k)) - 0.5\n    ) + min_clusters\n\n    n = max(int(np.floor(num_cluster_max)), 2)\n\n    return n\n\n\ndef get_cluster_bounds(\n    data_size: int,\n    min_cluster_size: int,\n    num_cluster_bound_lower: int,\n    num_cluster_bound_upper: int,\n):\n    \"\"\"\n    function which returns lower and upper bound based off config values and number of data points\n    \"\"\"\n\n    num_cluster_min = _get_num_cluster_min(\n        data_size=data_size,\n        min_cluster_size=min_cluster_size,\n        num_cluster_bound_lower=num_cluster_bound_lower,\n    )\n\n    num_cluster_max = _get_num_cluster_max(\n        data_size=data_size,\n        min_cluster_size=min_cluster_size,\n        num_cluster_bound_upper=num_cluster_bound_upper,\n    )\n\n    num_cluster_bounds = sorted([num_cluster_min, num_cluster_max])\n\n    if num_cluster_bounds[0] == num_cluster_bounds[1]:\n        num_cluster_bounds[1] += 1\n\n    return num_cluster_bounds[0], num_cluster_bounds[1]"
  },
  {
    "path": "opendsm/comparison_groups/cg_clustering/create_comparison_groups.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\nfrom typing import Optional\n\nimport pandas as pd\n\nfrom opendsm.comparison_groups.common.base_comparison_group import Comparison_Group_Algorithm\nfrom opendsm.comparison_groups.cg_clustering import settings as _settings\nimport opendsm.comparison_groups.cg_clustering.bounds as _bounds\nfrom opendsm.comparison_groups.cg_clustering import treatment_fit as _treatment_fit \nfrom opendsm.common.clustering.cluster import cluster_features as _cluster\n\n\nclass CG_Clustering(Comparison_Group_Algorithm):\n    clusters = None\n    comparison_pool_loadshape = None\n    treatment_loadshape = None\n\n    def __init__(self, settings: Optional[_settings.CG_Clustering_Settings] = None):\n        if settings is None:\n            settings = _settings.CG_Clustering_Settings()\n\n        self.settings = settings\n\n    def get_labels(self, comparison_pool_data):\n        self.comparison_pool_data = comparison_pool_data\n        self.comparison_pool_loadshape = comparison_pool_data.loadshape\n\n        # update cluster count\n        algo = f\"{self.settings.algorithm_selection.value}\"\n        algo_settings = getattr(self.settings, algo)\n\n        n_cluster_min, n_cluster_max = _bounds.get_cluster_bounds(\n            data_size=len(self.comparison_pool_data.ids),\n            min_cluster_size=algo_settings.scoring.min_cluster_size,\n            num_cluster_bound_lower=algo_settings.n_cluster.lower,\n            num_cluster_bound_upper=algo_settings.n_cluster.upper\n        )\n\n        settings_dict = self.settings.model_dump()\n        settings_dict[algo][\"n_cluster\"][\"lower\"] = n_cluster_min\n        settings_dict[algo][\"n_cluster\"][\"upper\"] = n_cluster_max\n        self.settings = _settings.CG_Clustering_Settings(**settings_dict)\n\n        # perform clustering\n        labels = _cluster(\n            self.comparison_pool_loadshape.copy(), # copy is only necessary for plotting later\n            self.settings\n        )\n\n        self.clusters = pd.DataFrame(\n            {\"cluster\": labels}, \n            index=self.comparison_pool_data.ids\n        )\n        self.clusters.index.name = \"id\"\n\n        return self.clusters\n\n    def match_treatment_to_clusters(self, treatment_data):\n        if self.clusters is None:\n            raise ValueError(\n                \"Comparison group has been not been clustered. Run 'get_labels' first.\"\n            )\n        \n        self.treatment_data = treatment_data\n        self.treatment_ids = treatment_data.ids\n        self.treatment_loadshape = treatment_data.loadshape\n\n        self.treatment_weights = _treatment_fit.match_treatment_to_clusters(\n            self.treatment_loadshape,\n            self.comparison_pool_loadshape,\n            self.clusters,\n            settings=self.settings\n        )\n\n        return self.treatment_weights\n\n    def get_comparison_group(self, treatment_data, comparison_pool_data):\n        df_cg = self.get_labels(comparison_pool_data)\n        df_t_coeffs = self.match_treatment_to_clusters(treatment_data)\n\n        return df_cg, df_t_coeffs\n"
  },
  {
    "path": "opendsm/comparison_groups/cg_clustering/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\nfrom typing import Optional, Union\n\nimport pydantic\n\nfrom opendsm.common.base_settings import BaseSettings\nfrom opendsm.common.clustering import settings as _settings\nfrom opendsm.common import const as _const\n\nfrom opendsm.common.stats.adaptive_loss import LOSS_ALPHA_MIN as _LOSS_ALPHA_MIN\n\n\n\nclass AdaptiveLossChoice(str, Enum):\n    SSE = \"sse\"\n    MAE = \"mae\"\n    L2 = \"l2\"\n    L1 = \"l1\"\n    ADAPTIVE = \"adaptive\"\n\n\nclass TreatmentMatchSettings(BaseSettings):\n    \"\"\"aggregation type for loadshape\"\"\"\n    agg_type: _settings.AggregateMethod = pydantic.Field(\n        default=_settings.AggregateMethod.MEDIAN\n    )\n\n    \"\"\"treatment meter match loss type\"\"\"\n    adaptive_loss_alpha: Union[AdaptiveLossChoice, float] = pydantic.Field(\n        default=AdaptiveLossChoice.MAE\n    )\n\n    adaptive_loss_sigma: float = pydantic.Field(\n        default=2.698,  # 1.5 IQR\n        gt= 0.0,\n    )\n\n    adaptive_loss_c_algo: _const.CAlgoChoice = pydantic.Field(\n        default=_const.CAlgoChoice.IQR\n    )\n\n    percent_cluster_minimum: float = pydantic.Field(\n        default=1E-6,\n        ge=0.0,\n    )\n\n    \"\"\"Check if valid settings for treatment meter match loss\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _check_treatment_match_loss(self):\n        self._adaptive_loss_alpha = self.adaptive_loss_alpha\n\n        if isinstance(self._adaptive_loss_alpha, str):\n            if self._adaptive_loss_alpha == \"adaptive\":\n                pass\n\n            elif self._adaptive_loss_alpha in [\"sse\", \"l2\"]:\n                self._adaptive_loss_alpha = 2.0\n\n            elif self._adaptive_loss_alpha in [\"mae\", \"l1\"]:\n                self._adaptive_loss_alpha = 1.0\n                \n            else:\n                raise ValueError(\"`treatment_match_loss` must be either ['SSE', 'MAE', 'L2', 'L1', 'adaptive'] or float\")\n            \n        else:\n            if self._adaptive_loss_alpha < _LOSS_ALPHA_MIN:\n                raise ValueError(f\"`treatment_match_loss` must be greater than {_LOSS_ALPHA_MIN:.0f}\")\n\n            if self._adaptive_loss_alpha > 2:\n                raise ValueError(\"`treatment_match_loss` must be less than 2\")\n\n        return self\n\n\nclass _CG_Clustering_Settings(_settings.ClusteringSettings):\n    treatment_match: TreatmentMatchSettings = pydantic.Field(\n        default=TreatmentMatchSettings(),\n    )\n\n\nclass ClusteringSettings(BaseSettings):\n    pass\n\n\ndef CG_Clustering_Settings(**kwargs) -> _CG_Clustering_Settings:\n    default_dict = {\n        \"normalize\": {\n            \"method\": _settings.NormalizeChoice.MIN_MAX_QUANTILE,\n            \"quantile\": 0.1,\n            \"pre_transform\": True,\n            \"post_transform\": False,\n            \"axis\": 1,\n        },\n        \"transform_selection\": _settings.TransformChoice.FPCA,\n        \"fpca_transform\": {\n            \"min_var_ratio\": 0.97,\n        },\n        \"algorithm_selection\": _settings.ClusterAlgorithms.BISECTING_KMEANS,\n        \"bisecting_kmeans\": {\n            \"recluster_count\": 3,\n            \"internal_recluster_count\": 5,\n            \"inner_algorithm\": _settings.BiKmeansInnerAlgorithms.ELKAN,\n            \"bisecting_strategy\": _settings.BiKmeansBisectingStrategies.LARGEST_CLUSTER,\n            \"n_cluster\": {\n                \"lower\": 8,\n                \"upper\": 1500,\n            },\n            \"scoring\": {\n                \"min_cluster_size\": 15,\n                \"max_non_outlier_cluster_count\": 200,\n                \"calinski_harabasz_weight\": 1.0,\n                \"davies_bouldin_weight\": 0.0,\n                \"density_based_clustering_validation_weight\": 0.0,\n                \"dunn_weight\": 0.0,\n                \"silhouette_weight\": 0.0,\n                \"silhouette_median_weight\": 0.0,\n                \"xie_beni_weight\": 0.0,\n                \"distance_metric\": _settings.DistanceMetric.EUCLIDEAN,\n            },\n        },\n        \"cluster_sort\": {\n            \"enable\": False,\n            \"method\": _settings.SortMethod.SIZE,\n            \"aggregation\": _settings.AggregateMethod.MEAN,\n            \"reverse\": False,\n        },\n        \"seed\": 42,\n    }\n\n    # Update default_dict with any provided keyword arguments\n    default_dict.update(kwargs)\n\n    return _CG_Clustering_Settings(**default_dict)\n\n\nif __name__ == \"__main__\":\n    s = CG_Clustering_Settings()\n\n    print(s.model_dump_json())"
  },
  {
    "path": "opendsm/comparison_groups/cg_clustering/treatment_fit.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\n\"\"\"\nfunctions for dealing with fitting to clusters\n\"\"\"\n\nimport scipy.optimize\nimport scipy.spatial.distance\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.common.clustering import transform as _transform\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.comparison_groups.cg_clustering import settings as _settings\n\n\n\ndef _get_cluster_ls(df_cp_ls: pd.DataFrame, cluster_df: pd.DataFrame, agg_type: str):\n    \"\"\"\n    original cp loadshape and cluster df\n    settings for agg_type\n    \"\"\"\n\n    cluster_df = cluster_df.join(df_cp_ls, on=\"id\")\n    cluster_df = cluster_df.reset_index().set_index([\"id\", \"cluster\"])  # type: ignore\n\n    # calculate cp_df\n    df_cluster_ls = cluster_df.groupby(\"cluster\").agg(agg_type)  # type: ignore\n    df_cluster_ls = df_cluster_ls[df_cluster_ls.index.get_level_values(0) > -1]  # don't match to outlier cluster\n\n    return df_cluster_ls\n\n\ndef fit_to_clusters(\n    t_ls, \n    cp_ls, \n    x0, \n    settings_dict,\n):\n    # instantiate settings from settings_dict\n    settings = _settings.CG_Clustering_Settings(**settings_dict)\n    match_settings = settings.treatment_match\n\n    _min_pct_cluster = match_settings.percent_cluster_minimum\n\n    def _remove_small_x(x: np.ndarray):\n        # remove small values and normalize to 1\n        x[x < _min_pct_cluster] = 0\n        x /= np.sum(x)\n\n        return x\n\n    def obj_fcn_dec(t_ls, cp_ls, idx=None):\n        if idx is not None:\n            cp_ls = cp_ls[idx, :]\n\n        def obj_fcn(x):\n            x = _remove_small_x(x)\n            resid = (t_ls - (cp_ls * x[:, None]).sum(axis=0)).flatten()\n\n            if match_settings._adaptive_loss_alpha == 2:\n                wSSE = np.sum(resid**2)\n\n            else:\n                weight, _, _ = adaptive_weights(\n                    resid, \n                    alpha=match_settings._adaptive_loss_alpha, \n                    sigma=match_settings.adaptive_loss_sigma, \n                    quantile=0.25,\n                    min_weight=0.0,\n                    C_algo=match_settings.adaptive_loss_c_algo,\n                ) # type: ignore\n\n                wSSE = np.sum(weight * resid**2)\n\n            return wSSE\n\n        return obj_fcn\n\n    def sum_to_one(x):\n        zero = np.sum(x) - 1\n        return zero\n\n    x0 = np.array(x0).flatten()\n\n    # only optimize if >= _MIN_PCT_CLUSTER\n    idx = np.argwhere(x0 >= _min_pct_cluster).flatten()\n    if len(idx) == 0:\n        idx = np.arange(0, len(x0))\n\n    x0_n = x0[idx]\n\n    bnds = np.repeat(np.array([0, 1])[:, None], x0_n.shape[0], axis=1).T\n    const = [{\"type\": \"eq\", \"fun\": sum_to_one}]\n\n    res = scipy.optimize.minimize(\n        obj_fcn_dec(t_ls, cp_ls, idx),\n        x0_n,\n        bounds=bnds,\n        constraints=const,\n        method=\"SLSQP\",\n    )  # trust-constr, SLSQP\n    # res = minimize(obj_fcn, x0, bounds=bnds, method='SLSQP') # trust-constr, SLSQP, L-BFGS-B\n    # res = differential_evolution(obj_fcn, bnds, maxiter=100)\n    # res = basinhopping(obj_fcn, x0, niter=10, minimizer_kwargs={'bounds': bnds, 'method': 'Powell'})\n\n    x = np.zeros_like(x0)\n    x[idx] = _remove_small_x(res.x)\n\n    return x\n\n\nclass ClusterTreatmentMatchError(Exception):\n    pass\n\n\ndef _match_treatment_to_cluster(\n    df_ls_t: pd.DataFrame, \n    df_ls_cluster: pd.Series, \n    settings: _settings.Settings\n):\n    # Create null dataframe\n    coeffs = np.empty((df_ls_t.shape[0], df_ls_cluster.shape[0]))\n    t_ids = df_ls_t.index\n    columns = [f\"pct_cluster_{int(n)}\" for n in df_ls_cluster.index]\n    df_t_coeffs = pd.DataFrame(coeffs, index=t_ids, columns=columns)\n\n    # error checking going into cdist\n    if df_ls_t.shape[0] == 0:\n        raise ClusterTreatmentMatchError(\"No valid treatment loadshapes\")\n    \n    if df_ls_cluster.shape[0] == 0:\n        raise ClusterTreatmentMatchError(\"No valid cluster loadshapes\")\n    \n    if df_ls_t.shape[1] != df_ls_cluster.shape[1]:\n        shape_str = f\"Treatment[{df_ls_t.shape[1]}] != Cluster[{df_ls_cluster.shape[1]}]\"\n        raise ClusterTreatmentMatchError(f\"Treatment and cluster loadshapes have different lengths: {shape_str}\")\n\n    # identify invalid rows\n    idx_invalid = df_ls_t.isnull().any(axis=1) | ~np.isfinite(df_ls_t).any(axis=1)\n    idx_valid = ~idx_invalid\n\n    # convert to numpy\n    t_ls = df_ls_t.to_numpy()\n    cp_ls = df_ls_cluster.to_numpy()\n\n    # filter to valid rows\n    t_ls = t_ls[idx_valid, :]\n\n    # Get percent from each cluster\n    distances = scipy.spatial.distance.cdist(t_ls, cp_ls, metric=\"euclidean\")  # type: ignore\n    distances_norm = (np.min(distances, axis=1) / distances.T).T\n    # change this number (20) to alter weights, larger centralizes the weight, smaller spreads them out\n    distances_norm = (distances_norm**20)  \n    distances_norm = (distances_norm.T / np.sum(distances_norm, axis=1)).T\n\n    coeffs = []\n    for n, t_id in enumerate(df_ls_t.index):\n        t_id_ls = t_ls[n, :]\n        x0 = distances_norm[n, :]\n\n        coeffs_n = fit_to_clusters(t_id_ls, cp_ls, x0, settings.model_dump())\n\n        coeffs.append(coeffs_n)\n\n    coeffs = np.vstack(coeffs)\n\n    # only update rows\n    df_t_coeffs.loc[idx_invalid, :] = np.nan\n    df_t_coeffs.loc[idx_valid, :] = coeffs\n\n    return df_t_coeffs\n\n\ndef match_treatment_to_clusters(\n    df_ls_t: pd.DataFrame,\n    df_ls_cluster: pd.DataFrame,\n    df_cluster: pd.DataFrame,\n    settings: _settings._CG_Clustering_Settings,\n):\n    \"\"\"\n    performs the matching logic to a provided treatment_loadshape dataframe\n\n    TODO: Handle call when no valid scores were found?\n\n    \"\"\"\n\n    # get cluster loadshape and normalize\n    df_ls_cluster_agg = _get_cluster_ls(\n        df_cp_ls=df_ls_cluster,\n        cluster_df=df_cluster,\n        agg_type=settings.treatment_match.agg_type,\n    )\n\n    df_ls_cluster_agg[:] = _transform.normalize(\n        data=df_ls_cluster_agg.to_numpy(),\n        settings=settings.normalize,\n    )\n\n    # normalize treatment loadshape\n    df_ls_t_norm = df_ls_t.copy()\n    df_ls_t_norm[:] = _transform.normalize(\n        data=df_ls_t_norm.to_numpy(),\n        settings=settings.normalize,\n    )\n\n    # fit treatment to clusters\n    df_t_coeffs = _match_treatment_to_cluster(\n        df_ls_t=df_ls_t_norm,\n        df_ls_cluster=df_ls_cluster_agg,\n        settings=settings,\n    )\n\n    return df_t_coeffs"
  },
  {
    "path": "opendsm/comparison_groups/common/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n\n   Copyright 2014-2024 OpenEEmeter contributors\n\n   Licensed under the Apache License, Version 2.0 (the \"License\");\n   you may not use this file except in compliance with the License.\n   You may obtain a copy of the License at\n\n       http://www.apache.org/licenses/LICENSE-2.0\n\n   Unless required by applicable law or agreed to in writing, software\n   distributed under the License is distributed on an \"AS IS\" BASIS,\n   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n   See the License for the specific language governing permissions and\n   limitations under the License.\n\n\"\"\"\n\nfrom opendsm.comparison_groups.common.data import Data\nfrom opendsm.comparison_groups.common.data_settings import Data_Settings\n\nfrom opendsm.comparison_groups.common.tutorial_data import load_tutorial_data"
  },
  {
    "path": "opendsm/comparison_groups/common/base_comparison_group.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\nimport numpy as np\n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\n\n\nclass Comparison_Group_Algorithm:\n    settings = None\n    _loadshape_aggregation = \"mean\"\n\n    treatment_ids = None\n    treatment_loadshape = None\n    treatment_match_loadshape = None\n    comparison_pool_loadshape = None\n\n    clusters = None\n    treatment_weights = None\n    \n\n    def _get_treatment_loadshape(self, id):\n        ls = self.treatment_loadshape.loc[id]\n\n        agg_ls = ls.agg(self._loadshape_aggregation).to_frame().T\n        \n        if len(id) == 1:\n            agg_ls.index = ['Treatment Meter']\n        else:\n            agg_ls.index = ['Treatment Group']\n\n        return agg_ls\n\n    def _set_treatment_match_loadshape(self):\n        pool_ls = self.comparison_pool_loadshape\n        cluster_ls = self.clusters[[\"cluster\"]].join(pool_ls)\n        cluster_ls = cluster_ls.groupby(\"cluster\").agg(self._loadshape_aggregation)\n\n        agg_ls = np.einsum(\"ij,ik->jk\", cluster_ls.loc[0:], self.treatment_weights.T).T\n        agg_ls = pd.DataFrame(agg_ls, columns=pool_ls.columns, index=self.treatment_weights.index)\n\n        return agg_ls\n    \n    def _get_treatment_match_loadshape(self, id):\n        if self.treatment_match_loadshape is None:\n            self.treatment_match_loadshape = self._set_treatment_match_loadshape()\n\n        ls = self.treatment_match_loadshape.loc[id]\n\n        agg_ls = ls.agg(self._loadshape_aggregation).to_frame().T\n        agg_ls.index = ['Comparison Group']\n\n        return agg_ls\n\n    def get_comparison_pool_loadshape(self):\n        ls = self.comparison_pool_loadshape\n\n        agg_ls = ls.agg(self._loadshape_aggregation).to_frame().T\n        agg_ls.index = ['Comparison Pool']\n\n        return agg_ls\n    \n    def get_loadshapes(self, id=None):\n        if id is None:\n            id = self.treatment_data.ids\n        if not isinstance(id, (list, np.ndarray, pd.Series)):\n            id = [id]\n\n        treatment_ls = self._get_treatment_loadshape(id)\n        treatment_match_ls = self._get_treatment_match_loadshape(id)\n        comparison_pool_ls = self.get_comparison_pool_loadshape()\n\n        # concat ls\n        ls = pd.concat([treatment_ls, treatment_match_ls, comparison_pool_ls])\n        ls.columns = [int(col) - 1 for col in ls.columns]\n\n        return ls\n    \n    def _validate_ls_weights(self, weights):\n        if weights is None:\n            return\n        \n        # if weights are all the same then return\n        if len(set(weights)) == 1:\n            return\n\n        if len(weights) != len(self.treatment_loadshape.iloc[0].values):\n            raise ValueError(\"weights must be the same length as the number of columns in the treatment group and comparison pool\")\n        \n        # normalize weights to 1\n        weights = np.array(weights) / np.sum(weights)\n\n        return weights\n\n    def plot_loadshapes(self, id=None):\n        ls = self.get_loadshapes(id=id)\n\n        t_min = ls.T.index[0]\n        t_max = ls.T.index[-1]\n\n        # plot ls\n        fig = plt.figure(figsize=(14, 4), dpi=300)\n        ax = fig.subplots()\n\n        for col in ls.T.columns:\n            ax.plot(ls.T.index, ls.T[col], label=col)\n\n        if (t_max - t_min) % 24 and (t_max - t_min) > 24:\n            ax.xaxis.set_minor_locator(mpl.ticker.AutoMinorLocator(4))\n            ax.set_xticks(np.arange(t_min, t_max, 24))\n\n        ax.set_xlim([t_min, t_max])\n        ax.set_xlabel('Time')\n        ax.set_ylabel('Loadshape')\n        ax.legend()\n\n        plt.close(fig) # prevent displaying immediately\n\n        return fig"
  },
  {
    "path": "opendsm/comparison_groups/common/const.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\n\nfrom opendsm.common.const import (\n    default_season_def,\n    default_weekday_weekend_def,\n)\n\n\nclass DistanceMetric(str, Enum):\n    \"\"\"\n    what distance method to use\n    \"\"\"\n\n    EUCLIDEAN = \"euclidean\"\n    SEUCLIDEAN = \"seuclidean\"\n    MANHATTAN = \"manhattan\"\n    COSINE = \"cosine\"\n\n\nclass AggType(str, Enum):\n    MEAN = \"mean\"\n    MEDIAN = \"median\"\n\n\n\"\"\"data_settings constants\"\"\"\n\n\nclass LoadshapeType(str, Enum):\n    OBSERVED = \"observed\"\n    MODELED = \"modeled\"\n    ERROR = \"error\"\n    MODEL_ERROR = \"error\"  # an alias for ERROR\n\n\nclass TimePeriod(str, Enum):\n    HOUR = \"hour\"\n    DAY_OF_WEEK = \"day_of_week\"\n    DAY_OF_YEAR = \"day_of_year\"\n    HOURLY_DAY_OF_WEEK = \"hourly_day_of_week\"\n    WEEKDAY_WEEKEND = \"weekday_weekend\"\n    HOURLY_WEEKDAY_WEEKEND = \"hourly_weekday_weekend\"\n    MONTH = \"month\"\n    HOURLY_MONTH = \"hourly_month\"\n    SEASONAL_DAY_OF_WEEK = \"seasonal_day_of_week\"\n    SEASONAL_HOURLY_DAY_OF_WEEK = \"seasonal_hourly_day_of_week\"\n    SEASONAL_WEEKDAY_WEEKEND = \"seasonal_weekday_weekend\"\n    SEASONAL_HOURLY_WEEKDAY_WEEKEND = \"seasonal_hourly_weekday_weekend\"\n\n\ndatetime_types = [\"datetime\", \"datetime64\", \"datetime64[ns]\", \"datetimetz\"]\n\n\nseason_num = {\n    \"january\": 1,\n    \"february\": 2,\n    \"march\": 3,\n    \"april\": 4,\n    \"may\": 5,\n    \"june\": 6,\n    \"july\": 7,\n    \"august\": 8,\n    \"september\": 9,\n    \"october\": 10,\n    \"november\": 11,\n    \"december\": 12,\n}\n\n\nweekday_num = {\n    \"monday\": 0,\n    \"tuesday\": 1,\n    \"wednesday\": 2,\n    \"thursday\": 3,\n    \"friday\": 4,\n    \"saturday\": 5,\n    \"sunday\": 6,\n}\n\ntime_period_row_counts = {\n    \"hourly\": 24,\n    \"month\": 12,\n    \"hourly_month\": 24 * 12,\n    \"day_of_week\": 7,\n    \"day_of_year\": 365,\n    \"hourly_day_of_week\": 24 * 7,\n    \"weekday_weekend\": 2,\n    \"hourly_weekday_weekend\": 24 * 2,\n    \"seasonal_day_of_week\": 3 * 7,\n    \"seasonal_hourly_day_of_week\": 3 * 24 * 7,\n    \"seasonal_weekday_weekend\": 3 * 2,\n    \"seasonal_hourly_weekday_weekend\": 3 * 24 * 2,\n}\n\n\nmin_granularity_per_time_period = {\n    # All the values are in minutes\n    \"hourly\": 60,\n    \"month\": 60 * 24 * 28, # this is not used since we can have a different day per month\n    \"hourly_month\": 60,\n    \"day_of_week\": 60 * 24 * 7,\n    \"day_of_year\": 60 * 24 * 7,\n    \"hourly_day_of_week\": 60,\n    \"weekday_weekend\":  60 * 24 * 7,\n    \"hourly_weekday_weekend\": 60,\n    \"seasonal_day_of_week\": 60 * 24 * 7,\n    \"seasonal_hourly_day_of_week\": 60,\n    \"seasonal_weekday_weekend\": 60 * 24 * 7,\n    \"seasonal_hourly_weekday_weekend\": 60,\n}\n\n\"\"\"\n    This list ordering is important for the groupby columns (refer _find_groupby_columns in data_processing.py)\n    The sorting is done on the basis of this ordering in the final dataframe. First the dataframe is sorted by season, then by day_of_week, then by hour in the seasonal_hourly_day_of_week case.\n    Similarly for other combinations.\n\"\"\"\nunique_time_periods = [\n    \"season\",\n    \"month\",\n    \"day_of_week\",\n    \"day_of_year\",\n    \"weekday_weekend\",\n    \"hour\",\n]\n"
  },
  {
    "path": "opendsm/comparison_groups/common/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom copy import deepcopy\nfrom typing import Optional\n\nfrom opendsm.comparison_groups.common.data_settings import Data_Settings\nfrom opendsm.comparison_groups.common import const as _const\nimport pandas as pd\nimport numpy as np\n\n\ndef is_datetime(x: pd.Series) -> bool:\n    is_dt = [\n        pd.api.types.is_datetime64_any_dtype(x),\n        # pd.api.types.is_datetime64_ns_dtype(x),\n        # pd.api.types.is_datetime64_dtype(x),\n        # isinstance(x.dtype, pd.DatetimeTZDtype)\n    ]\n\n    return any(is_dt)\n\n\nclass Data:\n    def __init__(self, \n        loadshape_df: Optional[pd.DataFrame] = None, \n        time_series_df: Optional[pd.DataFrame] = None, \n        features_df: Optional[pd.DataFrame] = None, \n        settings: Optional[Data_Settings] = None\n    ):\n        if settings is None:\n            if loadshape_df is None:\n                settings = Data_Settings()\n            else: # if loadshape is provided, then apply appropriate settings\n                settings = Data_Settings(agg_type=None, loadshape_type=None, time_period=None)\n\n        self._settings = settings\n\n        self._loadshape = None\n        self._features = None\n\n        # TODO: let's make id the index for the excluded ids dataframe\n        self._excluded_ids = pd.DataFrame(columns=[\"id\", \"reason\"])\n\n        # basic error checking\n        if loadshape_df is None and time_series_df is None and features_df is None:\n            raise ValueError(\n                \"A loadshape, time series, or features dataframe must be provided.\"\n            )\n\n        elif loadshape_df is not None and time_series_df is not None:\n            raise ValueError(\n                \"Both loadshape dataframe and time series dataframe are provided. Please provide only one.\"\n            )\n\n        if self._settings.time_period is not None and (loadshape_df is not None or time_series_df is None):\n            # Time period should only be set if a time series dataframe is provided\n            raise ValueError(\n                \"Time period is set, but no time series dataframe is provided. Please provide a time series dataframe.\"\n            )\n\n        # set the data\n        self._set_data(loadshape_df, time_series_df, features_df)\n\n\n    def extend(self, other):\n        \"\"\"\n            Extend the current Data instance with the Data instance(s) in other by concatenating the features and loadshape dataframes.\n        \"\"\"\n        if not isinstance(other, list):\n            other = [other]\n\n        for data_instance in other:\n            # TODO : What happens if the same id exists in multiple dataframes? Average them out?\n            if isinstance(data_instance, Data):\n                if self._settings.time_period != data_instance.settings.time_period:\n                    raise ValueError(\"Time period setting must be the same for all Data instances.\")\n                if self._features is not None and data_instance.features is not None:\n                    self._features = pd.concat([self._features, data_instance.features])\n                if self._loadshape is not None and data_instance.loadshape is not None:\n                    self._loadshape = pd.concat([self._loadshape, data_instance.loadshape])\n            else:\n                raise TypeError(\"All elements in other must be instances of Data\")\n            \n\n    def _find_groupby_columns(self) -> list:\n        \"\"\"\n        Create the list of columns to be grouped by based on the time_period selected in Settings.\n\n        Time_period : hour => group by (id, hour)\n        Time_period : month => group by (id, month)\n        Time_period : hourly_day_of_week => group by (id, day_of_week, hour)\n        Time_period : weekday_weekend => group by (id, weekday_weekend)\n        Time_period : season_day_of_week => group by (id, season, day_of_week)\n        Time_period : season_hourly_weekday_weekend => group by (id, season, weekday_weekend, hour)\n\n        \"\"\"\n        cols = [\"id\"]\n\n        for period in _const.unique_time_periods:\n            if period in self._settings.time_period:\n                cols.append(period)\n\n        return cols\n\n\n    def _add_index_columns_from_datetime(self, df: pd.DataFrame) -> pd.DataFrame:\n        # Add hour column\n        if \"hour\" in self._settings.time_period:\n            df[\"hour\"] = df['datetime'].dt.hour\n\n        # Add month column\n        if \"month\" in self._settings.time_period:\n            df[\"month\"] = df['datetime'].dt.month\n\n        # Add day_of_week column\n        if \"day_of_week\" in self._settings.time_period:\n            df[\"day_of_week\"] = df['datetime'].dt.dayofweek\n\n        # Add day_of_year column\n        if \"day_of_year\" in self._settings.time_period:\n            df[\"day_of_year\"] = df['datetime'].dt.dayofyear\n\n        # Add weekday_weekend column\n        if \"weekday_weekend\" in self._settings.time_period:\n            df[\"weekday_weekend\"] = df['datetime'].dt.dayofweek\n\n            # Setting the ordering to weekday, weekend\n            df[\"weekday_weekend\"] = (\n                df[\"weekday_weekend\"]\n                .map(self._settings.weekday_weekend._num_dict)\n                .map(self._settings.weekday_weekend._order)\n            )\n\n        # Add season column\n        if \"season\" in self._settings.time_period:\n            df[\"season\"] = df['datetime'].dt.month.map(self._settings.season._num_dict).map(\n                self._settings.season._order\n            )\n\n        return df\n\n\n    def _create_values_for_interpolation(self, df: pd.DataFrame) -> pd.DataFrame:\n        \"\"\"\n        Interpolate missing values in the dataframe based on the settings.\n        \n        - create a new dataframe with id's and correct time column\n        - join on new df and old\n        - interpolate nan values\n\n\n        \"\"\"\n\n        if self._settings.interpolate_missing:\n            unique_ids = df['id'].unique()\n            unique_time_counts = None\n\n            if self._settings.time_period is None: # loadshape type dataframe\n                unique_time_counts = df[\"time\"].max()\n\n            else: # timeseries type dataframe\n                unique_time_counts = _const.time_period_row_counts[self._settings.time_period]\n\n\n            time_values = range(1, unique_time_counts + 1)\n            # Create the expected dataframe having the correct number of timestamps for each id    \n            df_expected = pd.DataFrame({\n                'id': np.repeat(unique_ids, unique_time_counts),\n                'time': np.tile(time_values, len(unique_ids))\n            })\n\n            # Join the expected dataframe with the input dataframe\n            df = df_expected.merge(df, how='left', on=['id', 'time'])\n\n        return df\n\n\n    def _validate_unstacked_loadshape(self, df: pd.DataFrame) -> pd.DataFrame:\n        unstacked_cols = df.columns.drop('id')\n        unstacked_cols = sorted(map(int, unstacked_cols))\n\n        # TODO : Add the ids that are missing values to the excluded_ids dataframe\n        # expected_cols = range(1, max(unstacked_cols) + 1)\n\n        # if unstacked_cols != expected_cols:\n        #     if not self._settings.INTERPOLATE_MISSING or unstacked_cols.count() < expected_cols.count() * self._settings.MIN_DATA_PCT_REQUIRED:\n        #             raise ValueError(f\"Unique time counts per id don't have the minimum time counts required\")\n            \n\n        # Find the missing columns and add them to df with NaN as the default value\n        expected_cols = df.columns.union(range(1, max(unstacked_cols) + 1))\n        df.reindex(columns= expected_cols, fill_value=np.nan)\n\n        if self._settings.interpolate_missing:\n            # Get non-id columns\n            non_id_cols = df.columns[df.columns != 'id']\n\n            # Perform interpolation on non-id columns and update the original DataFrame\n            df[non_id_cols] = df[non_id_cols].interpolate(method=\"linear\", limit_direction=\"both\", axis=1)\n\n        return df\n\n\n    def _validate_format_loadshape(self, df: pd.DataFrame) -> pd.DataFrame:\n        # Reset index to remove any existing index\n        df = df.reset_index()\n        df = df.drop(columns=\"index\", axis=1, errors=\"ignore\")\n\n        # Check columns missing in loadshape_df\n        expected_columns = [\"id\", \"time\", \"loadshape\"]\n        missing_columns = [c for c in expected_columns if c not in df.columns]\n\n        if missing_columns:\n            # TODO : handle the case when index is the id. Then we don't need to check for id in the columns. But how to ensure we don't have wrong index?\n            if \"loadshape\" in missing_columns and \"time\" in missing_columns and \"id\" not in missing_columns:\n                # Handle loadshapes in unstacked version\n                return self._validate_unstacked_loadshape(df)\n            \n            else:  \n                raise ValueError(f\"Missing columns in loadshape_df: {missing_columns}\")\n\n        # Check if all values are present in the columns as required\n        # Else update the values via interpolation if missing, also ignore duplicates if present\n\n        # loadshape df has the \"time\" column, whereas timeseries df has the \"datetime\" column\n        subset_columns = expected_columns[:-1]\n\n        # To eliminate duplicates, sort the values by loadshape and the keep the first (i.e. the lowest) value\n        df = df.sort_values(by='loadshape', key=abs).drop_duplicates(subset=subset_columns, keep=\"first\")\n\n        # Check that the minimum time counts per id is consistent for the input loadshape_df\n        unique_time_counts = df[\"time\"].max()\n        unique_time_counts_per_id = df.groupby(\"id\")[\"time\"].nunique()\n\n        if self._settings.interpolate_missing:\n            if self._settings.time_period is None:\n                # for loadshape type dataframe\n                # if I input a loadshape, I don't want to have to tell it the time_period I used\n                # The time column should directly be pivoted, and the error checking should ensure that the number of values is consistent per meter\n\n                invalid_ids = unique_time_counts_per_id[\n                    unique_time_counts_per_id\n                    < unique_time_counts * self._settings.min_data_pct_required\n                ].index.tolist()\n                excluded_ids = pd.DataFrame(\n                    {\n                        \"id\": invalid_ids,\n                        \"reason\": \"Unique time counts per id don't have the minimum time counts required\",\n                    }\n                )\n                self._excluded_ids = pd.concat(\n                    [self._excluded_ids, excluded_ids], ignore_index=True\n                )\n\n            else:\n                # Check that the number of missing values is less than the threshold\n                for id, group in df.groupby(\"id\"):\n                    if (\n                        group.count().min()\n                        < self._settings.min_data_pct_required\n                        * _const.time_period_row_counts[self._settings.time_period]\n                    ):\n                        # throw out meters with missing values and record them, do not throw error\n\n                        excluded_ids = pd.DataFrame(\n                            {\n                                \"id\": [id],\n                                \"reason\": [\n                                    \"missing minimum number of values in loadshape_df\"\n                                ],\n                            }\n                        )\n\n                        self._excluded_ids = pd.concat(\n                            [self._excluded_ids, excluded_ids], ignore_index=True\n                        )\n\n            df = self._create_values_for_interpolation(df)\n\n            # Fill NaN values with interpolation\n            df['loadshape'] = (\n                df.groupby(\"id\")['loadshape']\n                .apply(lambda x: x.interpolate(method=\"linear\", limit_direction=\"both\"))\n                .reset_index(drop=True)\n            )\n\n        else:\n            if self._settings.time_period is None:\n                # for loadshape type dataframe\n                invalid_ids = unique_time_counts_per_id[\n                    unique_time_counts_per_id < unique_time_counts\n                ].index.tolist()\n                invalid_ids_df = pd.DataFrame(\n                    {\n                        \"id\": invalid_ids,\n                        \"reason\": \"Unique time counts per id don't have the minimum time counts required\",\n                    }\n                )\n                self._excluded_ids = pd.concat(\n                    [self._excluded_ids, invalid_ids_df], ignore_index=True\n                )\n\n            else:\n                # throw out id with null values and record them, do not throw error\n\n                # get a list of any rows with missing values\n                excluded_ids = df[df.isnull().any(axis=1)][\"id\"].values\n                if excluded_ids.size > 0:\n                    excluded_ids = pd.DataFrame({\"id\": excluded_ids})\n                    excluded_ids[\"reason\"] = \"null values in features_df\"\n                    self._excluded_ids = pd.concat([self._excluded_ids, excluded_ids])\n\n        df = df[ ~df[\"id\"].isin(self._excluded_ids[\"id\"])]\n\n        # pivot the loadshape_df to have the time as columns\n        df = df.pivot(index=\"id\", columns=[\"time\"], values=\"loadshape\")\n\n        # Convert multi level index to single level\n        df = (\n            df.rename_axis(None, axis=1)\n            .reset_index()\n            .set_index(\"id\")\n            .drop(columns=\"index\", axis=1, errors=\"ignore\")\n        )\n\n        # Convert columns to int\n        df.columns = df.columns.astype(int)\n\n        return df\n\n\n    def _validate_format_features(self, df: pd.DataFrame) -> pd.DataFrame:\n        # Reset index to remove any existing index\n        df = df.reset_index()\n        df = df.drop(columns=\"index\", axis=1, errors=\"ignore\")\n\n        # Check columns missing in features_df\n        if \"id\" not in df.columns:\n            raise ValueError(f\"Missing columns in features_df: 'id'\")\n\n        # get a list of any rows with missing values\n        excluded_ids = df[df.isnull().any(axis=1)][\"id\"].values\n        if excluded_ids.size > 0:\n            excluded_ids = pd.DataFrame({\"id\": excluded_ids})\n            excluded_ids[\"reason\"] = \"null values in features_df\"\n            self._excluded_ids = pd.concat([self._excluded_ids, excluded_ids])\n\n        # remove any rows with missing values\n        df = df.dropna()\n\n        df.drop_duplicates(keep=\"first\" , inplace = True)\n\n        # drop any ids that are in excluded_ids from loadshape (or init)\n        df = df[~df[\"id\"].isin(self._excluded_ids[\"id\"])]\n        df = (\n            df.reset_index()\n            .set_index(\"id\")\n            .drop(columns=\"index\", axis=1, errors=\"ignore\")\n        )\n\n        # sort by id\n        df = df.sort_index()\n\n        return df\n\n\n    def _convert_timeseries_to_loadshape(\n        self, time_series_df: pd.DataFrame\n    ) -> pd.DataFrame:\n        \"\"\"\n        Arguments:\n            Time series dataframe with columns = [id, datetime, observed, observed_error, modeled, modeled_error\n\n        Returns :\n            Loadshape dataframe with columns = [id, time, loadshape]\n        \"\"\"\n\n        base_df = time_series_df.copy()  # don't change the original dataframe\n\n        # Reset index to remove any existing index\n        base_df = base_df.reset_index()\n        base_df = base_df.drop(columns=\"index\", axis=1, errors=\"ignore\")\n\n        # Check columns missing in time_series_df\n        df_type = self._settings.LOADSHAPE_TYPE\n        expected_columns = [\"id\", \"datetime\"]\n        if (df_type == \"error\") and (\"error\" in base_df.columns):\n            expected_columns.append(\"error\")\n        elif (df_type == \"error\") and (\"error\" not in base_df.columns):\n            expected_columns.extend([\"observed\", \"modeled\"])\n        else:\n            expected_columns.append(df_type)\n\n        missing_columns = [c for c in expected_columns if c not in base_df.columns]\n        if missing_columns:\n            raise ValueError(f\"Missing columns in time_series_df: {missing_columns}\")\n\n        # Check that the datetime column is actually of type datetime\n        if is_datetime(base_df[\"datetime\"]):\n            base_df[\"datetime\"] = pd.to_datetime(base_df[\"datetime\"], utc=True) #TODO: should this be utc=True? should this be applied to all datetime types?\n        else:\n            raise ValueError(\"The 'datetime' column must be of datetime type\")\n\n        if df_type == \"error\" and (\"error\" not in base_df.columns):\n            base_df[\"error\"] = 1 - base_df[\"observed\"] / base_df[\"modeled\"]\n\n        # Remove duplicates\n        subset_columns = expected_columns[:-1]\n\n        # To eliminate duplicates, sort the values by loadshape and the keep the first (i.e. the lowest) value\n        base_df = base_df.sort_values(by=df_type, key=abs).drop_duplicates(subset=subset_columns, keep=\"first\")\n\n        base_df = self._add_index_columns_from_datetime(base_df) # Add month / day_of_week / hour / etc columns\n\n        # Check that each id has a minimum granularity lower than requested time period, otherwise we cannot aggregate\n        # get minimum time interval per id\n        base_df[\"time_diff\"] = base_df.groupby(\"id\")[\"datetime\"].diff()\n        min_time_diff_per_id = base_df.groupby(\"id\")[\"time_diff\"].min() / np.timedelta64(1, 'm')\n\n        # Get the ids that have a higher minimum granularity than defined\n        if self._settings.TIME_PERIOD != 'month':\n            invalid_ids = min_time_diff_per_id[\n                min_time_diff_per_id > _const.min_granularity_per_time_period[self._settings.TIME_PERIOD]\n            ].index.tolist()\n\n        else:\n            # Check that every ID has 12 months available.\n            unique_month_counts_per_id = base_df.groupby('id')['month'].nunique()\n            invalid_ids = unique_month_counts_per_id[unique_month_counts_per_id < 12].index.tolist()\n\n        # Remove the invalid ids from the base_df\n        base_df = base_df[~base_df[\"id\"].isin(invalid_ids)]        \n\n        # If there are any invalid ids, add them to the excluded_ids dataframe\n        if invalid_ids:\n            invalid_ids_df = pd.DataFrame(\n                {\n                    \"id\": invalid_ids,\n                    \"reason\": \"Minimum time interval is more than the specified TimePeriod\",\n                }\n            )\n            self._excluded_ids = pd.concat(\n                [self._excluded_ids, invalid_ids_df], ignore_index=True\n            )\n\n        # Set the index to datetime\n        base_df = base_df.set_index(\"datetime\")\n\n        # Aggregate the input time_series based on time_period\n\n        group_by_columns = self._find_groupby_columns()\n\n        base_df = base_df.groupby(group_by_columns)[self._settings.loadshape_type]\n\n        base_df = base_df.agg(loadshape=self._settings.agg_type).reset_index()\n\n        # Sort the values so that the ordering is maintained correctly\n        base_df = base_df.sort_values(by=group_by_columns)\n\n        # Create the count of the index per ID\n        base_df[\"time\"] = base_df.groupby(\"id\").cumcount() + 1\n\n        # Validate that all the values are correct\n        loadshape_df = self._validate_format_loadshape(base_df)\n\n        return loadshape_df\n\n\n    def _trim_data(self) -> None:\n        \"\"\"\n        Trim the loadshape and features dataframes to the maximum size allowed by the settings.\n        \"\"\"\n\n        max_size = self._settings.max_pool_size\n\n        ids = self.ids\n        excluded_ids = []\n        if len(ids) > max_size:\n            # randomly select ids to remove\n            excluded_ids = np.random.choice(ids, len(ids) - max_size, replace=False)\n\n            # add excluded ids to excluded_ids dataframe\n            excluded_ids_df = pd.DataFrame({\"id\": excluded_ids})\n            excluded_ids_df[\"reason\"] = \"randomly selected to reduce pool size\"\n            self._excluded_ids = pd.concat([self._excluded_ids, excluded_ids_df])\n\n        if (self._loadshape is not None) and (len(excluded_ids) > 0):\n            self._loadshape = self._loadshape[\n                ~self._loadshape.index.isin(self._excluded_ids[\"id\"])\n            ]\n\n        if (self._features is not None) and (len(excluded_ids) > 0):\n            self._features = self._features[\n                ~self._features.index.isin(self._excluded_ids[\"id\"])\n            ]\n\n\n    def _set_data(\n        self, loadshape_df=None, time_series_df=None, features_df=None\n    ) -> None:\n        \"\"\"\n            Loadshape, timeseries and features dataframes are input. The loadshape and features dataframes are validated and formatted.\n            The timeseries dataframe is converted to a loadshape dataframe and then validated and formatted.\n\n            Time period is only set if a timeseries dataframe is provided. If a loadshape dataframe is provided, \n            the aggregation type, loadshape type and time period all must be set to None.\n\n            Either loadshape or timeseries data is allowed, but not both. Atleast one of them must be provided as well.\n            Features is independent of the loadshape and timeseries dataframes.\n\n            Loadshape / timeseries only input => Clustering / IMM\n            Features only input => Stratified Sampling\n\n            Note the loadshape and features dataframe can only be set once per class.\n\n        Args:\n            Loadshape_df: columns = [id, time, loadshape]\n\n            Time_series_df: columns = [id, datetime, observed, observed_error, modeled, modeled_error]\n\n            Features_df: columns = [id, {feature_1}, {feature_2}, ...]\n\n        Output:\n            loadshape: index = id, columns = time, values = loadshape\n\n            features: index = id, columns = [{feature_1}, {feature_2}, ...]\n\n\n        \"\"\"\n\n        if loadshape_df is not None:\n            if self._loadshape is not None :\n                raise ValueError(\"Loadshape Data has already been set.\")\n            elif self._settings.loadshape_type is not None:\n                raise ValueError(\"Loadshape Type cannot be set for a loadshape dataframe.\")\n\n            loadshape_df = self._validate_format_loadshape(loadshape_df)\n\n        elif time_series_df is not None:\n            if self._loadshape is not None:\n                raise ValueError(\"Loadshape Data has already been set.\")\n\n            loadshape_df = self._convert_timeseries_to_loadshape(time_series_df)\n\n        if features_df is not None:\n            if self._features is not None:\n                raise ValueError(\"Features Data has already been set.\")\n            features_df = self._validate_format_features(features_df)\n\n        if loadshape_df is not None:\n            # If loadshape still has id as one of its columns, set it as index\n            if 'id' in loadshape_df.columns:\n                loadshape_df.set_index('id', inplace=True)\n\n            # drop any ids that are in the excluded_ids list\n            loadshape_df = loadshape_df[\n                ~loadshape_df.index.isin(self._excluded_ids[\"id\"])\n            ]\n\n        # If the dataframes are empty return None, not an empty dataframe\n        if features_df is not None:\n            self._features = features_df if not features_df.empty else None\n        self._loadshape = loadshape_df if not loadshape_df.empty else None\n\n        # filter pool to max size\n        self._trim_data()\n\n        return self\n\n    @property\n    def settings(self):\n        return self._settings.model_copy()\n    \n    @property\n    def loadshape(self):\n        if self._loadshape is None:\n            return None\n        else :\n            return self._loadshape.copy()\n    \n    @property\n    def features(self):\n        if self._features is None:\n            return None\n        else :\n            return self._features.copy()\n\n    @property\n    def ids(self):\n        if isinstance(self._loadshape, pd.DataFrame):\n            return deepcopy(self._loadshape.index.unique().to_list())\n        elif isinstance(self._features, pd.DataFrame):\n            return deepcopy(self._features.index.unique().to_list())\n        else:\n            return None\n\n    @property\n    def excluded_ids(self):\n        if self._excluded_ids is None:\n            return None\n        else :\n            return self._excluded_ids.copy()"
  },
  {
    "path": "opendsm/comparison_groups/common/data_settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport pydantic\n\nfrom typing import Optional,Union\n\nimport opendsm.comparison_groups.common.const as _const\nfrom opendsm.common.base_settings import BaseSettings\n\n\nmin_data_pct = 0.8\n\n\n# Note: Options list order defines how seasons will be ordered in the loadshape\nclass Season_Definition(BaseSettings):\n    january: str = pydantic.Field(default=\"winter\")\n    february: str = pydantic.Field(default=\"winter\")\n    march: str = pydantic.Field(default=\"shoulder\")\n    april: str = pydantic.Field(default=\"shoulder\")\n    may: str = pydantic.Field(default=\"shoulder\")\n    june: str = pydantic.Field(default=\"summer\")\n    july: str = pydantic.Field(default=\"summer\")\n    august: str = pydantic.Field(default=\"summer\")\n    september: str = pydantic.Field(default=\"summer\")\n    october: str = pydantic.Field(default=\"shoulder\")\n    november: str = pydantic.Field(default=\"winter\")\n    december: str = pydantic.Field(default=\"winter\")\n\n    options: list[str] = pydantic.Field(default=[\"summer\", \"shoulder\", \"winter\"])\n\n    \"\"\"Set dictionaries of seasons\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def set_numeric_dict(self) -> Season_Definition:\n        season_dict = {}\n        for month, num in _const.season_num.items():\n            val = getattr(self, month)\n            if val not in self.options:\n                raise ValueError(f\"SeasonDefinition: {val} is not a valid option. Valid options are {self.options}\")\n\n            season_dict[num] = val\n        \n        self._num_dict = season_dict\n        self._order = {val: i for i, val in enumerate(self.options)}\n\n        return self\n\n\nclass Weekday_Weekend_Definition(BaseSettings):\n    monday: str = pydantic.Field(default=\"weekday\")\n    tuesday: str = pydantic.Field(default=\"weekday\")\n    wednesday: str = pydantic.Field(default=\"weekday\")\n    thursday: str = pydantic.Field(default=\"weekday\")\n    friday: str = pydantic.Field(default=\"weekday\")\n    saturday: str = pydantic.Field(default=\"weekend\")\n    sunday: str = pydantic.Field(default=\"weekend\")\n\n    options: list[str] = pydantic.Field(default=[\"weekday\", \"weekend\"])\n\n    \"\"\"Set dictionaries of weekday/weekend\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def set_numeric_dict(self) -> Weekday_Weekend_Definition:\n        weekday_dict = {}\n        for day, num in _const.weekday_num.items():\n            val = getattr(self, day)\n            if val not in self.options:\n                raise ValueError(f\"WeekdayWeekendDefinition: {val} is not a valid option. Valid options are {self.options}\")\n            \n            weekday_dict[num] = val\n        \n        self._num_dict = weekday_dict\n        self._order = {val: i for i, val in enumerate(self.options)}\n\n        return self\n    \n\nclass Data_Settings(BaseSettings):\n    \"\"\"maximum number of meters to be used in the comparison pool\"\"\"\n    max_pool_size: int = pydantic.Field(\n        default=10000,\n        ge=1,\n        validate_default=True,\n    )\n\n    \"\"\"aggregation type for the loadshape\"\"\"\n    agg_type: Optional[_const.AggType] = pydantic.Field(\n        default=_const.AggType.MEAN,\n        validate_default=True,\n    )\n    \n    \"\"\"type of loadshape to be used\"\"\"\n    loadshape_type: Optional[_const.LoadshapeType] = pydantic.Field(\n        default=_const.LoadshapeType.MODELED, \n        validate_default=True,\n    )\n\n    \"\"\"time period to be used for the loadshape\"\"\"\n    time_period: Optional[_const.TimePeriod] = pydantic.Field(\n        default=_const.TimePeriod.SEASONAL_HOURLY_DAY_OF_WEEK, \n        validate_default=True,\n    )\n\n    \"\"\"interpolate missing values\"\"\"\n    interpolate_missing: bool = pydantic.Field(\n        default=True, \n        validate_default=True,\n    )\n\n    \"\"\"minimum percentage of data required for a meter to be included\"\"\"\n    min_data_pct_required: Optional[float] = pydantic.Field(\n        default=min_data_pct, \n        validate_default=True,\n    )\n\n    @pydantic.field_validator(\"min_data_pct_required\")\n    @classmethod\n    def validate_min_data_pct_required(cls, value):\n        if value is None:\n            pass\n\n        elif value != min_data_pct:\n            raise ValueError(f\"min_data_pct_required must be {min_data_pct}\")\n        \n        return value\n\n    \"\"\"season definition to be used for the loadshape\"\"\"\n    season: Union[dict, Season_Definition] = pydantic.Field(\n        default=_const.default_season_def, \n    )\n\n    \"\"\"weekday/weekend definition to be used for the loadshape\"\"\"\n    weekday_weekend: Union[dict, Weekday_Weekend_Definition] = pydantic.Field(\n        default=_const.default_weekday_weekend_def, \n    )\n\n    \"\"\"set season and weekday_weekend classes with given dictionaries\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _set_nested_classes(self):\n        self.model_config[\"frozen\"] = False\n        \n        if isinstance(self.season, dict):\n            self.season = Season_Definition(**self.season)\n\n        if isinstance(self.weekday_weekend, dict):\n            self.weekday_weekend = Weekday_Weekend_Definition(**self.weekday_weekend)\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n    \n    \"\"\"validate loadshape/time series settings\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _validate_loadshape_time_series_settings(self):\n        ls_dict = {\"agg_type\": self.agg_type, \"loadshape_type\": self.loadshape_type, \"time_period\": self.time_period}\n        is_set = {k: v is not None for k, v in ls_dict.items()}\n        if any(is_set.values()):\n            for k, v in is_set.items():\n                if v is False:\n                    raise ValueError(f\"{k} must be set if any of the following are set: {list(is_set.keys())}\")\n\n        return self\n\n    \"\"\"set min_data_pct_required\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _set_min_data_pct_on_interpolate(self):\n        self.model_config[\"frozen\"] = False\n\n        if self.interpolate_missing:\n            self.min_data_pct_required = min_data_pct\n        else:\n            self.min_data_pct_required = None\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n    \n\nif __name__ == \"__main__\":\n    # Test SeasonDefinition\n    # Note: Options list order defines how seasons will be orderd in the loadshape\n    season_dict = {\n        \"options\":  [\"summer\", \"shoulder\", \"winter\"],\n        \"January\":   \"winter\", \n        \"February\":  \"winter\", \n        \"March\":     \"shoulder\", \n        \"April\":     \"shoulder\", \n        \"May\":       \"shoulder\", \n        \"June\":      \"summer\", \n        \"July\":      \"summer\", \n        \"August\":    \"summer\", \n        \"September\": \"summer\", \n        \"October\":   \"shoulder\", \n        \"November\":  \"winter\", \n        \"December\":  \"winter\",\n        }\n\n    # season = SeasonDefinition(**season_def)\n    # print(season.model_dump_json())\n\n    # Test WeekdayWeekendDefinition\n    weekday_weekend_dict = {\n        \"options\":  [\"weekday\", \"weekend\", \"oops\"],\n        \"Monday\":    \"weekday\",\n        \"Tuesday\":   \"weekday\",\n        \"Wednesday\": \"weekday\",\n        \"Thursday\":  \"weekday\",\n        \"Friday\":    \"weekend\",\n        \"Saturday\":  \"weekend\",\n        \"Sunday\":    \"weekday\",\n        }\n    \n    # weekday_weekend = WeekdayWeekendDefinition(**weekday_weekend_def)\n    # weekday_weekend = WeekdayWeekendDefinition()\n    # print(weekday_weekend.model_dump_json())\n\n    # Test DataSettings\n    settings = Data_Settings(\n        agg_type=\"median\",\n        season=season_dict, \n        weekday_weekend=weekday_weekend_dict,\n    )\n    print(settings.model_dump_json())\n    print(settings.season._num_dict)\n    print(settings.season._order)"
  },
  {
    "path": "opendsm/comparison_groups/common/tutorial_data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\nfrom pathlib import Path\n\n# Define the current directory\ncurrent_dir = Path(__file__).parent\ndata_dir = current_dir.parents[2] / \"data\"\n\ndef load_tutorial_data(data_type: str):\n    data_type = data_type.lower()\n\n    if data_type == \"features\":\n        df = pd.read_csv(data_dir / \"features.csv\")\n    \n    elif data_type == \"seasonal_hourly_day_of_week_loadshape\":\n        df = pd.read_csv(data_dir / \"seasonal_hourly_day_of_week_loadshape.csv\")\n    \n    elif data_type == \"seasonal_day_of_week_loadshape\":\n        df = pd.read_csv(data_dir / \"seasonal_day_of_week_loadshape.csv\")\n    \n    elif data_type == \"month_loadshape\":\n        df = pd.read_csv(data_dir / \"month_loadshape.csv\")\n    \n    elif data_type == \"hourly_data\":\n        df = pd.read_parquet(data_dir / \"hourly_data.parquet\")\n\n    else:\n        raise ValueError(f\"Data type {data_type} not recognized.\")\n    \n    if data_type not in \"hourly_data\":\n        df = df.set_index(\"id\")\n    \n    return df"
  },
  {
    "path": "opendsm/comparison_groups/individual_meter_matching/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.individual_meter_matching.create_comparison_groups import Individual_Meter_Matching as IMM\nfrom opendsm.comparison_groups.individual_meter_matching.settings import Settings as IMM_Settings\n"
  },
  {
    "path": "opendsm/comparison_groups/individual_meter_matching/create_comparison_groups.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.common.base_comparison_group import Comparison_Group_Algorithm\n\nfrom opendsm.comparison_groups.individual_meter_matching.settings import Settings\nfrom opendsm.comparison_groups.individual_meter_matching.distance_calc_selection import DistanceMatching\n\n\n\nclass Individual_Meter_Matching(Comparison_Group_Algorithm):\n    def __init__(self, settings: Optional[Settings] = None):\n        if settings is None:\n            settings = Settings()\n        \n        self.settings = settings\n\n    def _create_clusters_df(self, df_raw):\n        clusters = df_raw\n        clusters[\"cluster\"] = 0\n\n        clusters = clusters.reset_index()\n\n        # add weight column as count of id\n        clusters[\"weight\"] = clusters.groupby(\"id\")[\"id\"].transform(\"count\")\n\n        # replace duplicates after the first with 0\n        clusters.loc[clusters.duplicated(subset=\"id\", keep=\"first\"), \"weight\"] = 0\n\n        # sort by id and weight\n        # clusters = clusters.sort_values(by=[\"id\", \"weight\"], ascending=[True, False])\n\n        # add duplicated column\n        clusters[\"duplicated\"] = clusters.duplicated(subset=\"id\", keep=False)\n\n        clusters = clusters.set_index(\"id\")\n\n        # reorder columns\n        cols = [\"treatment\", \"distance\", \"duplicated\", \"cluster\", \"weight\"]\n        clusters = clusters[cols]\n\n        return clusters\n    \n\n    def _create_treatment_weights_df(self, ids):\n        coeffs = np.ones(len(ids))\n\n        treatment_weights = pd.DataFrame(coeffs, index=ids, columns=[\"pct_cluster_0\"])\n        treatment_weights.index.name = \"id\"\n\n        return treatment_weights\n    \n\n    def get_comparison_group(self, treatment_data, comparison_pool_data, weights=None):\n        self.treatment_data = treatment_data\n        self.comparison_pool_data = comparison_pool_data\n\n        self.treatment_ids = treatment_data.ids\n        self.treatment_loadshape = treatment_data.loadshape\n        self.comparison_pool_loadshape = comparison_pool_data.loadshape\n        self.ls_weights = self._validate_ls_weights(weights)\n\n        # Get clusters\n        distance_matching = DistanceMatching(self.settings)\n        df_raw = distance_matching.get_comparison_group(\n            self.treatment_loadshape, \n            self.comparison_pool_loadshape, \n            weights=self.ls_weights\n        )\n\n        clusters = self._create_clusters_df(df_raw)\n\n        # Create treatment_weights\n        treatment_weights = self._create_treatment_weights_df(self.treatment_ids)\n\n        # Assign dfs to self\n        self.clusters = clusters\n        self.treatment_weights = treatment_weights\n\n        return clusters, treatment_weights\n    \n\n    def add_treatment_meters(self, treatment_data):\n        # need some code to make life easier when adding treatment meters. Need to recalculate duplicate weights and unc_multiplier\n        pass"
  },
  {
    "path": "opendsm/comparison_groups/individual_meter_matching/distance_calc_selection.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.individual_meter_matching import highs_settings as _highs_settings\nimport numpy as np\nimport pandas as pd\n\nfrom scipy.spatial.distance import cdist\nfrom scipy.optimize import linear_sum_assignment\nfrom scipy import sparse\nfrom qpsolvers import solve_ls\n\nfrom opendsm.comparison_groups.individual_meter_matching.settings import Settings\n\n__all__ = (\"DistanceMatching\",)\n\n\n\ndef cp_chunks(lst, n):\n    for i in range(0, len(lst), n):\n        yield lst[i : i + n]\n\n\ndef _distances(ls_t, ls_cp, weights=None, dist_metric=\"euclidean\", n_meters_per_chunk=10000):\n    if weights is not None:\n        ls_t = ls_t * weights\n\n    # calculate distances in chunks\n    n_chunk = len(ls_cp)\n    if n_meters_per_chunk < n_chunk:\n        n_chunk = n_meters_per_chunk\n\n    dist = []\n    for ls_cp_chunk in cp_chunks(ls_cp, n_meters_per_chunk):\n        if weights is not None:\n            ls_cp_chunk = ls_cp_chunk * weights\n\n        # perform weighted distance calculation\n        chunked_dist = cdist(ls_t, ls_cp_chunk, metric=dist_metric)\n\n        dist.append(chunked_dist)\n\n    dist = np.hstack(dist)\n\n    return dist\n\n\ndef highs_fit_comparison_group_loadshape(t_ls, cp_ls, coef_sum=1, solver=\"highs\", settings=None, verbose=False):\n    if settings is None:\n        if coef_sum == 1:\n            settings = _highs_settings.HiGHS_Settings(\n                primal_feasibility_tolerance=1E-4, \n                dual_feasibility_tolerance=1E-4, \n            )\n        else:\n            settings = _highs_settings.HiGHS_Settings(\n                primal_feasibility_tolerance=1, \n                dual_feasibility_tolerance=1, \n            )\n        settings = {k.lower(): v for k, v in dict(settings).items()}\n\n    if coef_sum == 1:\n        _MIN_X = 1E-6\n    else:\n        _MIN_X = 5E-3\n    \n    num_pool_meters = cp_ls.shape[0]\n\n    R = sparse.csc_matrix(cp_ls.T)\n\n    h = np.zeros(num_pool_meters)\n    eye = sparse.eye(num_pool_meters, format=\"csc\")\n    A = sparse.csc_matrix(np.ones(num_pool_meters))\n    b = np.array([coef_sum])\n\n    lb = np.zeros(num_pool_meters)\n    ub = np.ones(num_pool_meters)\n\n    x_opt = solve_ls(R, t_ls, G=-eye, h=h, A=A, b=b, lb=lb, ub=ub, solver=solver, verbose=verbose, **settings)\n\n    x_opt[x_opt < 0] = 0\n    x_opt[x_opt > 1] = 1\n    x_opt[np.abs(x_opt) < _MIN_X] = 0\n    x_opt *= coef_sum/x_opt.sum()\n\n    return x_opt\n\n\nclass DistanceMatchingError(Exception):\n    pass\n\n\nclass DistanceMatching:\n    \"\"\"\n    Parameters\n    ----------\n    treatment_group: pd.DataFrame\n        A dataframe representing treatment group meters, indexed by id, with each column being a data point in a usage pattern.\n    comparison_pool: pd.DataFrame\n        A dataframe representing comparison pool meters, indexed by id, with each column being a data point in a usage pattern.\n    weights: list | 1D np.array\n        A list of floats (must be of length of the treatment group columns) to scale the usage patterns in order to ensure that certain components of usage have higher weights towards matching than others.\n    n_treatments_per_chunk: int\n        Due to local memory limitations, treatment meters can be chunked so that the cdist calculation can happen in memory. 10,000 meters appear to be sufficient for most memory constraints.\n    \"\"\"\n\n    def __init__(\n        self,\n        settings=None,\n    ):\n        if settings is None:\n            self.settings = Settings()\n        elif isinstance(settings, Settings):\n            self.settings = settings\n        else:\n            raise Exception(\n                \"invalid settings provided to 'individual_metering_matching'\"\n            )\n\n        self.dist_metric = settings.distance_metric\n        if self.dist_metric == \"manhattan\":\n            self.dist_metric = \"cityblock\"\n\n    def _closest_idx_duplicates_allowed(self, distances, n_match=None):\n        if n_match is None:\n            n_match = self.settings.n_matches_per_treatment\n\n        if n_match > distances.shape[1]:\n            n_match = distances.shape[1]\n\n        # sort distances by row and get the indices of the sorted distances\n        # Note: pypi bottleneck is faster than numpy for this\n        cg_idx = np.argpartition(distances, n_match, axis=1)[:, :n_match]\n\n        return cg_idx\n\n    def _closest_idx_duplicates_not_allowed(self, ls_t, ls_cp, distances):\n        n_match = self.settings.n_matches_per_treatment\n        selection_method = self.settings.selection_method\n\n        n_treatment = ls_t.shape[0]\n        n_pool = ls_cp.shape[0]\n\n        if n_match*n_treatment > n_pool:\n            n_match = int(n_pool / n_treatment)\n\n        if n_match == 0:\n            raise DistanceMatchingError(f\"Not enough treatment pool meters {n_pool} to match with {n_treatment} treatment meters without duplicates\")\n        \n        if selection_method == \"minimize_meter_distance\":\n            # normalize distances by min distance of each row\n            # min_dist = np.take_along_axis(distances, self._closest_idx_duplicates_allowed(distances, n_match=1), axis=1)\n            # distances = distances / min_dist\n\n            # duplicate rows n_match times\n            distances = np.repeat(distances, n_match, axis=0)\n            t_idx = np.repeat(np.arange(distances.shape[0]), n_match)\n\n            row_idx, col_idx = linear_sum_assignment(distances)\n\n            cg_idx = [[] for _ in range(distances.shape[0])]\n            for i, cp_idx in zip(row_idx, col_idx):\n                cg_idx[t_idx[i]].append(cp_idx)\n\n        elif selection_method == \"minimize_loadshape_distance\":\n            coef_sum = n_match*len(ls_t)\n            ls_t_mean = np.mean(ls_t.values, axis=0)*coef_sum\n\n            x_opt = highs_fit_comparison_group_loadshape(\n                ls_t_mean, ls_cp.values, coef_sum=coef_sum, solver=\"highs\", settings=None, verbose=False\n            )\n\n            # argsort x_opt\n            x_opt_idx = np.argsort(x_opt)[::-1][:coef_sum]\n\n            # reshape distances to be ls_t.shape[0] x n_match\n            cg_idx = np.reshape(x_opt_idx, (ls_t.shape[0], n_match))\n\n        else:\n            raise DistanceMatchingError(f\"Invalid selection method: {selection_method}\")\n\n        return cg_idx\n    \n    \n    def get_comparison_group(\n        self,\n        treatment_group,\n        comparison_pool,\n        weights=None,\n    ):\n        ls_t = treatment_group\n        ls_cp = comparison_pool\n\n        n_match = self.settings.n_matches_per_treatment\n        max_distance_threshold = self.settings.max_distance_threshold\n        n_meters_per_chunk = self.settings.n_treatments_per_chunk\n\n        # TODO: if matching loadshapes, this isn't necessary\n        distances = _distances(ls_t, ls_cp, weights, self.dist_metric, n_meters_per_chunk)\n\n        if self.settings.allow_duplicate_matches:\n            cg_idx = self._closest_idx_duplicates_allowed(distances, n_match=n_match)\n        else:\n            cg_idx = self._closest_idx_duplicates_not_allowed(ls_t, ls_cp, distances)\n\n        data = []\n        for t_idx in range(ls_t.shape[0]):\n            t_id = ls_t.index[t_idx]\n            for cp_idx in cg_idx[t_idx]:\n                cg_id = ls_cp.index[cp_idx]\n\n                data.append([cg_id, t_id, distances[t_idx, cp_idx]])\n\n        df = pd.DataFrame(data, columns=[\"id\", \"treatment\", \"distance\"])\n\n        # check that the distance is less than the threshold\n        if max_distance_threshold is not None:\n            df = df[df[\"distance\"] <= max_distance_threshold]\n\n        # add column if id is duplicated\n        df[\"duplicated\"] = df.duplicated(subset=\"id\", keep=False)\n        \n        return df\n\n\nif __name__ == \"__main__\":\n    d = DistanceMatching()\n    print(d.settings)\n"
  },
  {
    "path": "opendsm/comparison_groups/individual_meter_matching/highs_settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pydantic\n\nfrom opendsm.common.base_settings import BaseSettings\n\nfrom typing import Optional, Literal\n\n\n# system maximum float\nMIN_FLOAT = np.finfo(np.float64).tiny\nMAX_FLOAT = np.finfo(np.float64).max\n    \n\nclass HiGHS_Settings(BaseSettings):\n    \"\"\"Settings for HiGHS optimization solver\"\"\"\n\n    \"\"\"Presolve option\"\"\"\n    presolve: Literal[\"off\", \"choose\", \"on\"] = pydantic.Field(\n        default=\"choose\",\n        validate_default=True,\n    )\n\n    \"\"\"If 'simplex'/'ipm'/'pdlp' is chosen then, for a MIP (QP) the integrality constraint (quadratic term) will be ignored\"\"\"\n    # solver: Literal[\"simplex\", \"choose\", \"ipm\", \"pdlp\"] = pydantic.Field(\n    #     default=\"choose\",\n    #     validate_default=True,\n    # )\n\n    \"\"\"Parallel option\"\"\"\n    parallel: Literal[\"off\", \"choose\", \"on\"] = pydantic.Field(\n        default=\"off\", # was \"choose\"\n        validate_default=True,\n    )\n\n    \"\"\"Run IPM crossover\"\"\"\n    run_crossover: Literal[\"off\", \"choose\", \"on\"] = pydantic.Field(\n        default=\"on\",\n        validate_default=True,\n    )\n\n    \"\"\"Time limit (seconds)\"\"\"\n    time_limit: float = pydantic.Field(\n        default=float('inf'),\n        ge=0,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Compute cost, bound, RHS and basic solution ranging\"\"\"\n    ranging: Literal[\"off\", \"on\"] = pydantic.Field(\n        default=\"off\",\n        validate_default=True,\n    )\n\n    \"\"\"Limit on |cost coefficient|: values greater than or equal to this will be treated as infinite\"\"\"\n    infinite_cost: float = pydantic.Field(\n        default=1e+20,\n        ge=1e+15,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Limit on |constraint bound|: values greater than or equal to this will be treated as infinite\"\"\"\n    infinite_bound: float = pydantic.Field(\n        default=1e+20,\n        ge=1e+15,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Lower limit on |matrix entries|: values less than or equal to this will be treated as zero\"\"\"\n    small_matrix_value: float = pydantic.Field(\n        default=1e-09,\n        ge=1e-12,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Upper limit on |matrix entries|: values greater than or equal to this will be treated as infinite\"\"\"\n    large_matrix_value: float = pydantic.Field(\n        default=1e+15,\n        ge=1,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Primal feasibility tolerance\"\"\"\n    primal_feasibility_tolerance: float = pydantic.Field(\n        default=1e-07,\n        ge=1e-10,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Dual feasibility tolerance\"\"\"\n    dual_feasibility_tolerance: float = pydantic.Field(\n        default=1e-07,\n        ge=1e-10,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"IPM optimality tolerance\"\"\"\n    ipm_optimality_tolerance: float = pydantic.Field(\n        default=1e-08,\n        ge=1e-12,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Objective bound for termination of the dual simplex solver\"\"\"\n    objective_bound: float = pydantic.Field(\n        default=float('inf'),\n        ge=float('-inf'),\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Objective target for termination of the MIP solver\"\"\"\n    objective_target: float = pydantic.Field(\n        default=float('-inf'),\n        ge=float('-inf'),\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Random seed used in HiGHS\"\"\"\n    random_seed: Optional[int] = pydantic.Field(\n        default=None,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Number of threads used by HiGHS (0: automatic)\"\"\"\n    threads: int = pydantic.Field(\n        default=0,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Exponent of power-of-two bound scaling for model\"\"\"\n    user_bound_scale: int = pydantic.Field(\n        default=0,\n        ge=-2147483647,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Exponent of power-of-two cost scaling for model\"\"\"\n    user_cost_scale: int = pydantic.Field(\n        default=0,\n        ge=-2147483647,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Strategy for simplex solver [0: Choose; 1: Dual (serial); 2: Dual (PAMI); 3: Dual (SIP); 4: Primal]\"\"\"\n    simplex_strategy: int = pydantic.Field(\n        default=1,\n        ge=0,\n        le=4,\n        validate_default=True,\n    )\n\n    \"\"\"Simplex scaling strategy: [0: off; 1: choose; 2: equilibration; 3: forced equilibration; 4: max value 0; 5: max value 1]\"\"\"\n    simplex_scale_strategy: int = pydantic.Field(\n        default=1,\n        ge=0,\n        le=5,\n        validate_default=True,\n    )\n\n    \"\"\"Strategy for simplex dual edge weights: [-1: Choose; 0: Dantzig; 1: Devex; 2: Steepest Edge]\"\"\"\n    simplex_dual_edge_weight_strategy: int = pydantic.Field(\n        default=-1,\n        ge=-1,\n        le=2,\n        validate_default=True,\n    )\n\n    \"\"\"Strategy for simplex primal edge weights: [-1: Choose; 0: Dantzig; 1: Devex; 2: Steepest Edge]\"\"\"\n    simplex_primal_edge_weight_strategy: int = pydantic.Field(\n        default=-1,\n        ge=-1,\n        le=2,\n        validate_default=True,\n    )\n\n    \"\"\"Iteration limit for simplex solver when solving LPs, but not subproblems in the MIP solver\"\"\"\n    simplex_iteration_limit: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Limit on the number of simplex UPDATE operations\"\"\"\n    simplex_update_limit: int = pydantic.Field(\n        default=5000,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Maximum level of concurrency in parallel simplex\"\"\"\n    simplex_max_concurrency: int = pydantic.Field(\n        default=8,\n        ge=1,\n        le=8,\n        validate_default=True,\n    )\n\n    \"\"\"Enables or disables solver output\"\"\"\n    # output_file: bool = pydantic.Field(\n    #     default=True,\n    #     validate_default=True,\n    # )\n\n    \"\"\"Enables or disables console logging\"\"\"\n    # log_to_console: bool = pydantic.Field(\n    #     default=True,\n    #     validate_default=True,\n    # )\n\n    \"\"\"Solution file\"\"\"\n    solution_file: str = pydantic.Field(\n        default=\"\",\n        validate_default=True,\n    )\n\n    \"\"\"Log file\"\"\"\n    log_file: str = pydantic.Field(\n        default=\"\",\n        validate_default=True,\n    )\n\n    \"\"\"Write the primal and dual solution to a file\"\"\"\n    write_solution_to_file: bool = pydantic.Field(\n        default=False,\n        validate_default=True,\n    )\n\n    \"\"\"Style of solution file: [0: HiGHS raw; 1: HiGHS pretty; 2: Glpsol raw; 3: Glpsol pretty; 4: HiGHS sparse raw] (raw = computer-readable, pretty = human-readable)\"\"\"\n    write_solution_style: int = pydantic.Field(\n        default=0,\n        ge=0,\n        le=4,\n        validate_default=True,\n    )\n\n    \"\"\"Location of cost row for Glpsol file: -2 => Last; -1 => None; 0 => None if empty, otherwise data file location; 1 <= n <= num_row => Location n; n > num_row => Last\"\"\"\n    glpsol_cost_row_location: int = pydantic.Field(\n        default=0,\n        ge=-2,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Write model file\"\"\"\n    write_model_file: str = pydantic.Field(\n        default=\"\",\n        validate_default=True,\n    )\n\n    \"\"\"Write the model to a file\"\"\"\n    write_model_to_file: bool = pydantic.Field(\n        default=False,\n        validate_default=True,\n    )\n\n    \"\"\"Whether MIP symmetry should be detected\"\"\"\n    mip_detect_symmetry: bool = pydantic.Field(\n        default=True,\n        validate_default=True,\n    )\n\n    \"\"\"Whether MIP restart is permitted\"\"\"\n    mip_allow_restart: bool = pydantic.Field(\n        default=True,\n        validate_default=True,\n    )\n\n    \"\"\"MIP solver max number of nodes\"\"\"\n    mip_max_nodes: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"MIP solver max number of nodes where estimate is above cutoff bound\"\"\"\n    mip_max_stall_nodes: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Whether improving MIP solutions should be saved\"\"\"\n    mip_improving_solution_save: bool = pydantic.Field(\n        default=False,\n        validate_default=True,\n    )\n\n    \"\"\"Whether improving MIP solutions should be reported in sparse format\"\"\"\n    mip_improving_solution_report_sparse: bool = pydantic.Field(\n        default=False,\n        validate_default=True,\n    )\n\n    \"\"\"File for reporting improving MIP solutions: not reported for an empty string ''\"\"\"\n    mip_improving_solution_file: str = pydantic.Field(\n        default=\"\",\n        validate_default=True,\n    )\n\n    \"\"\"MIP solver max number of leave nodes\"\"\"\n    mip_max_leaves: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Limit on the number of improving solutions found to stop the MIP solver prematurely\"\"\"\n    mip_max_improving_sols: int = pydantic.Field(\n        default=2147483647,\n        ge=1,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Maximal age of dynamic LP rows before they are removed from the LP relaxation in the MIP solver\"\"\"\n    mip_lp_age_limit: int = pydantic.Field(\n        default=10,\n        ge=0,\n        le=32767,\n        validate_default=True,\n    )\n\n    \"\"\"Maximal age of rows in the MIP solver cutpool before they are deleted\"\"\"\n    mip_pool_age_limit: int = pydantic.Field(\n        default=30,\n        ge=0,\n        le=1000,\n        validate_default=True,\n    )\n\n    \"\"\"Soft limit on the number of rows in the MIP solver cutpool for dynamic age adjustment\"\"\"\n    mip_pool_soft_limit: int = pydantic.Field(\n        default=10000,\n        ge=1,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Minimal number of observations before MIP solver pseudo costs are considered reliable\"\"\"\n    mip_pscost_minreliable: int = pydantic.Field(\n        default=8,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Minimal number of entries in the MIP solver cliquetable before neighbourhood queries of the conflict graph use parallel processing\"\"\"\n    mip_min_cliquetable_entries_for_parallelism: int = pydantic.Field(\n        default=100000,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"MIP feasibility tolerance\"\"\"\n    mip_feasibility_tolerance: float = pydantic.Field(\n        default=1e-06,\n        ge=1e-10,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Effort spent for MIP heuristics\"\"\"\n    mip_heuristic_effort: float = pydantic.Field(\n        default=0.05,\n        ge=0,\n        le=1,\n        validate_default=True,\n    )\n\n    \"\"\"Tolerance on relative gap, |ub-lb|/|ub|, to determine whether optimality has been reached for a MIP instance\"\"\"\n    mip_rel_gap: float = pydantic.Field(\n        default=0.0001,\n        ge=0,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Tolerance on absolute gap of MIP, |ub-lb|, to determine whether optimality has been reached for a MIP instance\"\"\"\n    mip_abs_gap: float = pydantic.Field(\n        default=1e-06,\n        ge=0,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"MIP minimum logging interval\"\"\"\n    mip_min_logging_interval: float = pydantic.Field(\n        default=5,\n        ge=0,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n    \"\"\"Iteration limit for IPM solver\"\"\"\n    ipm_iteration_limit: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Use native termination for PDLP solver: Default = false\"\"\"\n    pdlp_native_termination: bool = pydantic.Field(\n        default=False,\n        validate_default=True,\n    )\n\n    \"\"\"Scaling option for PDLP solver: Default = true\"\"\"\n    pdlp_scaling: bool = pydantic.Field(\n        default=True,\n        validate_default=True,\n    )\n\n    \"\"\"Iteration limit for PDLP solver\"\"\"\n    pdlp_iteration_limit: int = pydantic.Field(\n        default=2147483647,\n        ge=0,\n        le=2147483647,\n        validate_default=True,\n    )\n\n    \"\"\"Restart mode for PDLP solver: 0 => none; 1 => GPU (default); 2 => CPU\"\"\"\n    pdlp_e_restart_method: int = pydantic.Field(\n        default=1,\n        ge=0,\n        le=2,\n        validate_default=True,\n    )\n\n    \"\"\"Duality gap tolerance for PDLP solver: Default = 1e-4\"\"\"\n    pdlp_d_gap_tol: float = pydantic.Field(\n        default=0.0001,\n        ge=1e-12,\n        le=float('inf'),\n        validate_default=True,\n    )\n\n\n    \"\"\"Make seed random if None\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _random_seed(self):\n        self.model_config[\"frozen\"] = False\n\n        if self.random_seed is None:\n            try:\n                min_int = self.model_fields[\"random_seed\"].metadata[0].ge\n                max_int = self.model_fields[\"random_seed\"].metadata[1].le\n            except:\n                min_int = 0\n                max_int = 2147483647\n\n            self.random_seed = np.random.randint(min_int, max_int)\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n\nif __name__ == \"__main__\":\n    s = HiGHS_Settings()\n\n    print(s.model_dump_json())\n"
  },
  {
    "path": "opendsm/comparison_groups/individual_meter_matching/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport pydantic\n\nimport opendsm.comparison_groups.common.const as _const\nfrom opendsm.common.base_settings import BaseSettings\n\nfrom enum import Enum\nfrom typing import Optional\n\n\nclass SelectionMethod(str, Enum):\n    MINIMIZE_METER_DISTANCE = \"minimize_meter_distance\"\n    MINIMIZE_LOADSHAPE_DISTANCE = \"minimize_loadshape_distance\"\n\n\nclass Settings(BaseSettings):\n    \"\"\"Settings for individual meter matching\"\"\"\n\n    \"\"\"distance metric to determine best comparison pool matches\"\"\"\n    distance_metric: _const.DistanceMetric = pydantic.Field(\n        default=_const.DistanceMetric.EUCLIDEAN, \n        validate_default=True,\n    )\n\n    \"\"\"selection method for comparison group matching\"\"\"\n    selection_method: SelectionMethod = pydantic.Field(\n        default=SelectionMethod.MINIMIZE_METER_DISTANCE, \n        validate_default=True,\n    )\n\n    \"\"\"number of comparison pool matches to each treatment meter\"\"\"\n    n_matches_per_treatment: int = pydantic.Field(\n        default=4, \n        ge=1, \n        validate_default=True,\n    )\n    \n    \"\"\"number of treatments to be calculated per chunk to prevent memory issues\"\"\"\n    n_treatments_per_chunk: int = pydantic.Field(\n        default=10000, \n        ge=1, \n        validate_default=True,\n    )\n    \n    \"\"\"allow duplicate matches in comparison group\"\"\"\n    allow_duplicate_matches: bool = pydantic.Field(\n        default=False, \n        validate_default=True,\n    )\n    \n    \"\"\"The maximum distance that a comparison group match can have with a given\n       treatment meter. These meters are filtered out after all matching has completed.\"\"\"\n    max_distance_threshold: Optional[float] = pydantic.Field(\n        default=None, \n        validate_default=True,\n    )\n\n    \"\"\"Check if valid settings for treatment meter match loss\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _check_allow_duplicates(self):\n        if self.allow_duplicate_matches:\n            if self.selection_method != SelectionMethod.MINIMIZE_METER_DISTANCE:\n                distance = SelectionMethod.MINIMIZE_METER_DISTANCE.value\n                raise ValueError(f\"If `allow_duplicate_matches` is True then `selection_method` must be '{distance}'\")\n\n        return self\n    \n\nif __name__ == \"__main__\":\n    s = Settings()\n\n    print(s.model_dump_json())\n"
  },
  {
    "path": "opendsm/comparison_groups/random_sampling/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.random_sampling.create_comparison_groups import Random_Sampling\nfrom opendsm.comparison_groups.random_sampling.settings import Settings as RS_Settings\n"
  },
  {
    "path": "opendsm/comparison_groups/random_sampling/create_comparison_groups.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.common.base_comparison_group import Comparison_Group_Algorithm\n\nfrom opendsm.comparison_groups.random_sampling.settings import Settings\n\n\nclass Random_Sampling(Comparison_Group_Algorithm):\n    def __init__(self, settings: Optional[Settings] = None):\n        if settings is None:\n            settings = Settings()\n        \n        self.settings = settings\n\n    def _create_clusters_df(self, df_raw):\n        clusters = df_raw\n        clusters[\"cluster\"] = 0\n        clusters[\"weight\"] = 1.0\n        clusters = clusters.reset_index().set_index(\"id\")\n\n        # reorder columns\n        clusters = clusters[[\"cluster\", \"weight\"]]\n\n        return clusters\n    \n\n    def _create_treatment_weights_df(self, ids):\n        coeffs = np.ones(len(ids))\n\n        treatment_weights = pd.DataFrame(coeffs, index=ids, columns=[\"pct_cluster_0\"])\n        treatment_weights.index.name = \"id\"\n\n        return treatment_weights\n    \n\n    def get_comparison_group(self, treatment_data, comparison_pool_data, weights=None):\n        settings = self.settings\n\n        if settings.n_meters_total is not None:\n            n_meters = self.settings.n_meters_total\n\n        elif settings.n_meters_per_treatment is not None:\n            n_treatment_meters = len(treatment_data.ids)\n            n_meters = n_treatment_meters * settings.n_meters_per_treatment\n\n        else:\n            raise ValueError(\"`n_meters_total` or `n_meters_per_treatment` must be defined\")\n\n        self.treatment_data = treatment_data\n        self.comparison_pool_data = comparison_pool_data\n\n        self.treatment_ids = treatment_data.ids\n        self.treatment_loadshape = treatment_data.loadshape\n        self.comparison_pool_loadshape = comparison_pool_data.loadshape\n\n        # randomly sample n_meters from comparison pool\n        df_cg = comparison_pool_data.loadshape.sample(n_meters, random_state=settings.seed)\n\n        clusters = self._create_clusters_df(df_cg)\n\n        # Create treatment_weights\n        treatment_weights = self._create_treatment_weights_df(self.treatment_ids)\n\n        # Assign dfs to self\n        self.clusters = clusters\n        self.treatment_weights = treatment_weights\n\n        return clusters, treatment_weights\n"
  },
  {
    "path": "opendsm/comparison_groups/random_sampling/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\nfrom typing import Optional\n\nimport pydantic\n\nfrom opendsm.common.base_settings import BaseSettings\n\n\nclass Settings(BaseSettings):\n    \"\"\"Settings for random sampling\"\"\"\n    \n    \"\"\"number meters to randomly sample from comparison pool\"\"\"\n    n_meters_total: Optional[int] = pydantic.Field(\n        default=None, \n        validate_default=True,\n    )\n\n    \"\"\"number of meters to randomly sample per treatment\"\"\"\n    n_meters_per_treatment: Optional[int] = pydantic.Field(\n        default=4, \n        validate_default=True,\n    )\n\n    seed: Optional[int] = pydantic.Field(\n        default=None, \n        validate_default=True,\n    )\n\n    \"\"\"Check if valid settings\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _check_n_meters_choice(self):\n        if self.n_meters_total is None and self.n_meters_per_treatment is None:\n            raise ValueError(\"`n_meters_total` or `n_meters_per_treatment` must be defined\")\n        \n        elif self.n_meters_total is not None and self.n_meters_per_treatment is not None:\n            raise ValueError(\"`n_meters_total` and `n_meters_per_treatment` cannot be defined together\")\n        \n        elif self.n_meters_total is not None and self.n_meters_total < 1:\n            raise ValueError(\"`n_meters_total` must be greater than or equal to 1\")\n\n        elif self.n_meters_per_treatment is not None and self.n_meters_per_treatment < 1:\n            raise ValueError(\"`n_meters_per_treatment` must be greater than or equal to 1\")\n\n        return self\n    \n\nif __name__ == \"__main__\":\n    s = Settings()\n\n    print(s.model_dump_json())\n"
  },
  {
    "path": "opendsm/comparison_groups/savings/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License."
  },
  {
    "path": "opendsm/comparison_groups/savings/archived_dev.py",
    "content": "\n\n\n# TODO: need to cap IMM size if doing this for memory reasons\n# TODO: In GRIDmeter potentially could reduce df_t_coeffs to remove unused clusters\n# Data classes previously made should be used here\n# should be accessible both low level and gridmeter+model correction together\n# should we move the loadshape methodology to the data class? - Yes\n\"\"\"\nEEmeter\nid: [datetime, temp, ghi, observed] -> EEmeter data -> EEmeter model -> model prediction\n\nGridmeter\n[id, datetime, observed] -> gridmeter data -> loadshape -> gridmeter cg assignment -> [df_cluster_id, df_t_coeffs]\n\nGridmeter (Model Correction)\n[[ids, EEmeter data] + df_cluster_id, df_t_coeffs] -> model correction -> corrected model\n\nGridmeter (Savings)\n[ids, datetime, temp, ghi, observed, corrected_model] -> per unit time savings (or aggregation)\n\n\"\"\"\n\n# class Model_Correction:\n#     def __init__(self, df_t_reporting, df_cg_reporting, df_cluster_id, df_t_coeffs, df_t_baseline=None, df_cg_baseline=None, settings=None):\n#         # should we also use a data class for df_t and df_cp? probably?\n#         self.df_t_reporting = df_t_reporting\n#         self.df_cg_reporting = df_cg_reporting\n#         self.df_cluster_id = df_cluster_id\n#         self.df_t_coeffs = df_t_coeffs\n\n#         # TODO: need to pull from settings and make settings class\n#         self.agg_type = \"mean\"\n#         self.reject_outliers = False\n#         self.scale_diff = True\n#         self.correction_method = \"ordinary_difference_in_differences\"\n\n#         self.data_settings = Data_Settings(AGG_TYPE=self.agg_type, LOADSHAPE_TYPE=\"modeled\")\n\n#         # calculate diffs for df_t and df_cp\n#         self.df_t = self._initialize_df(self.df_t, is_treatment=True)\n#         self.df_cg = self._initialize_df(self.df_cg, is_treatment=False)\n\n#         self.df_cluster = self._agg_cluster_data()\n#         self._df_t_cg = self._get_treatment_cg_data()\n\n    \n#     def _initialize_df(self, df, is_treatment=False):\n#         data_settings = self.data_settings\n        \n#         df[\"ratio\"] = df[\"observed\"]/df[\"modeled\"]\n#         df[\"diff\"] = df[\"modeled\"] - df[\"observed\"]\n\n#         df = add_datetime_loadshape_mapping_col(df, data_settings)\n\n#         if not is_treatment and self.scale_diff:\n#             period = \"baseline\"\n#             groupby_keys = [\"id\", \"ls_key\"]\n#             # groupby_keys = [\"id\"]\n\n#             df_p = df[df[\"period\"] == period]\n            \n#             df_p_grouped = df_p.groupby(groupby_keys)\n#             for col in [\"diff\"]:\n#                 # calculate IQR\n#                 scale = df_p_grouped[col].quantile(0.75) - df_p_grouped[col].quantile(0.25)\n#                 scale = scale.rename(f\"{col}_scale\")\n\n#                 # add to df\n#                 df = df.merge(scale, left_on=groupby_keys, right_index=True)\n\n#                 df[col] /= df[f\"{col}_scale\"]\n\n#         # get columns to aggregate on etc\n#         # TODO: remove unnecessary columns such as temperature, observed, modeled?\n#         cols_drop = [\"id\", \"datetime\", \"period\"]\n#         # cols_drop.extend([col for col in df.columns if col.endswith(\"_scale\")])\n#         self.df_cols = [col for col in df.columns if col not in cols_drop]\n\n#         return df\n\n\n#     def _agg_cluster_data(self):\n#         df_cp = self.df_cg\n#         df_cluster_id = self.df_cluster_id\n#         agg_type = self.agg_type\n\n#         # get cluster data\n#         # merge df_cp_period with df_cg to get cluster number\n#         df_cp = df_cp.merge(df_cluster_id[[\"cluster\"]], left_on=\"id\", right_index=True)\n\n#         df_cols = [col for col in self.df_cols if col not in [\"temperature\", \"ls_key\"]]\n#         df_cp_groupby = df_cp[[\"cluster\", \"datetime\", *df_cols]].groupby([\"cluster\", \"datetime\"])\n\n#         if self.reject_outliers:\n#             label_dict = {0.25: \"Q1\", 0.75: \"Q3\"}\n#             df_cp_iqr = df_cp_groupby.quantile([0.25, 0.75]).unstack()\n#             df_cp_iqr.columns = [f\"{col}_{label_dict[q]}\" for col, q in df_cp_iqr.columns]\n\n#             # join iqr data with original data\n#             df_cp = df_cp.merge(df_cp_iqr, on=[\"cluster\", \"datetime\"])\n\n#             # get cluster data\n#             df_cluster = pd.concat(\n#                 [df_cp[[\"cluster\", \"datetime\", \"ls_key\"]].groupby([\"cluster\", \"datetime\"]).first(),\n#                  df_cp[[\"cluster\", \"datetime\", \"temperature\"]].groupby([\"cluster\", \"datetime\"]).median()], \n#                 axis=1)\n\n#             for col in df_cols:\n#                 Q1 = df_cp[f\"{col}_Q1\"]\n#                 Q3 = df_cp[f\"{col}_Q3\"]\n#                 IQR = Q3 - Q1\n\n#                 temp = df_cp[[\"cluster\", \"datetime\", col]]\n#                 temp = temp[(temp[col] >= Q1 - 1.5*IQR) & (temp[col] <= Q3 + 1.5*IQR)]\n#                 temp = temp[[\"cluster\", \"datetime\", col]].groupby([\"cluster\", \"datetime\"]).median()\n\n#                 df_cluster = pd.concat([df_cluster, temp], axis=1)\n\n#             df_cluster = df_cluster.reset_index()\n            \n#         else:\n#             agg_dict = {col: agg_type for col in self.df_cols}\n\n#             df_cluster = df_cp.groupby([\"cluster\", \"datetime\"]).agg(agg_dict).reset_index()\n\n#         # get columns that end in _scale\n#         cols_scaled = [col.replace(\"_scale\", \"\") for col in df_cluster.columns if col.endswith(\"_scale\")]\n#         for col in cols_scaled:\n#             df_cluster[col] *= df_cluster[f\"{col}_scale\"]\n        \n#         return df_cluster\n\n\n#     def _get_treatment_cg_data(self):\n#         df_t = self.df_t\n#         df_cluster = self.df_cluster\n#         df_t_coeffs = self.df_t_coeffs\n\n#         # rescale \n#         # get comparison group data for each id\n#         df_cluster = df_cluster[df_cluster[\"cluster\"] != -1]\n#         g = df_cluster.groupby('cluster', sort=False).cumcount()\n        \n#         cluster_data = np.array(df_cluster.set_index(['cluster', g])[self.df_cols]\n#             .unstack(fill_value=1E30)    # replace any empty values with one\n#             .stack().groupby(level=0)\n#             .apply(lambda x: x.values.tolist())\n#             .tolist())\n\n#         t_coeffs = df_t_coeffs.values\n\n#         # multiplies each cluster by the percentage for each treatment meter and sums them per hour\n#         cg = {}\n#         for n, col in enumerate(self.df_cols):\n#             cg[col] = np.einsum(\"ij,ik->jk\", cluster_data[:,:,n], t_coeffs.T).T\n\n#         t_datetime_contiguous = np.sort(df_t[\"datetime\"].unique())\n#         cg_datetime_contiguous = np.sort(df_cluster[\"datetime\"].unique())\n\n#         if np.all(t_datetime_contiguous != cg_datetime_contiguous):\n#             raise ValueError(\"Treatment and Comparison Group datetime arrays do not match\")\n\n#         # repeat datetime array for each treatment meter\n#         cg_datetime = np.tile(cg_datetime_contiguous, cg[\"temperature\"].shape[0])\n#         cg_ids = np.repeat(df_t_coeffs.index, cg[\"temperature\"].shape[1])\n\n#         df_cg_dict = {\"id\": cg_ids, \"datetime\": cg_datetime}\n#         df_cg_dict.update({col: cg[col].flatten() for col in self.df_cols})\n        \n#         df_cg = pd.DataFrame(df_cg_dict)\n\n#         df_cg[\"datetime\"] = pd.to_datetime(df_cg[\"datetime\"])\n\n#         # join df_t_period and df_cg_period on id and datetime\n#         df_t_cg = pd.merge(df_t, df_cg, on=[\"id\", \"datetime\"], suffixes=[\"_t\", \"_cg\"])\n\n#         return df_t_cg\n\n\n#     def add_pct_did(self, simplified_eqn=False):\n#         df = self._df_t_cg\n\n#         if simplified_eqn:\n#             cg_factor = df[\"ratio_cg\"]\n#             res = cg_factor*df[\"modeled_t\"] - df[\"observed_t\"]\n\n#         else:\n#             res = df[\"diff_t\"] - df[\"diff_cg\"]*df[\"modeled_t\"]/df[\"modeled_cg\"]\n\n#         self._df_t_cg[\"%did\"] = res\n\n\n#     def add_abs_pct_did(self, simplified_eqn=False):\n#         df = self._df_t_cg\n\n#         if simplified_eqn:\n#             res = np.empty(len(df))\n\n#             # get sign matching indices\n#             match = np.sign(df[\"modeled_t\"]) == np.sign(df[\"modeled_cg\"])          \n#             res[match] = df[\"ratio_cg\"][match]*df[\"modeled_t\"][match] - df[\"observed_t\"][match]\n#             res[~match] = (2 - df[\"ratio_cg\"][~match])*df[\"modeled_t\"][~match] - df[\"observed_t\"][~match]\n\n#         else:\n#             res = df[\"diff_t\"] - df[\"diff_cg\"]*(df[\"modeled_t\"]/df[\"modeled_cg\"]).abs()\n\n#         self._df_t_cg[\"abs_%did\"] = res\n\n\n#     def add_sig_pct_did(self, k=0.01, m_0=0.1):\n#         df = self._df_t_cg\n\n#         if \"abs_%did\" not in df.columns:\n#             self.add_abs_pct_did(simplified_eqn=True)\n\n#         # df[\"scale\"] = (df[\"abs_%did\"] + df[\"observed_t\"])/df[\"modeled_t\"]\n\n#         scale = (\n#             ((df[\"modeled_t\"] - df[\"observed_t\"])*sigmoid(np.abs(df[\"modeled_t\"]), m_0, k) + df[\"observed_t\"]) / \n#             ((df[\"modeled_cg\"] - df[\"observed_cg\"])*sigmoid(np.abs(df[\"modeled_cg\"]), m_0, k) + df[\"observed_cg\"])\n#         )\n\n#         # scale = (\n#         #     (df[\"diff_t\"]*sigmoid(np.abs(df[\"modeled_t\"]), m_0, k) + df[\"observed_t\"]) / \n#         #     (df[\"diff_cg\"]*sigmoid(np.abs(df[\"modeled_cg\"]), m_0, k) + df[\"observed_cg\"])\n#         # )\n\n#         res = df[\"diff_t\"] - df[\"diff_cg\"]*np.abs(scale)\n\n#         self._df_t_cg[\"sig_%did\"] = res\n    \n\n#     def add_scaled_ordinary_did(self):\n#         # calculate scaled ordinary difference in differences\n\n#         df = self._df_t_cg\n#         cols = df.columns\n#         data_settings = self.data_settings\n\n#         comparison_col = \"diff\" # modeled or diff?\n\n#         comp_t = f\"{comparison_col}_t\"\n#         df_t_baseline = df[df[\"period\"] == \"baseline\"][[\"id\", \"datetime\", comp_t]]\n#         df_t_baseline = df_t_baseline.rename(columns={comp_t: \"modeled\"})\n#         data_t = gm.Data(time_series_df=df_t_baseline, settings=data_settings)\n\n#         comp_cg = f\"{comparison_col}_cg\"\n#         df_cg_baseline = df[df[\"period\"] == \"baseline\"][[\"id\", \"datetime\", comp_cg]]\n#         df_cg_baseline = df_cg_baseline.rename(columns={comp_cg: \"modeled\"})\n#         data_cg = gm.Data(time_series_df=df_cg_baseline, settings=data_settings)\n\n#         # scale based on loadshape in baseline period\n#         df_cg_scale = data_t.loadshape/data_cg.loadshape\n\n#         df_cg_scale = df_cg_scale.unstack().reset_index().rename(columns={\"level_0\": \"ls_key\", 0: \"scale\"})\n\n#         # merge df_cg_scale with df_t_cg on ls_key and id\n#         df = add_datetime_loadshape_mapping_col(df, data_settings)\n#         df = df.merge(df_cg_scale, on=[\"id\", \"ls_key\"])\n\n#         df[\"sodid\"] = df[\"diff_t\"] - df[\"diff_cg\"]*df[\"scale\"]\n#         df = df.rename(columns={\"scale\": \"sodid_scale\"})\n\n#         # remove all columns except input and sodid columns\n#         self._df_t_cg = df[[*cols, \"sodid\", \"sodid_scale\"]]\n\n\n#     def add_modeled_scaled_ordinary_did(self):\n#         df_t_cg = self._df_t_cg\n\n#         df_t_cg[\"diff_ratio\"] = df_t_cg[\"diff_t\"]/df_t_cg[\"diff_cg\"]\n\n#         df_ratio = df_t_cg[[\"id\", \"datetime\", \"period\", \"temperature_cg\", \"diff_ratio\"]]\n#         df_ratio = df_ratio.rename(columns={\"temperature_cg\": \"temperature\", \"diff_ratio\": \"observed\"})\n\n#         ratio_modeled = []\n#         for id in df_ratio[\"id\"].unique():\n#             df_ratio_id = df_ratio[df_ratio[\"id\"] == id]\n#             df_ratio_id_baseline =  df_ratio_id[df_ratio_id[\"period\"] == \"baseline\"][[\"datetime\", \"temperature\", \"observed\"]]\n\n#             settings = em.HourlySettings()\n#             model = em.HourlyModel(settings)\n#             model.fit(df_ratio_id_baseline)\n\n#             df_predict = model.predict(df_ratio_id[[\"datetime\", \"temperature\", \"observed\"]])\n#             df_predict = df_predict.reset_index()\n#             df_predict.insert(0, \"id\", id)\n\n#             ratio_modeled.append(df_predict)\n\n#         df_scale = pd.concat(ratio_modeled, ignore_index=True)\n\n#         # merge df_t_cg and df_scale on id and datetime\n#         df_t_cg[\"scale_predicted\"] = df_scale[\"predicted\"]\n\n#         # calculate model scale did\n#         res = df_t_cg[\"diff_t\"] - df_t_cg[\"diff_cg\"]*df_t_cg[\"scale_predicted\"]\n\n#         self._df_t_cg[\"modeled_sodid\"] = res\n\n\n#     def _get_did_cols(self):\n#         all_did_cols = [\"%did\", \"abs_%did\", \"sig_%did\", \"sodid\", \"modeled_sodid\"]\n#         did_cols = [col for col in all_did_cols if col in self._df_t_cg.columns]\n\n#         return did_cols\n\n#     @cached_property\n#     def df(self):\n#         # get which columns exist in %did, abs_%did sodid, modeled_sodid\n#         did_cols = self._get_did_cols()\n\n#         # if observed_t, observed_cg, modeled_t, or modeled_cg are nan, then did cols are nan\n#         df_t_cg = self._df_t_cg\n#         measured_cols = [\"observed_t\", \"observed_cg\", \"modeled_t\", \"modeled_cg\"]\n#         df_t_cg[did_cols] = df_t_cg[did_cols].where(\n#             ~df_t_cg[measured_cols].isna().any(axis=1),\n#             np.nan\n#         )\n\n#         # remove diff_t and diff_cg columns\n#         df_t_cg = df_t_cg.drop(columns=[\"diff_t\", \"diff_cg\"])\n\n#         return df_t_cg\n    \n\n#     def df_agg(self, period=\"reporting\"):\n#         # TODO: This is only for testing new did methods\n#         self.add_pct_did(simplified_eqn=True)\n#         self.add_abs_pct_did(simplified_eqn=True)\n#         # self.add_sig_pct_did()\n#         self.add_scaled_ordinary_did()\n\n#         did_cols = self._get_did_cols()\n\n#         df_t_cg = self.df[self.df[\"period\"] == period]\n\n#         # groupby id and aggregate observed, modeled and did_cols\n#         agg_dict = {\n#             \"observed_t\": \"sum\",\n#             \"modeled_t\": \"sum\",\n#             \"observed_cg\": \"sum\",\n#             \"modeled_cg\": \"sum\",\n#         }\n#         agg_dict.update({col: \"sum\" for col in did_cols})\n\n#         return df_t_cg.groupby(\"id\").agg(agg_dict)\n\n\n#     def df_stats(self, period=\"reporting\"):\n#         df_res = self.df_agg(period)\n\n#         # count number of unique ids\n#         id_count = len(df_res)\n\n#         # calculate mean and uncertainty of each did_column\n#         stats = {}\n#         for col in self._get_did_cols():\n#             mean = df_res[col].mean()\n#             unc = df_res[col].std()*unc_factor(id_count, alpha=0.05, interval=\"CI\")\n\n#             stats.update({f\"{col}\": [mean], f\"{col}_unc\": [unc]})\n\n#         return pd.DataFrame(stats)\n"
  },
  {
    "path": "opendsm/comparison_groups/savings/cg_correction_testing.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\nimport numpy as np\n\nimport random\n\nfrom functools import cached_property\n\nfrom opendsm import eemeter as em\nfrom opendsm import comparison_groups as cg\n\nfrom opendsm.common.utils import unc_factor\nfrom opendsm.common.utils import sigmoid\n\n\ndef get_t_cg_df(data, num_treatment=None, num_control=None, seed=21):\n    def get_subpopulation(df, ids):\n        df = df.reset_index()\n        df = df[df[\"dpsm_id\"].isin(ids)]\n        df = df.rename(columns={\"dpsm_id\": \"id\", \"start_local\": \"datetime\", \"model\": \"modeled\"})\n\n        period = [\"baseline\", \"reporting\"]\n        df = df[df[\"period\"].isin(period)]\n\n        # remove datetimes after 1 year from first reporting period date\n        first_reporting_date = df[df[\"period\"] == \"reporting\"][\"datetime\"].min()\n        df = df[df[\"datetime\"] < first_reporting_date + pd.Timedelta(days=365)]\n        \n        return df\n    \n    # get list of ids\n    id_list = list(data.df[\"meter\"].index.unique())\n\n    random.seed(seed)\n    if num_treatment is None:\n        # assign treatment ids in same proportion as original cluster research (1000 cp, 100 t)\n        num_treatment = int(round(100*(len(id_list)/1100)))\n\n    treatment_ids = random.sample(id_list, num_treatment)\n    pool_ids = [x for x in id_list if x not in treatment_ids]\n\n    if num_control is not None:\n        num_control = int(num_control*len(treatment_ids))\n        if num_control <= len(pool_ids):         \n            pool_ids = random.sample(pool_ids, num_control)\n\n    # get treatment and pool dataframes\n    df_t = get_subpopulation(data.df[\"meter\"], treatment_ids)\n    df_cp = get_subpopulation(data.df[\"meter\"], pool_ids)\n\n    return df_t, df_cp\n\n\ndef get_comparison_groups(df_t, df_cp, agg, cg_type=\"cluster\", multiprocessing=True):\n    # set data classes\n    data_settings = gm.Data_Settings(AGG_TYPE=agg, LOADSHAPE_TYPE=\"modeled\")\n\n    data_cls = {\n        \"t\":  gm.Data(time_series_df=df_t[df_t[\"period\"] == \"baseline\"], settings=data_settings), \n        \"cp\": gm.Data(time_series_df=df_cp[df_cp[\"period\"] == \"baseline\"], settings=data_settings),\n    }\n\n    if \"cluster\" in cg_type.lower():\n        # get clustered comparison groups\n        clustering_settings = gm.Clustering_Settings(USE_MULTIPROCESSING=multiprocessing)\n        clustering = gm.Clustering(clustering_settings)\n        return clustering.get_comparison_group(data_cls[\"t\"], data_cls[\"cp\"])\n\n    elif \"imm\" in cg_type.lower():\n        # get IMM comparison groups\n        imm_settings = gm.IMM_Settings(USE_MULTIPROCESSING=multiprocessing)\n        imm = gm.IMM(imm_settings)\n        return imm.get_comparison_group(data_cls[\"t\"], data_cls[\"cp\"])\n    \n    else:\n        raise ValueError(\"cg_type must be either 'cluster' or 'imm'\")\n\n\ndef add_datetime_loadshape_mapping_col(df, data_settings):\n    # get mapping between datetime and loadshape\n    if data_settings.TIME_PERIOD != 'seasonal_hourly_day_of_week':\n        raise ValueError(\"This only works for seasonal_hourly_day_of_week\")\n\n    df_key = pd.DataFrame({\"datetime\": df[\"datetime\"].unique()})\n    df_key[\"month\"] = df_key[\"datetime\"].dt.month\n\n    # map month to season using _NUM_DICT\n    df_key[\"season\"] = df_key[\"month\"].map(data_settings.SEASON._NUM_DICT)\n    df_key[\"season_num\"] = df_key[\"season\"].map(data_settings.SEASON._ORDER)\n\n    df_key[\"hour_of_week\"] = df_key[\"datetime\"].dt.dayofweek*24 + df_key[\"datetime\"].dt.hour\n\n    df_key[\"ls_key\"] = df_key[\"season_num\"]*24*7 + df_key[\"hour_of_week\"] + 1\n\n    df_key = df_key.set_index(\"datetime\")\n    df_key = df_key[\"ls_key\"]\n\n    # merge df_t_cg and df_dt_ls_key on datetime\n    df = df.merge(df_key, left_on=\"datetime\", right_index=True)\n\n    return df\n\n# TODO: need to cap IMM size if doing this for memory reasons\nclass Savings:\n    def __init__(self, df_t, df_cp, df_cluster_id, df_t_coeffs, agg_type=\"mean\", reject_outliers=False, scale_diff=True):\n        self.df_t = df_t\n        self.df_cp = df_cp\n        self.df_cluster_id = df_cluster_id\n        self.df_t_coeffs = df_t_coeffs\n\n        self.agg_type = agg_type\n        self.reject_outliers = reject_outliers\n        self.scale_diff = scale_diff\n\n        self.data_settings = gm.Data_Settings(AGG_TYPE=agg_type, LOADSHAPE_TYPE=\"modeled\")\n\n        # calculate diffs for df_t and df_cp\n        self.df_t = self._initialize_df(self.df_t, is_treatment=True)\n        self.df_cp = self._initialize_df(self.df_cp, is_treatment=False)\n\n        self.df_cluster = self._agg_cluster_data()\n        self._df_t_cg = self._get_treatment_cg_data()\n\n    \n    def _initialize_df(self, df, is_treatment=False):\n        data_settings = self.data_settings\n        \n        df[\"ratio\"] = df[\"observed\"]/df[\"modeled\"]\n        df[\"diff\"] = df[\"modeled\"] - df[\"observed\"]\n\n        df = add_datetime_loadshape_mapping_col(df, data_settings)\n\n        if not is_treatment and self.scale_diff:\n            period = \"baseline\"\n            groupby_keys = [\"id\", \"ls_key\"]\n            # groupby_keys = [\"id\"]\n\n            df_p = df[df[\"period\"] == period]\n            \n            df_p_grouped = df_p.groupby(groupby_keys)\n            for col in [\"diff\"]:\n                # calculate IQR\n                scale = df_p_grouped[col].quantile(0.75) - df_p_grouped[col].quantile(0.25)\n                scale = scale.rename(f\"{col}_scale\")\n\n                # add to df\n                df = df.merge(scale, left_on=groupby_keys, right_index=True)\n\n                df[col] /= df[f\"{col}_scale\"]\n\n        # get columns to aggregate on etc\n        # TODO: remove unnecessary columns such as temperature, observed, modeled?\n        cols_drop = [\"id\", \"datetime\", \"period\"]\n        # cols_drop.extend([col for col in df.columns if col.endswith(\"_scale\")])\n        self.df_cols = [col for col in df.columns if col not in cols_drop]\n\n        return df\n\n\n    def _agg_cluster_data(self):\n        df_cp = self.df_cp\n        df_cluster_id = self.df_cluster_id\n        agg_type = self.agg_type\n\n        # get cluster data\n        # merge df_cp_period with df_cg to get cluster number\n        df_cp = df_cp.merge(df_cluster_id[[\"cluster\"]], left_on=\"id\", right_index=True)\n\n        df_cols = [col for col in self.df_cols if col not in [\"temperature\", \"ls_key\"]]\n        df_cp_groupby = df_cp[[\"cluster\", \"datetime\", *df_cols]].groupby([\"cluster\", \"datetime\"])\n\n        if self.reject_outliers:\n            label_dict = {0.25: \"Q1\", 0.75: \"Q3\"}\n            df_cp_iqr = df_cp_groupby.quantile([0.25, 0.75]).unstack()\n            df_cp_iqr.columns = [f\"{col}_{label_dict[q]}\" for col, q in df_cp_iqr.columns]\n\n            # join iqr data with original data\n            df_cp = df_cp.merge(df_cp_iqr, on=[\"cluster\", \"datetime\"])\n\n            # get cluster data\n            df_cluster = pd.concat(\n                [df_cp[[\"cluster\", \"datetime\", \"ls_key\"]].groupby([\"cluster\", \"datetime\"]).first(),\n                 df_cp[[\"cluster\", \"datetime\", \"temperature\"]].groupby([\"cluster\", \"datetime\"]).median()], \n                axis=1)\n\n            for col in df_cols:\n                Q1 = df_cp[f\"{col}_Q1\"]\n                Q3 = df_cp[f\"{col}_Q3\"]\n                IQR = Q3 - Q1\n\n                temp = df_cp[[\"cluster\", \"datetime\", col]]\n                temp = temp[(temp[col] >= Q1 - 1.5*IQR) & (temp[col] <= Q3 + 1.5*IQR)]\n                temp = temp[[\"cluster\", \"datetime\", col]].groupby([\"cluster\", \"datetime\"]).median()\n\n                df_cluster = pd.concat([df_cluster, temp], axis=1)\n\n            df_cluster = df_cluster.reset_index()\n            \n        else:\n            agg_dict = {col: agg_type for col in self.df_cols}\n\n            df_cluster = df_cp.groupby([\"cluster\", \"datetime\"]).agg(agg_dict).reset_index()\n\n        # get columns that end in _scale\n        cols_scaled = [col.replace(\"_scale\", \"\") for col in df_cluster.columns if col.endswith(\"_scale\")]\n        for col in cols_scaled:\n            df_cluster[col] *= df_cluster[f\"{col}_scale\"]\n        \n        return df_cluster\n\n\n    def _get_treatment_cg_data(self):\n        df_t = self.df_t\n        df_cluster = self.df_cluster\n        df_t_coeffs = self.df_t_coeffs\n\n        # rescale \n        # get comparison group data for each id\n        df_cluster = df_cluster[df_cluster[\"cluster\"] != -1]\n        g = df_cluster.groupby('cluster', sort=False).cumcount()\n        \n        cluster_data = np.array(df_cluster.set_index(['cluster', g])[self.df_cols]\n            .unstack(fill_value=1E30)    # replace any empty values with one\n            .stack().groupby(level=0)\n            .apply(lambda x: x.values.tolist())\n            .tolist())\n\n        t_coeffs = df_t_coeffs.values\n\n        # multiplies each cluster by the percentage for each treatment meter and sums them per hour\n        cg = {}\n        for n, col in enumerate(self.df_cols):\n            cg[col] = np.einsum(\"ij,ik->jk\", cluster_data[:,:,n], t_coeffs.T).T\n\n        t_datetime_contiguous = np.sort(df_t[\"datetime\"].unique())\n        cg_datetime_contiguous = np.sort(df_cluster[\"datetime\"].unique())\n\n        if np.all(t_datetime_contiguous != cg_datetime_contiguous):\n            raise ValueError(\"Treatment and Comparison Group datetime arrays do not match\")\n\n        # repeat datetime array for each treatment meter\n        cg_datetime = np.tile(cg_datetime_contiguous, cg[\"temperature\"].shape[0])\n        cg_ids = np.repeat(df_t_coeffs.index, cg[\"temperature\"].shape[1])\n\n        df_cg_dict = {\"id\": cg_ids, \"datetime\": cg_datetime}\n        df_cg_dict.update({col: cg[col].flatten() for col in self.df_cols})\n        \n        df_cg = pd.DataFrame(df_cg_dict)\n\n        df_cg[\"datetime\"] = pd.to_datetime(df_cg[\"datetime\"])\n\n        # join df_t_period and df_cg_period on id and datetime\n        df_t_cg = pd.merge(df_t, df_cg, on=[\"id\", \"datetime\"], suffixes=[\"_t\", \"_cg\"])\n\n        return df_t_cg\n\n\n    def add_pct_did(self, simplified_eqn=False):\n        df = self._df_t_cg\n\n        if simplified_eqn:\n            cg_factor = df[\"ratio_cg\"]\n            res = cg_factor*df[\"modeled_t\"] - df[\"observed_t\"]\n\n        else:\n            res = df[\"diff_t\"] - df[\"diff_cg\"]*df[\"modeled_t\"]/df[\"modeled_cg\"]\n\n        self._df_t_cg[\"%did\"] = res\n\n\n    def add_abs_pct_did(self, simplified_eqn=False):\n        df = self._df_t_cg\n\n        if simplified_eqn:\n            res = np.empty(len(df))\n\n            # get sign matching indices\n            match = np.sign(df[\"modeled_t\"]) == np.sign(df[\"modeled_cg\"])          \n            res[match] = df[\"ratio_cg\"][match]*df[\"modeled_t\"][match] - df[\"observed_t\"][match]\n            res[~match] = (2 - df[\"ratio_cg\"][~match])*df[\"modeled_t\"][~match] - df[\"observed_t\"][~match]\n\n        else:\n            res = df[\"diff_t\"] - df[\"diff_cg\"]*(df[\"modeled_t\"]/df[\"modeled_cg\"]).abs()\n\n        self._df_t_cg[\"abs_%did\"] = res\n\n\n    def add_sig_pct_did(self, k=0.01, m_0=0.1):\n        df = self._df_t_cg\n\n        if \"abs_%did\" not in df.columns:\n            self.add_abs_pct_did(simplified_eqn=True)\n\n        # df[\"scale\"] = (df[\"abs_%did\"] + df[\"observed_t\"])/df[\"modeled_t\"]\n\n        scale = (\n            ((df[\"modeled_t\"] - df[\"observed_t\"])*sigmoid(np.abs(df[\"modeled_t\"]), m_0, k) + df[\"observed_t\"]) / \n            ((df[\"modeled_cg\"] - df[\"observed_cg\"])*sigmoid(np.abs(df[\"modeled_cg\"]), m_0, k) + df[\"observed_cg\"])\n        )\n\n        # scale = (\n        #     (df[\"diff_t\"]*sigmoid(np.abs(df[\"modeled_t\"]), m_0, k) + df[\"observed_t\"]) / \n        #     (df[\"diff_cg\"]*sigmoid(np.abs(df[\"modeled_cg\"]), m_0, k) + df[\"observed_cg\"])\n        # )\n\n        res = df[\"diff_t\"] - df[\"diff_cg\"]*np.abs(scale)\n\n        self._df_t_cg[\"sig_%did\"] = res\n    \n\n    def add_scaled_ordinary_did(self):\n        # calculate scaled ordinary difference in differences\n\n        df = self._df_t_cg\n        cols = df.columns\n        data_settings = self.data_settings\n\n        comparison_col = \"diff\" # modeled or diff?\n\n        comp_t = f\"{comparison_col}_t\"\n        df_t_baseline = df[df[\"period\"] == \"baseline\"][[\"id\", \"datetime\", comp_t]]\n        df_t_baseline = df_t_baseline.rename(columns={comp_t: \"modeled\"})\n        data_t = gm.Data(time_series_df=df_t_baseline, settings=data_settings)\n\n        comp_cg = f\"{comparison_col}_cg\"\n        df_cg_baseline = df[df[\"period\"] == \"baseline\"][[\"id\", \"datetime\", comp_cg]]\n        df_cg_baseline = df_cg_baseline.rename(columns={comp_cg: \"modeled\"})\n        data_cg = gm.Data(time_series_df=df_cg_baseline, settings=data_settings)\n\n        # scale based on loadshape in baseline period\n        df_cg_scale = data_t.loadshape/data_cg.loadshape\n\n        df_cg_scale = df_cg_scale.unstack().reset_index().rename(columns={\"level_0\": \"ls_key\", 0: \"scale\"})\n\n        # merge df_cg_scale with df_t_cg on ls_key and id\n        df = add_datetime_loadshape_mapping_col(df, data_settings)\n        df = df.merge(df_cg_scale, on=[\"id\", \"ls_key\"])\n\n        df[\"sodid\"] = df[\"diff_t\"] - df[\"diff_cg\"]*df[\"scale\"]\n        df = df.rename(columns={\"scale\": \"sodid_scale\"})\n\n        # remove all columns except input and sodid columns\n        self._df_t_cg = df[[*cols, \"sodid\", \"sodid_scale\"]]\n\n\n    def add_modeled_scaled_ordinary_did(self):\n        df_t_cg = self._df_t_cg\n\n        df_t_cg[\"diff_ratio\"] = df_t_cg[\"diff_t\"]/df_t_cg[\"diff_cg\"]\n\n        df_ratio = df_t_cg[[\"id\", \"datetime\", \"period\", \"temperature_cg\", \"diff_ratio\"]]\n        df_ratio = df_ratio.rename(columns={\"temperature_cg\": \"temperature\", \"diff_ratio\": \"observed\"})\n\n        ratio_modeled = []\n        for id in df_ratio[\"id\"].unique():\n            df_ratio_id = df_ratio[df_ratio[\"id\"] == id]\n            df_ratio_id_baseline =  df_ratio_id[df_ratio_id[\"period\"] == \"baseline\"][[\"datetime\", \"temperature\", \"observed\"]]\n\n            settings = em.HourlySettings()\n            model = em.HourlyModel(settings)\n            model.fit(df_ratio_id_baseline)\n\n            df_predict = model.predict(df_ratio_id[[\"datetime\", \"temperature\", \"observed\"]])\n            df_predict = df_predict.reset_index()\n            df_predict.insert(0, \"id\", id)\n\n            ratio_modeled.append(df_predict)\n\n        df_scale = pd.concat(ratio_modeled, ignore_index=True)\n\n        # merge df_t_cg and df_scale on id and datetime\n        df_t_cg[\"scale_predicted\"] = df_scale[\"predicted\"]\n\n        # calculate model scale did\n        res = df_t_cg[\"diff_t\"] - df_t_cg[\"diff_cg\"]*df_t_cg[\"scale_predicted\"]\n\n        self._df_t_cg[\"modeled_sodid\"] = res\n\n\n    def _get_did_cols(self):\n        all_did_cols = [\"%did\", \"abs_%did\", \"sig_%did\", \"sodid\", \"modeled_sodid\"]\n        did_cols = [col for col in all_did_cols if col in self._df_t_cg.columns]\n\n        return did_cols\n\n    @cached_property\n    def df(self):\n        # get which columns exist in %did, abs_%did sodid, modeled_sodid\n        did_cols = self._get_did_cols()\n\n        # if observed_t, observed_cg, modeled_t, or modeled_cg are nan, then did cols are nan\n        df_t_cg = self._df_t_cg\n        measured_cols = [\"observed_t\", \"observed_cg\", \"modeled_t\", \"modeled_cg\"]\n        df_t_cg[did_cols] = df_t_cg[did_cols].where(\n            ~df_t_cg[measured_cols].isna().any(axis=1),\n            np.nan\n        )\n\n        # remove diff_t and diff_cg columns\n        df_t_cg = df_t_cg.drop(columns=[\"diff_t\", \"diff_cg\"])\n\n        return df_t_cg\n    \n\n    def df_agg(self, period=\"reporting\"):\n        # TODO: This is only for testing new did methods\n        self.add_pct_did(simplified_eqn=True)\n        self.add_abs_pct_did(simplified_eqn=True)\n        # self.add_sig_pct_did()\n        self.add_scaled_ordinary_did()\n\n        did_cols = self._get_did_cols()\n\n        df_t_cg = self.df[self.df[\"period\"] == period]\n\n        # groupby id and aggregate observed, modeled and did_cols\n        agg_dict = {\n            \"observed_t\": \"sum\",\n            \"modeled_t\": \"sum\",\n            \"observed_cg\": \"sum\",\n            \"modeled_cg\": \"sum\",\n        }\n        agg_dict.update({col: \"sum\" for col in did_cols})\n\n        return df_t_cg.groupby(\"id\").agg(agg_dict)\n\n\n    def df_stats(self, period=\"reporting\"):\n        df_res = self.df_agg(period)\n\n        # count number of unique ids\n        id_count = len(df_res)\n\n        # calculate mean and uncertainty of each did_column\n        stats = {}\n        for col in self._get_did_cols():\n            mean = df_res[col].mean()\n            unc = df_res[col].std()*unc_factor(id_count, alpha=0.05, interval=\"CI\")\n\n            stats.update({f\"{col}\": [mean], f\"{col}_unc\": [unc]})\n\n        return pd.DataFrame(stats)"
  },
  {
    "path": "opendsm/comparison_groups/savings/model_correction.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.common.data_settings import Data_Settings\nfrom opendsm.common.stats.outliers_transformed import remove_outliers\nfrom opendsm.common.stats.basic import fast_std, unc_factor\n\nimport opendsm.comparison_groups.savings.settings as _settings\n\n\n\ndef _unit_correction_unc(\n    oTr, \n    mTr,\n    oCGr, \n    mCGr,\n    scale,\n    CG_diff,\n    correction,\n    oTr_unc,\n    mTr_unc,\n    oCGr_unc,\n    mCGr_unc,\n    CGr_corr, # only needed if oCGr_unc != 0\n    method=None\n):\n    \"\"\"Calculates correction uncertainty for each comparison group meter of a single treatment meter for a single hour\n    \n    Args:\n        oTr_unc: treatment meter observed uncertainty from reporting period\n        mTr_unc: treatment meter model uncertainty from reporting period\n        oCGr_unc: comparison group observed uncertainty from reporting period\n        mCGr_unc: comparison group model uncertainty from reporting period\n        CGr_corr: correlation between oCGr and mCGr over entire reporting period for each meter\n        scale: scale factor used in correction calculation\n        scale_var: variance of scale factor used in correction calculation\n    \"\"\"\n    # The generalized function: m_cT = m_T - s_CG∙(m_CG - o_CG)\n    # Correction = s_CG∙(m_CG - o_CG)\n\n    mTr_var = mTr_unc**2\n    mCGr_var = mCGr_unc**2\n\n    if method == \"ordinary_difference_in_differences\":\n        # scale = 1\n        scale_var = 0\n\n    elif method == \"percent_difference_in_differences\":\n        # scale = mTr/mCGr\n        # neglecting covariance between MTr and MCGr\n        cov_term = 0\n        # cov = mTr_unc*mCGr_unc*corr_mT_mCG\n        # cov_term = 2*cov/(mTr*mCGr)\n        \n        scale_var = scale**2*(mTr_var/mTr**2 + mCGr_var/mCGr**2 - cov_term)\n\n    elif method == \"absolute_percent_difference_in_differences\":\n        # scale = np.abs(mTr/mCGr)\n        # can take uncertainty of interior, then (partial of abs(x))^2 = 1\n        # neglecting covariance between MTr and MCGr\n        cov_term = 0\n        # cov = mTr_unc*mCGr_unc*corr_mT_mCG\n        # cov_term = 2*cov/(mTr*mCGr)\n\n        scale_var = scale**2*(mTr_var/mTr**2 + mCGr_var/mCGr**2 - cov_term)\n\n    if np.all(oCGr_unc == 0):\n        CG_diff_var = mCGr_unc**2\n    else: # if observed has uncertainty, it and it's covariance with model should be considered\n        cov = mCGr_unc*oCGr_unc*CGr_corr\n        CG_diff_var = mCGr_var + oCGr_unc**2 - 2*cov\n\n    # neglect covariance between scale and CG_diff\n    correction_var = correction**2*(CG_diff_var/CG_diff**2 + scale_var/scale**2)\n    correction_unc = np.sqrt(correction_var)\n\n    return correction_unc  \n\n\ndef _unit_correction(\n    oTr, \n    mTr,\n    oCGr, \n    mCGr,\n    oTr_unc,\n    mTr_unc,\n    oCGr_unc,\n    mCGr_unc,\n    CGr_corr, # only needed if oCGr_unc != 0\n    calculate_unc,\n    method=None\n):\n    \"\"\"Calculates corrections for each comparison group meter of a single treatment meter for a single hour\n       for a single cluster\n    \n    Args:\n        oTr: treatment meter observed from reporting period\n        mTr: treatment meter model from reporting period\n        oCGr: comparison group observed from reporting period\n        mCGr: comparison group model from reporting period\n        oTr_unc: treatment meter observed uncertainty from reporting period\n        mTr_unc: treatment meter model uncertainty from reporting period\n        oCGr_unc: comparison group observed uncertainty from reporting period\n        mCGr_unc: comparison group model uncertainty from reporting period\n        CGr_corr: correlation between oCGr and mCGr over entire reporting period for each meter\n    \"\"\"\n    # The generalized function: m_cT = m_T - s_CG∙(m_CG - o_CG)\n    # Correction = s_CG∙(m_CG - o_CG)\n\n    if method is None:\n        # scale = 0\n        # scale_unc = 0\n        correction = np.zeros_like(mTr)\n        correction_unc = np.zeros_like(mTr)\n        return correction, correction_unc\n    \n    if method == \"ordinary_difference_in_differences\":\n        scale = 1\n\n    elif method == \"percent_difference_in_differences\":\n        # equivalent to simplified savings = mT*oCG/mCG - oT \n        scale = mTr/mCGr\n\n    elif method == \"absolute_percent_difference_in_differences\":\n        # simplified savings = mT(1 - np.sign(mT)*np.sign(mCG) + oCG/mCG) - oT\n        scale = np.abs(mTr/mCGr)\n\n    CG_diff = mCGr - oCGr\n\n    # correction\n    correction = scale*CG_diff\n\n    if calculate_unc:\n        correction_unc = _unit_correction_unc(\n            oTr, \n            mTr,\n            oCGr, \n            mCGr,\n            scale,\n            CG_diff,\n            correction,\n            oTr_unc,\n            mTr_unc,\n            oCGr_unc,\n            mCGr_unc,\n            CGr_corr, # only needed if oCGr_unc != 0\n            method=method\n        )\n    else:\n        correction_unc = np.full_like(correction, np.nan)\n\n    return correction, correction_unc\n\n\ndef _update_mask(global_mask, mask=None, idx_valid=None, idx_invalid=None):\n    if sum(arg is not None for arg in [mask, idx_valid, idx_invalid]) > 1:\n        raise ValueError(\"Only one of `mask`, `idx_valid`, or `idx_invalid` can be provided.\")\n    \n    if mask is not None:\n        pass\n    \n    elif idx_valid is not None:\n        mask = np.full_like(global_mask, False, dtype=bool)\n        mask[idx_valid] = True\n\n    elif idx_invalid is not None:\n        mask = np.full_like(global_mask, True, dtype=bool)\n        mask[idx_invalid] = False\n\n    return global_mask & mask\n\n\ndef _apply_mask(mask, *arrays):\n    res = []\n    for arr in arrays:\n        arr_updated = None\n        if arr is not None:\n            arr_updated = arr[mask]\n\n            if len(arr_updated) < 3:\n                raise ValueError(\"After applying mask, array has insufficient length.\")\n\n        res.append(arr_updated)\n\n    if len(res) == 1:\n        return res[0]\n    \n    return tuple(res)\n\n\ndef _effective_sample_size(weight):\n    # Kish's effective sample size, weights normalized https://doi.org/10.1002/bimj.19680100122\n    n = 1 / np.sum(np.power(weight, 2))\n\n    return n\n\n\ndef _cluster_correction(\n    oTr: float, \n    mTr: float,\n    oCGr: np.ndarray, \n    mCGr: np.ndarray,\n    oTr_unc: Optional[float],\n    mTr_unc: Optional[float],\n    oCGr_unc: Optional[np.ndarray],\n    mCGr_unc: Optional[np.ndarray],\n    CGr_corr: Optional[np.ndarray], # only needed if oCGr_unc != 0\n    calculate_unc: bool,\n    settings: _settings.CGCorrectionSettings,\n):\n    # Operates on a single cluster's data for a single hour\n    \n    mask = np.full_like(mCGr, True, dtype=bool)\n\n    # get correction and correction uncertainty\n    correct, correct_unc = _unit_correction(\n        oTr, \n        mTr,\n        oCGr, \n        mCGr,\n        oTr_unc,\n        mTr_unc,\n        oCGr_unc,\n        mCGr_unc,\n        CGr_corr, # only needed if oCGr_unc != 0\n        calculate_unc,\n        method=settings.algorithm\n    )\n\n    # set initial weights\n    if settings.weight_cluster_aggregation is None:\n        cluster_weight = None\n    elif settings.weight_cluster_aggregation == _settings.WeightClusterAggChoice.MODEL:\n        cluster_weight = np.abs(mCGr) / np.sum(np.abs(mCGr))\n\n    # remove outliers\n    if settings.outlier_rejection.enabled:\n        # remove outliers\n        _, idx_no_outliers = remove_outliers(\n            correct, # if normalized (correct / mTr), small denominator issue introduced\n            weights=cluster_weight, \n            sigma_threshold=settings.outlier_rejection.std_threshold, \n            quantile=settings.outlier_rejection.quantile, \n            transform=settings.outlier_rejection.transform\n        )\n\n        # update global mask and cluster mask\n        mask = _update_mask(mask, idx_valid=idx_no_outliers)\n\n        # remove outliers from data\n        correct, correct_unc = _apply_mask(mask, correct, correct_unc)\n        mCGr = _apply_mask(mask, mCGr)\n\n        # renormalize weights\n        if cluster_weight is not None:\n            cluster_weight = np.abs(mCGr) / np.sum(np.abs(mCGr))\n\n    # apply caps\n    # decision: should capped values have their uncertainty considered or excluded?\n    if settings.correction_cap.enabled:\n        cap = np.abs(mTr)*settings.correction_cap.value\n        if settings.correction_cap.type == _settings.CorrectionCapChoice.GLOBAL:\n            correct = np.clip(correct, -cap, cap)\n            \n        elif settings.correction_cap.type == _settings.CorrectionCapChoice.SOLAR:\n            solar_threshold = settings.correction_cap.solar_threshold\n            solar_mask = np.abs(mCGr) < solar_threshold\n            \n            correct[solar_mask] = np.clip(correct[solar_mask], -cap, cap)\n\n    # compute mean and unc\n    cluster_mean = np.average(correct, weights=cluster_weight)\n\n    # check n to see if unc can be calculated\n    if calculate_unc:\n        if cluster_weight is None:\n            n = len(correct)\n        else:\n            n = _effective_sample_size(cluster_weight)\n\n        if n < 2:\n            calculate_unc = False\n\n    # uncertainty calculation\n    cluster_unc = np.nan\n    if calculate_unc:\n        # aggregation uncertainty\n        correct_std = fast_std(\n            correct,\n            mean = cluster_mean,\n            weights = cluster_weight\n        )\n        # uncertain if this should be a confidence interval or prediction interval, CI for now\n        _unc_factor = unc_factor(n, interval=\"CI\", alpha=settings.alpha)\n        correct_agg_unc = correct_std * _unc_factor\n\n        # model uncertainty\n        model_var = np.average(correct_unc**2, weights=cluster_weight)\n\n        cluster_unc = np.sqrt(correct_agg_unc**2 + model_var)\n\n    return cluster_mean, cluster_unc, mask\n\n\ndef model_correction(\n    oTr: float,         # observed treatment meter value during reporting period\n    mTr: float,         # model treatment meter value during reporting period\n    oCGr: np.ndarray, \n    mCGr: np.ndarray,\n    oTr_unc: Optional[float],\n    mTr_unc: Optional[float],\n    oCGr_unc: Optional[np.ndarray],\n    mCGr_unc: Optional[np.ndarray],\n    CGr_corr: Optional[np.ndarray], # only needed if oCGr_unc != 0\n    CG_label: np.ndarray,\n    T_weight: np.ndarray,\n    settings: _settings.CGCorrectionSettings,\n):\n    # if no did, return\n    if settings.algorithm is None:\n        # scale = 0\n        # scale_unc = 0\n        mTrc = float(mTr)\n        mTrc_unc = float(mTr_unc) if mTr_unc is not None else np.nan\n        mask = np.full_like(mTr, False, dtype=bool)\n\n        return mTrc, mTrc_unc, mask\n    \n    # input validation\n    if mTr is None or not np.isfinite(mTr):\n        raise ValueError(\"`mTr` must be a finite number\")\n\n    if len(oCGr) < 5:\n        raise ValueError(\"`oCGr` cannot have a length less than 5\")\n    \n    if not (len(oCGr) == len(mCGr) == len(CG_label)):\n        raise ValueError(\"`oCGr`, `mCGr`, and `CG_label` must have the same length\")\n    \n    if len(T_weight) != np.sum(np.unique(CG_label) >= 0):\n        raise ValueError(\"`T_weight` must have the same number of elements as the unique number of labels in `CG_label`\")\n\n    if oCGr_unc is None:\n        oCGr_unc = np.zeros_like(oCGr)\n\n    if not (len(oCGr) == len(oCGr_unc)):\n        raise ValueError(\"`oCGr` and `oCGr_unc` must have the same length\")\n\n    if mCGr_unc is not None:\n        if not (len(mCGr) == len(mCGr_unc)):\n            raise ValueError(\"`mCGr` and `mCGr_unc` must have the same length\")\n\n    if CGr_corr is None:\n        CGr_corr = np.zeros_like(oCGr)\n\n    if not (len(oCGr_unc) == len(CGr_corr)):\n        raise ValueError(\"`oCGr_unc` and `CGr_corr` must have the same length\")\n    \n    # check length of CG inputs and set global_mask to exclude non-finite values\n    global_mask = np.isfinite(oCGr) & np.isfinite(mCGr) & np.isfinite(CG_label)\n    global_mask = global_mask & (oCGr is not None) & (mCGr is not None)\n    global_mask = global_mask & (CG_label is not None)\n\n    calculate_unc = False\n    if mTr_unc is not None and mCGr_unc is not None:\n        calculate_unc = True\n        global_mask = global_mask & np.isfinite(mCGr_unc) & (mCGr_unc is not None)\n\n    if calculate_unc and oCGr_unc is not None and CGr_corr is not None:\n        global_mask = global_mask & np.isfinite(oCGr_unc) & (oCGr_unc is not None)\n        global_mask = global_mask & np.isfinite(CGr_corr) & (CGr_corr is not None)\n\n    unique_labels = np.unique(CG_label)\n    unique_labels = unique_labels[np.isfinite(unique_labels)]\n    unique_labels = unique_labels[unique_labels >= 0] # exclude outlier label(s)\n\n    cluster_correct = np.empty(unique_labels.shape)\n    cluster_correct_unc = np.empty(unique_labels.shape)\n    for label in unique_labels:\n        # get label mask\n        label_mask = CG_label == label\n        mask = global_mask & label_mask\n\n        if T_weight[label] == 0:\n            _correct = np.nan\n            _correct_unc = np.nan\n\n            # update global mask\n            global_mask[label_mask] = False\n        \n        else:\n            _correct, _correct_unc, _mask = _cluster_correction(\n                oTr, \n                mTr,\n                _apply_mask(mask, oCGr), \n                _apply_mask(mask, mCGr),\n                oTr_unc,\n                mTr_unc,\n                _apply_mask(mask, oCGr_unc),\n                _apply_mask(mask, mCGr_unc),\n                _apply_mask(mask, CGr_corr), # only needed if oCGr_unc != 0\n                calculate_unc,\n                settings,\n            )\n\n            if not np.isfinite(_correct_unc):\n                calculate_unc = False\n\n            # update global mask\n            global_mask[mask] = _update_mask(global_mask[mask], mask=_mask)\n\n        cluster_correct[label] = _correct\n        cluster_correct_unc[label] = _correct_unc\n\n    # combine clusters with weights to get corrected model\n    idx_valid = (T_weight > 0).flatten()\n    correction = np.average(cluster_correct[idx_valid], weights=T_weight[idx_valid])\n    mTrc = float(mTr - correction)\n\n    mTrc_unc = np.nan\n    if calculate_unc:\n        correction_var = np.sum((T_weight[idx_valid]**2)*(cluster_correct_unc[idx_valid]**2))\n        mTrc_unc = float(np.sqrt(mTr_unc**2 + correction_var))\n\n    return mTrc, mTrc_unc, global_mask"
  },
  {
    "path": "opendsm/comparison_groups/savings/scratch.ipynb",
    "content": "{\n \"cells\": [\n  {\n   \"cell_type\": \"markdown\",\n   \"metadata\": {},\n   \"source\": [\n    \"class\\n\",\n    \"\\n\",\n    \"inputs:\\n\",\n    \"- Treatment data (8760) (data + model uncertainty)\\n\",\n    \"- Comparison group data (8760) (data + model uncertainty)\\n\",\n    \"- df_cg\\n\",\n    \"- df_t_coeffs\\n\",\n    \"- Settings\\n\",\n    \"\\n\",\n    \"Settings\\n\",\n    \"- outlier_removal bool\\n\",\n    \"- ratio_weight\\n\",\n    \"- uncertainty confidence level\\n\",\n    \"- outlier rejection level - outlier_std\\n\",\n    \"- solar_cap\\n\",\n    \"- type of correction (None, \\\"Ordinary DiD\\\", \\\"Pct DiD\\\", \\\"Abs Pct DiD\\\")\\n\",\n    \"\\n\",\n    \"correction func = (m_T - o_T) - scale_fcn(m_T, m_CG, o_T, o_CG)*(m_CG - o_CG)\\n\",\n    \"\\n\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": null,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"# do we need to be able to generate the SAME EXACT numbers (me and Caleb don't want to)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": null,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"# when do we aggregate? Does CG come in aggregated or do we do it here?\\n\",\n    \"def unit_correction(oT, mT, oCG, mCG, settings):\\n\",\n    \"\\n\",\n    \"    if settings.method is None:\\n\",\n    \"        scale = 0\\n\",\n    \"    \\n\",\n    \"    elif settings.method == \\\"ordinary_difference_in_differences\\\":\\n\",\n    \"        scale = 1\\n\",\n    \"\\n\",\n    \"    elif settings.method == \\\"percent_difference_in_differences\\\":\\n\",\n    \"        # simplified\\n\",\n    \"        # savings = mT*oCG/mCG - oT \\n\",\n    \"\\n\",\n    \"        scale = mT/mCG\\n\",\n    \"\\n\",\n    \"    elif settings.method == \\\"absolute_percent_difference_in_differences\\\":\\n\",\n    \"        # simplified\\n\",\n    \"        # savings = mT(1 - np.sign(mT)*np.sign(mCG) + oCG/mCG) - oT\\n\",\n    \"\\n\",\n    \"        scale = np.abs(mT/mCG)\\n\",\n    \"\\n\",\n    \"    correction = scale*(mCG - oCG)\\n\",\n    \"\\n\",\n    \"    # outlier rejection\\n\",\n    \"\\n\",\n    \"    if settings.agg == \\\"mean\\\":\\n\",\n    \"        correction_agg = np.mean(correction)\\n\",\n    \"\\n\",\n    \"    elif settings.agg == \\\"median\\\":\\n\",\n    \"        correction_agg = np.median(correction)\\n\",\n    \"\\n\",\n    \"    # [avg_cg_o1, .9,\\n\",\n    \"    #  avg_cg_o2, .01]\\n\",\n    \"    #    cg_o2, .01]\\n\",\n    \"\\n\",\n    \"    # uncertainty\\n\",\n    \"\\n\",\n    \"    return correction\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"def unit_cluster(oCG, oT, settings):\\n\",\n    \"    if settings.agg == \\\"mean\\\":\\n\",\n    \"        agg_fcn = np.mean\\n\",\n    \"    else:\\n\",\n    \"        agg_fcn = np.median\\n\",\n    \"\\n\",\n    \"    \\n\",\n    \"\\n\",\n    \"\\n\",\n    \"# if we can not repeat calculations with treatment meters then skip\\n\",\n    \"def _corrected_model(treatment_data, comparison_data, df_cg, df_t_coeffs, Settings):\\n\",\n    \"    # treatment_data = [id, observed, model] # for a single unit of time\\n\",\n    \"    # comparison_data = [id, observed, model] # for a single unit of time\\n\",\n    \"\\n\",\n    \"    return \\\"1 unit corrected model\\\"\\n\",\n    \"\\n\",\n    \"def _corrected_model_dec(args):\\n\",\n    \"    return _corrected_model(*args)\\n\",\n    \"\\n\",\n    \"def _comparison_group_data(comparison_data, df_cg):\\n\",\n    \"    #aggregate comparison group data based on df_cg\\n\",\n    \"    return \\\"full time period cg\\\"\\n\",\n    \"\\n\",\n    \"def mp_fcn():\\n\",\n    \"    if mp:\\n\",\n    \"        pass\\n\",\n    \"    else:\\n\",\n    \"        pass\\n\",\n    \"\\n\",\n    \"    return\\n\",\n    \"\\n\",\n    \"class Model_Corrected:\\n\",\n    \"    def __init__(self, settings):\\n\",\n    \"        self.settings = settings\\n\",\n    \"\\n\",\n    \"    def \\n\",\n    \"\\n\",\n    \"class Savings:\\n\",\n    \"    def __init__(self, settings):\\n\",\n    \"        self.settings = settings\\n\",\n    \"\\n\",\n    \"    def base_savings(self):\\n\",\n    \"        return _corrected_model()\\n\",\n    \"\\n\",\n    \"    def agg_savings(self):\\n\",\n    \"        pass\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": null,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"# transform:\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"mu, sigma = robust_mu_sigma(x, robust_type, c=1.5, tol=1e-08)\\n\",\n    \"x_std = (x - mu)/sigma\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"x_std = bisymlog_transform(x, rescale_quantile=0.10)\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"\\n\",\n    \"x_std = robust_YJ_transform(x, Q_perc=0.25, c=c, outlier_alpha=outlier_alpha, robust_type=robust_type)\\n\",\n    \"\\n\",\n    \"x_std = scipy_YJ_transform(x, robust_type=robust_type)\\n\",\n    \"\\n\",\n    \"x_outliers = IQR_outlier(x_std, weights=weight, sigma_threshold=3, quantile=0.25)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 127,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"image/png\": \"iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/SrBM8AAAACXBIWXMAAA9hAAAPYQGoP6dpAABhDklEQVR4nO3dd3QU9cLG8e9ueiAJBEgBQi+hJ/RQBAVBaUZRKSpFBESqWPG1X+/FXiiKHRtdigKiFOmhJxCq9J5Qk5Bedt4/VoMoJYFsdjd5PufMOdnZmZ1nDLn73PlNMRmGYSAiIiLiJMz2DiAiIiKSHyovIiIi4lRUXkRERMSpqLyIiIiIU1F5EREREaei8iIiIiJOReVFREREnIrKi4iIiDgVV3sHKGgWi4VTp07h4+ODyWSydxwRERHJA8MwuHTpEuXLl8dsvv6xlSJXXk6dOkVISIi9Y4iIiMhNOH78OBUrVrzuMkWuvPj4+ADWnff19bVzGhEREcmLpKQkQkJCcr/Hr6fIlZe/hop8fX1VXkRERJxMXk750Am7IiIi4lRUXkRERMSpqLyIiIiIU1F5EREREaei8iIiIiJOReVFREREnIrKi4iIiDgVlRcRERFxKiovIiIi4lRsWl4++eQTGjZsmHu324iICH755ZfrrjN79mxCQ0Px9PSkQYMGLF682JYRRURExMnYtLxUrFiRN998k61bt7JlyxbuuOMO7rnnHnbt2nXV5devX0+fPn0YNGgQ0dHRREZGEhkZyc6dO20ZU0RERJyIyTAMozA36O/vzzvvvMOgQYP+9V6vXr1ISUlh4cKFufNatmxJWFgYU6ZMydPnJyUl4efnR2Jiop5tJCIi4iTy8/1daOe85OTkMGPGDFJSUoiIiLjqMlFRUXTs2PGKeZ07dyYqKuqan5uRkUFSUtIVky0YhsGTM2OYs/WETT5fRERE8sbm5SU2NpaSJUvi4eHB448/zrx586hbt+5Vl42LiyMwMPCKeYGBgcTFxV3z88ePH4+fn1/uFBISUqD5//LLzjjmRZ/k6dnbeWrWdlIzs22yHREREbk+m5eX2rVrExMTw8aNGxk2bBj9+/dn9+7dBfb548aNIzExMXc6fvx4gX3233WuF8TYO2thNsGP207QfeJa9sVdssm2RERE5NpsXl7c3d2pUaMGTZo0Yfz48TRq1IiPPvroqssGBQURHx9/xbz4+HiCgoKu+fkeHh65VzP9NdmCi9nEqA41mTa4JYG+Hhw8m0KPSWuZsekYhXzakIiISLFW6Pd5sVgsZGRkXPW9iIgIli9ffsW8pUuXXvMcGXtoWa0Mi0e15bZa5cjItvD83FjGzIwhOUPDSCIiIoXBpuVl3LhxrF69miNHjhAbG8u4ceNYuXIlDz30EAD9+vVj3LhxucuPHj2aJUuW8N5777F3715effVVtmzZwogRI2wZM9/KlPRg6oBmPHtXbVzMJhbEnKL7xLXsOpVo72giIiJFnk3Ly5kzZ+jXrx+1a9emQ4cObN68mV9//ZU777wTgGPHjnH69Onc5Vu1asW0adP47LPPaNSoEXPmzGH+/PnUr1/fljFvitls4on2NZg5pCXBfp4cPpfCvR+v57sNRzWMJCIiYkOFfp8XW7PZfV5ysuHz9lCpFYR2hcqtwcUVgIspmTw9ezvL954BoGuDYMb3bICvp1vBbV9ERKQIy8/3t8pLXh1eDd90v/zaqzTUugtCu0H1OzDcvPhizWHeWrKXbItBJX9vJvUNp2HFUgWXQUREpIhSebFFeclKg0MrYc9C2LcY0i5cfs/VC2p0gNCubPduyRNzj3AyIQ03FxMvdKnDgFZVMJlMBZdFRESkiFF5sfXjAXKy4fhG2LvQOiUcu/yeyYXskAhmp4Qx8WQtTlGWzvUCebtnI/y8NYwkIiJyNSovhflsI8OA+J3WIzJ7F0F87BVv7zSqsiS7KdEl2vL0wz0Ir1Ta9plEREScjMqLPR/MeOGwdVhpz0I4vgEMS+5bB4wKJFbtSuO7B2AKqAsaShIREQFUXhznqdLJZ+GPJWTt+gkO/Y6bkZX7Vo5/TVzqR0LdSAispyIjIiLFmsqLo5SXvzHSElj3yw+kx/xIW9MOPEyXiwxlakDde6xFJqiBioyIiBQ7Ki8OWF7+sutUIs/+sI7qF9fSzXUjd7jswNXIvLyAfzVrial7DwQ3UpEREZFiQeXFgcsLQHJGNv83L5YFMacoQRojKhzgUf/teBxeAdnplxcsXRXq3wcNHoCAOvYLLCIiYmMqLw5eXgAMw2Dm5uO88tMuMrItBPp6MLFnLZpnbYZd82H/UshOu7xCQD1ocD/U7wmlK9stt4iIiC2ovDhBefnL3rgkhv+wjYNnUzCbYEzHWgy/vQYuWSnwxxLY+aO1yFj+do5MSAuofz/UuxdKlrNfeBERkQKi8uJE5QUgNTObl+bv4sdtJwBoXaMMH/QKI8DH888FLsCen2HnHDi8BvjzV2ZygWrtrMNKod3A0zn2V0RE5J9UXpysvPxlztYTvDR/J2lZOZQt6cFHvcNoXaPslQslnYZd8yB2Npzadnm+iwfU6gQNHoRancHVo3DDi4iI3AKVFyctLwD74y8xYlo0++IvYTLByDtqMrpDTVzMV7nq6PxB67BS7Gw498fl+Z6lrOfGNOoDFZvqiiUREXF4Ki9OXF4A0jJzeO3nXczYfByAFlX9mdAnnEBfz6uvYBgQF2stMbFz4NKpy++VqQGNekPDXlCqUiGkFxERyT+VFycvL39ZEHOSF+bGkpKZQ5kS7rzfK4x2tW5wgq4lBw6vhu3TrefJZKVefq9KW2uRqXsPePjYNryIiEg+qLwUkfICcOhsMsOnRbPndBIAw9pX56k7a+HqYr7xyhmXrAVm+/QrT/R19YI63a1Fplp7MLvYLL+IiEheqLwUofICkJ6VwxuLdvP9hmMANK1cmgl9wilfyivvH5JwHHbMhO0z4Pz+y/N9giGsL4Q/bL27r4iIiB2ovBSx8vKXhTtO8fyPsSRnZFPK2433H2zEHaGB+fsQw4CT26xHY3bOgbSLl9+r0hYa97celXG7xvk1IiIiNqDyUkTLC8DR8ymMmBZN7MlEAAa3rcqzd4XilpdhpH/KzoB9v8C2b+HgCnKHlTz9rCf4Nu5nfVCkiIiIjam8FOHyApCRncP4xXuZuv4IAGEhpZjYJ5wQf++b/9CE4xDzA0R/D4nHL88PDrOWmAb3W0uNiIiIDai8FPHy8pclO+N4ds52ktKz8fV05Z0HGtG5XtCtfaglBw6ttB6N2bvo8mMJXL2gXqS1yFSK0L1jRESkQKm8FJPyAnD8Qiojpkez/XgCAANbV+H5u0PxcC2AK4hSzsOOGbDtOzi75/L8cnWg2SDr0JIeSSAiIgVA5aUYlReAzGwL7/y6l8/XHAagQQU/JvdtTKUytzCM9HeGASe2wLZvrHf0/eveMW4loOED0HQQBDcsmG2JiEixpPJSzMrLX5btjufpOdtJSM3Cx8OVt+5vSJcGwQW7kfRE6+XWm7+Ec/suz6/Q1Ho0pt694JaPS7hFRERQeSm25QXgVEIaI6dHs/Wo9RLoR1pW5v+61sHTrYBvRGcYcHQdbPkKdv90+dwYr9IQ9hA0fRTKVC/YbYqISJGl8lKMywtAVo6F95f+wScrDwJQN9iXyQ81pmrZErbZYPIZiP4OtkyFxGOX51e/A1oMgxodwXwTl3KLiEixofJSzMvLX1buO8PYWdu5kJJJCXcX/ndfA+4Jq2C7DVpy4MAy65DS/t/IvW+Mf3VoMdR6J189U0lERK5C5UXlJVdcYjqjZkSz6fAFAPo0D+GV7vUKfhjpny4egU2fW69UyrDeUA93H+tjCFoM0aMIRETkCiovKi9XyM6x8NHy/Uz6/QCGAbUDfZj8UDg1AgrhKEhGsvVRBBs//dszlUxQ6y5o+ThUbad7xoiIiMqLysvVrd1/jjEzYziXnIGXmwtvRNanZ5OKhbNxiwUOrYANU+DA0svzy9WxDik16q2rlEREirH8fH/b9CzK8ePH06xZM3x8fAgICCAyMpJ9+/Zdd52pU6diMpmumDw99ZDAgtCmZlkWj25Dq+plSMvK4anZ23l69nZSM7Ntv3Gz2Xri7sNzYMRWaD4E3Etab363cAx8UB9WvQ2pF2yfRUREnJpNy8uqVasYPnw4GzZsYOnSpWRlZdGpUydSUlKuu56vry+nT5/OnY4ePWrLmMVKgI8n3w1qwdg7a2E2wZytJ+gxaR374i4VXoiyNaDLOzB2N3T+H/hVgtRz8Pt/4f26sOhpuHC48PKIiIhTKdRho7NnzxIQEMCqVau47bbbrrrM1KlTGTNmDAkJCTe1DQ0b5V3UwfOMnhHNmUsZeLqZea1HPR5sGoKpsM9BycmG3fNh/QQ4vd06z2SGOj2g9Sio0KRw84iISKFzmGGjf0pMtF514u/vf93lkpOTqVy5MiEhIdxzzz3s2rXrmstmZGSQlJR0xSR5E1G9DItHt6VtzbKkZ1l47sdYnpwZQ3JGIQwj/Z2Lq/Wp1UNWQb+frMNLhsVaaD6/A77uAvuWWM+bERGRYq/QjrxYLBZ69OhBQkICa9euveZyUVFR7N+/n4YNG5KYmMi7777L6tWr2bVrFxUr/vvk0ldffZXXXnvtX/N15CXvLBaDKasP8t5vf5BjMahWtgST+jambnk7/veL3wXrJ0LsbLD8WabKhUKbsVC/p7XwiIhIkeGQVxsNGzaMX375hbVr1161hFxLVlYWderUoU+fPvznP//51/sZGRlkZGTkvk5KSiIkJETl5SZsPnKBUdOjOZ2YjrurmZe71eWhFpUKfxjp7xJPwsZPrHfvzfzzvJzSVaDNk9CoD7h62C+biIgUGIcrLyNGjGDBggWsXr2aqlWr5nv9Bx54AFdXV6ZPn37DZXXOy625kJLJ07O3s2LvGQC6Ngzmzfsa4OPpZt9g6Ymw+QuImgyp563zfMpbz4lp3B/cC+gJ2iIiYhcOc86LYRiMGDGCefPmsWLFipsqLjk5OcTGxhIcXMBPR5ar8i/hzhf9mvJCl1BczSYW7ThNt4lriT2RaN9gnn7Q9ikYEwudx4NPMFw6BUuehw8bwJr3IV3nO4mIFAc2PfLyxBNPMG3aNBYsWEDt2rVz5/v5+eHlZb0hWb9+/ahQoQLjx48H4PXXX6dly5bUqFGDhIQE3nnnHebPn8/WrVupW7fuDbepIy8FZ9uxi4ycFs3JhDTcXcy80CWU/q2q2HcY6S/ZGRAzDdZ+AAl/Xkrv6QfNh0LLYeB9/ZPCRUTEsTjMkZdPPvmExMRE2rdvT3BwcO40c+bM3GWOHTvG6dOnc19fvHiRwYMHU6dOHbp06UJSUhLr16/PU3GRgtW4UmkWj2pLp7qBZOZYePXn3Tz+/VYSU7PsHc16rkvTgTByG9z7KZStZR1aWv02fNgQfv8fpCXYO6WIiNiAHg8gN2QYBlPXH+F/i/eQlWNQsbQXk/o2JiyklL2jXWaxwJ6fYPU7EL/TOs/TDyJGWh8/4Kl/CyIijszhTtgtTCovtrPjRAIjpkVz7EIqrmYTz98dyqA2VR1jGOkvFgvs/Rl+H2999ACAV2loNcr6SAKPkvbNJyIiV6XyovJiM0npWTz/4w4Wx8YB0LFOAO8+0IhS3u52TvYPFgvsmgsr37z8NGvvstBmDDQdpKuTREQcjMqLyotNGYbB9xuP8Z+Fu8nMtlDez5OJfcNpUtkBT5K15EDsHFj1Jlw4ZJ1XIgBuewaaDABXBytdIiLFlMqLykuh2HUqkRHTojl8LgUXs4mnO9Vm6G3VMJsdaBjpLznZsGMGrHoLEo5Z55WuAne8BPXusz71WkRE7EblReWl0CRnZPPC3Fh+2n4KgPa1y/HeA40oU9JB73ybnQnR31lLTHK8dV5QQ+j4KlS/Axzp/B0RkWJE5UXlpVAZhsGMzcd59addZGRbCPT1YELvcFpUK2PvaNeWmQIbPoa1H11+7EDV26wlRk+xFhEpdCovKi92sTcuieE/bOPg2RTMJniyYy2euL0GLo44jPSXlPOw5j3Y/DnkZFrn1Y2EDi9Dmep2jSYiUpyovKi82E1KRjYvLdjJ3G0nAWhToywf9AqjnI+DDiP95eJRWDkets8ADDC7QrPB0O5Z3a1XRKQQqLyovNjd7C3HeXnBLtKycihb0oMJvcNoVaOsvWPdWNxOWP4a7P/N+tqzFLQfB80GgYudH04pIlKEqbyovDiE/fGXGD5tG3/EJ2Mywcg7ajK6Q03HHkb6y8EV8Ov/wZnd1tdlakLn/0LNTjqpV0TEBlReVF4cRlpmDq/+tIuZW44D0LKaPx/1DifQ19POyfIgJxuiv4UV/4XUc9Z51e+ATv+FQD1rS0SkIKm8qLw4nPnRJ3lhXiypmTmUKeHOB73CuK1WOXvHypv0ROtJvRs+sZ7UazJbb3B3+4tQwoGvqBIRcSIqLyovDunQ2WSGT4tmz+kkAJ5oX52xd9bC1cVJbhB34TAsfdn6AEiwPjOpw8vQuD+YXeybTUTEyam8qLw4rPSsHP6zcDc/bLTe5bZZldJM6BNOsJ+XnZPlw5F18Muzl59eXT4curwHFXV/GBGRm6XyovLi8BbuOMXzP8aSnJFNaW833n8wjNtDA+wdK+9ysmHLl7DiDchIAkzQuB90eEVDSSIiN0HlReXFKRw5l8KI6dvYedI6jDT0tmo83bk2bs4yjASQfAaWvgLbp1lfayhJROSmqLyovDiNjOwcxi/ey9T1RwAIr1SKiX3CqVja277B8utoFCx++sqhpO4TILihfXOJiDgJlReVF6ezZOdpnpmzg0vp2fh5ufHO/Q3pVC/I3rHy559DSSYXaDUC2j0P7k5WxkRECpnKi8qLUzp+IZUR06PZfjwBgIGtqzDu7jq4uzrRMBLApXhY8hzsmmd9XboKdPvAeo8YERG5qvx8fzvZt4IUZSH+3sweGsFjbaoC8PW6I9w/ZT3HzqfaOVk++QTCA1OhzwzwrQAXj8B398LcodYHQYqIyC1ReRGH4u5q5sVudfmiX1P8vNzYcSKRrhPWsDj2tL2j5V/tu2H4RmjxOGCCHTNgUlPrwx+L1gFPEZFCpWEjcVgnE9IYNT2arUcvAvBIy8r8X9c6eLo54VU8J7bCz6Mun9BbvQP0mAh+FeybS0TEQWjYSIqECqW8mDGkJY+3qw7AdxuO0vOT9Rw+l2LnZDehYhMYshI6vgqunnBwOXwcAdHf6yiMiEg+6ciLOIXf953hqVnbuZCSSQl3F8b3bEiPRuXtHevmnNsP84fBic3W1zU7WS+r9g22by4RETvSkRcpcm6vHcDiUW1pXsWflMwcRk2PZtzcWNKzcuwdLf/K1oRHf4WOr4GLO+z/DT5uoXNhRETySOVFnEaQnyfTBrdg5B01MJlg+qZjRE5ex4EzyfaOln9mF2gzBoausd7QLj0R5g2FGX2tl1qLiMg1qbyIU3F1MfNUp9p892gLypZ0Z2/cJXpMWsvcbSfsHe3mBITCoGVwx0tgdoN9i+HjlrB3kb2TiYg4LJUXcUptapZl8ai2tKpehtTMHMbO2s4zs7eTmplt72j55+IKtz0NQ1dBUANIu2A9AvPzGMh0wpOTRURsTOVFnFaAryffDWrBkx1rYTbB7K0nuGfSOv6Iv2TvaDcnsB48thxajbS+3vo1fNoOTsXYNZaIiKNReRGn5mI2MbpjTX54rCUBPh7sP5NMj0lrmbX5OE55IZ2rB3R6Ax6ZDz7BcH4/fNER1n0EFou904mIOASVFykSIqqXYfHotrStWZb0LAvP/riDJ2fGkJLhhMNIANVvh2HrIbQbWLJg6cvw3T2QdMreyURE7M6m5WX8+PE0a9YMHx8fAgICiIyMZN++fTdcb/bs2YSGhuLp6UmDBg1YvHixLWNKEVG2pAffDGzOM51r42I2MT/mFN0nrmX3qSR7R7s53v7Q63vrPWDcvOHwapjSBvYvs3cyERG7sml5WbVqFcOHD2fDhg0sXbqUrKwsOnXqRErKtU9CXL9+PX369GHQoEFER0cTGRlJZGQkO3futGVUKSLMZhPDb6/BjCEtCfL15NC5FCI/XscPG4865zCSyQRN+lsvqQ5qCKnn4YeesPx1yHHSo0oiIreoUO+we/bsWQICAli1ahW33XbbVZfp1asXKSkpLFy4MHdey5YtCQsLY8qUKTfchu6wK3+5kJLJU7Ni+H3fWQC6NQxm/H0N8PF0s3Oym5SVDr++AFu+tL6u3AZ6fqE784pIkeCwd9hNTEwEwN/f/5rLREVF0bFjxyvmde7cmaioqKsun5GRQVJS0hWTCIB/CXe+7N+MF7qE4mo2sXDHabpNXMvOk4n2jnZz3Dyh2/vQ80twLwlH11qHkQ6usHcyEZFCVWjlxWKxMGbMGFq3bk39+vWvuVxcXByBgYFXzAsMDCQuLu6qy48fPx4/P7/cKSQkpEBzi3Mzm00Mua06sx6PoEIpL46eT+W+j9fzzfojzjmMBNDgfhiyCgLrQ+o5+O4++P1/uhpJRIqNQisvw4cPZ+fOncyYMaNAP3fcuHEkJibmTsePHy/Qz5eioXGl0iwa1YY76waSmWPhlZ92Mez7bSSmZdk72s0pWwMeWwZNBgAGrHoLpveGtAQ7BxMRsb1CKS8jRoxg4cKF/P7771SsWPG6ywYFBREff+WzXeLj4wkKCrrq8h4eHvj6+l4xiVxNKW93PnukCS93q4ubi4klu+LoOmENMccT7B3t5rh5QfeP4N5PwdUT9v8Kn98BZ/baO5mIiE3ZtLwYhsGIESOYN28eK1asoGrVqjdcJyIiguXLl18xb+nSpURERNgqphQjJpOJR9tUZc7jrQjx9+LExTQemLKeL9Ycct5hpEa9rU+p9guBCwfhiw6w52d7pxIRsRmblpfhw4fz/fffM23aNHx8fIiLiyMuLo60tLTcZfr168e4ceNyX48ePZolS5bw3nvvsXfvXl599VW2bNnCiBEjbBlViplGIaVYOLItd9cPIivH4I1Fexj87RYSUjPtHe3mlA+DISuhSlvITIaZD8OKN3QejIgUSTa9VNpkMl11/tdff82AAQMAaN++PVWqVGHq1Km578+ePZsXX3yRI0eOULNmTd5++226dOmSp23qUmnJD8Mw+H7DUf6zcA+ZORbK+3kysW84TSpf+4o4h5aTbb0b74bJ1tc1O0PPz8HTz765RERuID/f34V6n5fCoPIiN2PnyURGTNvGkfOpuJhNPNO5NkPaVsNsvnoBd3jbZ8LPoyA7HcqFQt+ZULqKvVOJiFyTw97nRcRR1a/gx88j29C9UXlyLAZv/rKXR7/ZzPnkDHtHuzmNelnPg/EJhrN74fMOcHyTvVOJiBQIlReRP/l4ujGhdxjj72uAh6uZlfvO0mXCGjYdvmDvaDenfBgMXvHnYwXOwdRuEDvH3qlERG6ZyovI35hMJvo0r8T84a2pVq4E8UkZ9P4sikkr9mOxOOEIq295eHSJ9enUORnw4yBY+SYUrdFiESlmVF5ErqJOsC8/j2jDfeEVsBjw7m9/0P/rTZy95ITDSO4l4MHvoNUo6+uV42HuYMh2wn0REUHlReSaSni48n6vMN65vyGebmbW7D9HlwlrWH/gnL2j5Z/ZDJ3+A90ngNkVYmfD9z0h3Umf8yQixZrKi8gNPNA0hJ9HtKFWYEnOXsrgoS838sHSP8hxxmGkJv3hoTng7gNH1sDXXeHS1Z8bJiLiqFReRPKgZqAPC4a34cGmFTEM+Gj5fh7+YiNnktLtHS3/qt8OAxdBiQCIj4Uv74Rz++2dSkQkz1ReRPLIy92Ft+9vxAe9GuHt7kLUofPc/dEaVv9x1t7R8i+4EQz6DfyrQcIx+LITnNhi71QiInmi8iKST/eGV+TnkW0IDfLhfEom/b/exDu/7iU7x8luxe9fFR79Dco3hrQL8E132L/U3qlERG5I5UXkJlQvV5L5w1vTt0UlDAMm/36Qvp9v5HRi2o1XdiQly0H/n6F6B8hKhel9YPdP9k4lInJdKi8iN8nTzYX/3duAiX3CKenhyqYjF+jy0Rp+33vG3tHyx6Ok9fEB9e4DSxbMHmB9vICIiINSeRG5Rd0blWfhyDbUr+DLxdQsBk7dzPjFe8hypmEkFzfo+QWEPQRGDswbClu+tncqEZGrUnkRKQBVypbgx2GtGNCqCgCfrj5Er0+jOJngRMNIZhfoMQmaDQYMWDgGoibbO5WIyL+ovIgUEA9XF17tUY8pDzfGx9OVbccS6PLRGpbujrd3tLwzm6HLO9B6tPX1ry/A6nfsm0lE5B9UXkQK2F31g1k8qi2NKvqRmJbF4G+38PrPu8nMdpJhJJMJOr4Gt/+f9fWKN2CVCoyIOA6VFxEbCPH3ZvbjrRjUpioAX607zANT1nP8Qqqdk+WRyQTtnrWWGIDf34A179s3k4jIn1ReRGzE3dXMS93q8nm/pvh5ubH9RCJdJqxhyc7T9o6Wd23GQIeXrT8vfw3WTbBrHBERUHkRsbk76wayeHRbGlcqxaX0bB7/fhuvLNhJelaOvaPlTdunLg8hLX1JJ/GKiN2pvIgUggqlvJg5NIKh7aoB8E3UUXp+sp4j51LsnCyP2j0L7Z6z/vzrC7Bhin3ziEixpvIiUkjcXMyMu7sOXw9oRmlvN3adSqLbxLX8tP2UvaPlTftx1qMwAEueg+jv7ZtHRIotlReRQnZ7aACLR7eleRV/kjOyGTU9mnFzYx1/GMlkgjtegogR1tc/jYQ9C+2bSUSKJZUXETsI9vNi2uAWjLyjBiYTTN90jMjJ6zh4Ntne0a7PZIJOb0DYw2BYYM5AOLTK3qlEpJhReRGxE1cXM091qs23jzanbEl39sZdovvEtcyLPmHvaNdnMkH3jyC0G+Rkwoy+cHKrvVOJSDGi8iJiZ21rlmPxqLZEVCtDamYOT87czjOzt5OW6cDDSC6u0PNLqHobZCbD9/fD2X32TiUixYTKi4gDCPD15PvHWjCmY01MJpi99QQ9Jq3lj/hL9o52bW6e0HsalA+HtAvw3b2Q5CQnH4uIU1N5EXEQLmYTYzrW4ofHWlDOx4P9Z5LpMWkts7YcxzAMe8e7Og8feOhHKFMTkk7CtAchw4ELl4gUCSovIg6mVfWy/DK6LW1rliU9y8Kzc3bw1KztpGRk2zva1ZUoAw/PgRLlIC4WZg+AHAfNKiJFgsqLiAMqW9KDbwY255nOtTGbYG70SbpPWsue00n2jnZ1patA35ng6gUHlsHip8BRjxaJiNNTeRFxUGazieG312DGkAiCfD05dDaFyMnrmLbxmGMOI1VoAvd/CZhg61RY96GdA4lIUaXyIuLgmlf1Z/HotrSvXY6MbAsvzItl1IwYLqVn2Tvav4V2hbvetP687FXYNc+ucUSkaFJ5EXEC/iXc+ap/M8bdHYqr2cTP20/RfeJadp5MtHe0f2v5OLR8wvrzvGFwert984hIkWPT8rJ69Wq6d+9O+fLlMZlMzJ8//7rLr1y5EpPJ9K8pLi7OljFFnILZbGJou+rMHBpBhVJeHDmfyn0fr+fbqCOON4zU6Q2o0RGy02B6X0g+a+9EIlKE2LS8pKSk0KhRIyZPnpyv9fbt28fp06dzp4CAABslFHE+TSqXZtGoNnSsE0hmjoWXF+ziiR+2kZjmQMNIZhfo+QX4V4ekEzDrEcjOtHcqESkibFpe7r77bt544w3uvffefK0XEBBAUFBQ7mQ2a3RL5O9Kebvzeb8mvNStLm4uJn7ZGUe3iWvYfjzB3tEu8yoNfWaAhy8ci4JfntEVSCJSIByyFYSFhREcHMydd97JunXrrrtsRkYGSUlJV0wixYHJZGJQm6rMebwVIf5eHL+Qxv1T1vPl2sOOM4xUrpb1MQJ/XYG0+Qt7JxKRIsChyktwcDBTpkzhxx9/5McffyQkJIT27duzbdu2a64zfvx4/Pz8cqeQkJBCTCxif41CSrFwZFvurh9EVo7BfxbuZvC3W0lIdZBhmlqdoOMr1p+XPA/HN9k3j4g4PZNRSP8XzWQyMW/ePCIjI/O1Xrt27ahUqRLffffdVd/PyMggIyMj93VSUhIhISEkJibi6+t7K5FFnIphGHy34ShvLNxDZo6FCqW8mNAnnCaVS9s7mnW4aM5A66XTvhVg6GooUdbeqUTEgSQlJeHn55en72+HOvJyNc2bN+fAgQPXfN/DwwNfX98rJpHiyGQy0S+iCnOfaEXlMt6cTEij16dRfLrqIBaLnYeRTCboMfHyM5DmDgaLAz81W0QcmsOXl5iYGIKDg+0dQ8Rp1K/gx8KRbejWMJhsi8H4X/Yy6JvNXEix8zCShw88+K31EQIHV8Dqd+ybR0Sclk3LS3JyMjExMcTExABw+PBhYmJiOHbsGADjxo2jX79+uct/+OGHLFiwgAMHDrBz507GjBnDihUrGD58uC1jihQ5Pp5uTOwTzv/ubYCHq5nf952ly0dr2HT4gn2DBdaF7h9af175JhxYbtc4IuKcbFpetmzZQnh4OOHh4QCMHTuW8PBwXn75ZQBOnz6dW2QAMjMzeeqpp2jQoAHt2rVj+/btLFu2jA4dOtgypkiRZDKZ6NuiEvOHt6ZauRLEJaXT5/MNTP79gH2HkRr1hiYDAMM6fJR02n5ZRMQpFdoJu4UlPyf8iBQXKRnZvDR/J3OjTwLQtmZZPugVRtmSHvYJlJUOX3aEuFio1h4enge6n5NIsVakTtgVkVtXwsOV9x5sxNv3N8TTzcya/ee4+6M1rD94zj6B3Dyh51fW818OrYQN+bsLt4gUbyovIsWEyWTiwaYh/DSiDTUDSnL2UgYPf7GRD5f9QY49hpHK1YK7xlt/XvYanIop/Awi4pRUXkSKmVqBPvw0og0PNq2IxYAPl+3nkS83ciYpvfDDNBkAod3AkgU/PgaZKYWfQUScjsqLSDHk5e7C2/c34v0HG+Ht7sL6g+fpMmENa/YX8tOf/7r/i08wnN8PS8YV7vZFxCmpvIgUY/c1rshPI9oQGuTDueRM+n21iXd/3Ud2jqXwQnj7w72fAibY9g388WvhbVtEnJLKi0gxVyOgJPOHt6Zvi0oYBkz6/QB9P99IXGIhDiNVawcRf97P6adRkHax8LYtIk5H5UVE8HRz4X/3NmBCn3BKeriy6cgFukxYw+/7zhReiDtetD4+IDlOw0cicl0qLyKSq0ej8iwc2YZ65X25kJLJwK83M/6XPWQVxjCSmxdEfgwmM2yfDvt+sf02RcQpqbyIyBWqlC3Bj8Na0T+iMgCfrjpEr0+jOJmQZvuNhzS/PHz08xgNH4nIVam8iMi/eLq58No99fnkocb4eLqy7VgCXT5aw9Ld8bbf+O3/d3n46Jfnbb89EXE6Ki8ick13Nwhm0ci2NKroR2JaFoO/3cJ/Fu4mM9uGw0huXhD5iXX4aMcM2L/UdtsSEaek8iIi11WpjDezH2/Fo62rAvDl2sM88GkUxy+k2m6jIc2g5RPWnxeNhUwbbktEnI7Ki4jckLurmZe71+Xzfk3x83Jj+/EEukxYw5KdNnwidPtx4FsREo7Bqrdstx0RcToqLyKSZ3fWDWTRqDaEVyrFpfRsHv9+G68s2ElGdk7Bb8yjJHR5x/pz1CSI31Xw2xARp6TyIiL5UrG0N7OGRjC0XTUAvok6Ss9P1nPknA2eSxTa5c9nH2Vbrz6yFOKdf0XEYam8iEi+ubmYGXd3Hb4e0IzS3m7sPJlEt4lrWbjjVMFv7O63wb0knNgE26YW/OeLiNNReRGRm3Z7aACLR7elWZXSJGdkM2JaNC/MiyU9qwCHkfwqWO++C7DsNUgu5IdHiojDUXkRkVsS7OfF9MEtGXF7DUwmmLbxGJGT13HwbHLBbaTZYAhqAOkJsOyVgvtcEXFKKi8icstcXcw83bk23z7anDIl3Nkbd4nuE9cyL/pEwWzAxRW6fmD9OeYHOLq+YD5XRJySyouIFJi2Ncvxy+i2RFQrQ2pmDk/O3M6zc7aTllkAw0ghzaBxf+vPi56CnKxb/0wRcUoqLyJSoAJ8Pfn+sRaM7lATkwlmbTnBPZPXsj/+0q1/eMdXwcsfzuyGjZ/e+ueJiFNSeRGRAudiNvHknbX4YVALyvl48Ed8Mt0nrWX2luO39sHe/nDna9afV46HxJO3HlZEnI7Ki4jYTKsaZVk8qi1ta5YlPcvCM3N2MHZWDCkZ2Tf/oWEPQ8XmkJkMv75QcGFFxGmovIiITZXz8eCbgc15ulMtzCaYu+0kPSatZW9c0s19oNkM3d63Prhx93w4sLxA84qI41N5ERGbM5tNjLijJjOGRBDk68nBsyncM2kd0zcdwzCM/H9gUANoPtT68+KnISu9YAOLiENTeRGRQtO8qj+LR7elfe1yZGRbGDc3llEzYriUfhNXDt3+ApQMgguHYP2Egg8rIg5L5UVECpV/CXe+6t+McXeH4mI28fP2U3SfuJadJxPz90GevtD5v9af17wHFw4XfFgRcUgqLyJS6MxmE0PbVWfW0AjK+3ly5Hwq9328nu+ijuRvGKl+T6h6G2Snw8IxcDNDUCLidFReRMRumlQuzeLRbelYJ5DMHAsvLdjF8GnbSMrrMJLJBN0+BFdPOLQStn5ty7gi4iBUXkTErkp5u/N5vya82LUObi4mFsfG0XXCGrYfT8jbB5SpDh1etv7820tw8ajNsoqIY1B5ERG7M5lMPNa2GrMfb0XF0l4cv5DG/VPW89Xaw3kbRmrxOIS0tN775aeRGj4SKeJsWl5Wr15N9+7dKV++PCaTifnz599wnZUrV9K4cWM8PDyoUaMGU6dOtWVEEXEgYSGlWDSqLXfVCyIrx+D1hbsZ8t1WElIzr7+i2QUiPwZXLzi8CrZ8VTiBRcQubFpeUlJSaNSoEZMnT87T8ocPH6Zr167cfvvtxMTEMGbMGB577DF+/fVXW8YUEQfi5+XGJw835rUe9XB3MbN0dzxdJ6xl27GL11/x78NHS1/W8JFIEWYybuoOUTexIZOJefPmERkZec1lnnvuORYtWsTOnTtz5/Xu3ZuEhASWLFmSp+0kJSXh5+dHYmIivr6+txpbROxo58lEhk/bxtHzqbiaTTx7V20ea1MNs9l09RUsFpjaBY5FQaVWMGCh9aiMiDi8/Hx/O9Q5L1FRUXTs2PGKeZ07dyYqKuqa62RkZJCUlHTFJCJFQ/0Kfiwc2YZuDYPJthj8b/FeHvt2CxdSrjGMZDZbh4/cS8Kx9bD2/cINLCKFwqHKS1xcHIGBgVfMCwwMJCkpibS0tKuuM378ePz8/HKnkJCQwogqIoXEx9ONiX3C+d+9DXB3NbNi7xm6TljD5iMXrr6CfzXo8o7159/Hw/HNhRdWRAqFQ5WXmzFu3DgSExNzp+PHj9s7kogUMJPJRN8WlVgwvDXVypbgdGI6vT/bwOTfD2CxXGXku1Ef6w3sjByY+xik64isSFHiUOUlKCiI+Pj4K+bFx8fj6+uLl5fXVdfx8PDA19f3iklEiqY6wb78PLIN94ZXIMdi8M6v++j/9SbOJWdcuaDJBF3fB79KcPEILH7GLnlFxDYcqrxERESwfPmVj7dfunQpERERdkokIo6mhIcr7z/YiLd7NsTTzcya/efo8tEaog6ev3JBr1LQ83MwmWHHDNgx2y55RaTg2bS8JCcnExMTQ0xMDGC9FDomJoZjx44B1iGffv365S7/+OOPc+jQIZ599ln27t3Lxx9/zKxZs3jyySdtGVNEnIzJZOLBZiH8NKINNQNKcuZSBg99sYGPlu0n5+/DSJVawm3PWn9eOAbO/mGXvCJSsGxaXrZs2UJ4eDjh4eEAjB07lvDwcF5+2XovhtOnT+cWGYCqVauyaNEili5dSqNGjXjvvff44osv6Ny5sy1jioiTqhXow4IRrXmgSUUsBnyw7A8e+XIjZy6lX17otmegSlvr3Xdn9YPMFPsFFpECUWj3eSksus+LSPE0d9sJXpy/k9TMHMqWdOfDXuG0qVnW+ualePi0LSTHQ8NecO+n1vNiRMRhOO19XkREbtZ9jSvy04g2hAb5cC45k0e+2sh7v+0jO8cCPoFw/9dgcoEdM2HrVHvHFZFboPIiIkVGjYCSzB/emj7NK2EYMHHFAfp+sZG4xHSo0vry4wN+eRZORds3rIjcNJUXESlSPN1cGH9fAyb0CaeEuwubDl+gy4Q1rNx3BlqNgtpdICcTZvaDlHP2jisiN0HlRUSKpB6NyrNwVFvqlfflQkomA77ezJu//kFW98nWu/AmHrOewJt9gydWi4jDUXkRkSKratkS/DisFf0iKgMwZdVBen+3lzPdpoK7DxxdB0ues29IEck3lRcRKdI83Vx4/Z76fPxQY3w8XNl69CKdvo8npvm7gAm2fAWbv7R3TBHJB5UXESkWujQIZtGotjSs6EdCahaRy3xYUWGo9c1fnoUja+0bUETyTOVFRIqNSmW8mfN4Kx5tXRWARw+2ZbVHO7BkW89/uXjEvgFFJE9UXkSkWHF3NfNy97p89kgTfD3dGJI4gN1UhdTz8P39kHrB3hFF5AZUXkSkWOpUL4jFo9tSp1IgA9Kf5pThD+f3Y5nxMGRn3PgDRMRuVF5EpNiqWNqbWUMjuPe2JgzMfJYkwwvzsXWkzBoKFou944nINai8iEix5uZiZlyXOjzX/z6eNT9NluFCiT/msX/m8/aOJiLXoPIiIgLcERrIK2OG85nfKABq7vuUeV/8l/SsHDsnE5F/UnkREflTsJ8XQ0e/zPoKjwLQ4/g7vPPhuxw6m2znZCLydyovIiJ/4+piptVj73O6+oO4mAyeTX6bNyZ+wvzok/aOJiJ/UnkREfknk4ngh6aQXrMbHqZsJpje5atZP/LcnB2kZWoYScTeVF5ERK7G7IJnr68wqranpCmdqe5vsWXrBiInr+PAmUv2TidSrKm8iIhci6sHpt4/QIUm+JuSmeYxnuT4Q3SfuI45W0/YO51IsaXyIiJyPR4l4aE5UC6UQC4wt8Rb+GWd4enZ2xk7K4aUjGx7JxQpdlReRERuxNsfHpkHpSoTmHOKJaXfIch0kbnbTtJj0lr2xiXZO6FIsaLyIiKSF77lYcBCKFWJUmnH+L3ce9T1SeXg2RTumbSOGZuOYRiGvVOKFAsqLyIieVWqEvRfCH4heCUd4ieft+le3ZWMbAvPz41l9IwYkjWMJGJzKi8iIvlRujL0/wl8K+B64Q8mZL7Cqx0CcDGb+Gn7KbpPXMuuU4n2TilSpKm8iIjkl3816P8z+ARjOruHAX+MYO4j1Sjv58nhcync+/F6vttwVMNIIjai8iIicjPKVM8tMJzdS6OlfVg8oCod6wSQmW3hpfk7GTEtmqT0LHsnFSlyVF5ERG5W2Zow8BfruTAXDlFqRg8+7+bPi13r4Go2sSj2NN0mrGXHiQR7JxUpUlReRERuhX9VGLgEytSAxOOYvr6bx2pnMGdYKyqW9uLYhVR6frKer9cd1jCSSAFReRERuVV+FaxHYALqQXI8fN2FMJcjLBrVls71AsnKMXjt590M/W4riakaRhK5VSovIiIFoWSA9T4w5RtD2gX4pjt+cRuY8nATXutRD3cXM7/tjqfLhDVEH7to77QiTk3lRUSkoHj7Q78FULk1ZCTB9/dh2r2A/q2q8OOwVlQu483JhDQemBLF56sPYbFoGEnkZqi8iIgUJE9fePhHCO0GOZkwewBs+pwGFf1YOLINXRsGk20x+O/iPTz27RYupmTaO7GI0ymU8jJ58mSqVKmCp6cnLVq0YNOmTddcdurUqZhMpismT0/PwogpIlIw3LzgwW+h6aOAAYufhuWv4+PhyqQ+4fz33vq4u5pZsfcMXSasYcuRC/ZOLOJUbF5eZs6cydixY3nllVfYtm0bjRo1onPnzpw5c+aa6/j6+nL69Onc6ejRo7aOKSJSsMwu0PV9uP3/rK/XvAcLRmCy5PBQi8rMf6I11cqW4HRiOr0+28DHKw9oGEkkj2xeXt5//30GDx7MwIEDqVu3LlOmTMHb25uvvvrqmuuYTCaCgoJyp8DAQFvHFBEpeCYTtHsWuk8AkxlivodpD0J6InXL+/LTyDZEhpUnx2Lw9pJ9DJi6mXPJGfZOLeLwbFpeMjMz2bp1Kx07dry8QbOZjh07EhUVdc31kpOTqVy5MiEhIdxzzz3s2rXrmstmZGSQlJR0xSQi4lCa9IdeP4CbNxxcDl92gotHKOnhyge9wni7Z0M83cys/uMsXT5aw4ZD5+2dWMSh2bS8nDt3jpycnH8dOQkMDCQuLu6q69SuXZuvvvqKBQsW8P3332OxWGjVqhUnTpy46vLjx4/Hz88vdwoJCSnw/RARuWWhXaz3gvnzcQJ83gGOb8JkMvFgsxAWDG9DjYCSnLmUQd/PNzBh+X5yNIwkclUOd7VRREQE/fr1IywsjHbt2jF37lzKlSvHp59+etXlx40bR2JiYu50/PjxQk4sIpJH5cPgseUQ1ABSz8HUbhA7B4DaQT78NKI19zepiMWA95f+Qb+vNnLmUrp9M4s4IJuWl7Jly+Li4kJ8fPwV8+Pj4wkKCsrTZ7i5uREeHs6BAweu+r6Hhwe+vr5XTCIiDsuvgvVxArW7QE4G/DgIVr4FhoG3uyvvPtCI9x5ohJebC+sOnKfLR2tZd+CcvVOLOBSblhd3d3eaNGnC8uXLc+dZLBaWL19OREREnj4jJyeH2NhYgoODbRVTRKRweZSEXt9DxAjr65X/s94PJiMZgJ5NKvLzyDaEBvlwLjmDh7/cyPu/7dMwksifbD5sNHbsWD7//HO++eYb9uzZw7Bhw0hJSWHgwIEA9OvXj3HjxuUu//rrr/Pbb79x6NAhtm3bxsMPP8zRo0d57LHHbB1VRKTwmF2g83+h+0dgdoPd8+HLO+HCIQBqBJRk/vDW9GkegmHAhBUH6Pv5BuKTNIwkYvPy0qtXL959911efvllwsLCiImJYcmSJbkn8R47dozTp0/nLn/x4kUGDx5MnTp16NKlC0lJSaxfv566devaOqqISOFrMsD6TKSSgXBmN3zWHvYvA8DTzYXx9zXko95hlHB3YePhC9z90RpW7rv2fbJEigOTUcSe0Z6UlISfnx+JiYk6/0VEnEfSaZj1CJzYDJigw0vQZqz1XjHA4XMpDP9hG7tPW28HMax9dZ66sxauLg533YXITcnP97f+1YuIOALfYBiwCBr3BwxY/jrM7p97HkzVsiWY+0QrHmlZGYBPVh6k92cbOJWQZsfQIvah8iIi4ihcPaDHBOj24Z/nwSyAz2+HM3sA6zDSfyLr8/FDjfHxcGXL0Yt0mbCG5Xvir/+5IkWMyouIiKNpOtB6FKZkEJz7Az67HaJ/yH27S4NgFo1qS8OKfiSkZjHomy38d9FuMrMtdgwtUnhUXkREHFGlFvD4Wqh2O2SnwYInYP4TkJlqfbuMN7Mfj2Bg6yoAfL7mMA9+GsXxC6l2DC1SOFReREQcVcly8PBcuP3FPx/s+AN8fgec3QeAh6sLr3Svx6ePNMHX05WY4wl0nbCGX3dd/fErIkWFyouIiCMzm6HdM9BvgfVy6rN7rMNI22fkLtK5XhCLR7clvFIpktKzGfrdVl79aRcZ2Tl2DC5iOyovIiLOoOpt1mGkqu0gKwXmDYUfB0N6IgAVS3sza2gEQ26rBsDU9Ue4/5Mojp5PsWdqEZtQeRERcRYlA+CRedD+BTC5QOwsmNIGjm0AwM3FzAtd6vDVgKaU9nYj9mQi3SasZdGO0zf4YBHnovIiIuJMzC7Q/jl4dAmUqgwJx+Dru+H3/0FONgB3hAayeHRbmlYuzaWMbIZP28aL82NJz9IwkhQNKi8iIs4opLl1GKlhbzAssOot+PouuHAYgGA/L2YMackT7asD8P2GY9z78XoOnU22Z2qRAqHyIiLirDx94b5PoeeX4OFnfbTAlDbWe8IYBq4uZp69K5RvHm1OmRLu7DmdRPeJa1kQc9LeyUVuicqLiIiza3A/DFsLlVpBZrL1njDTe8Ml6yXT7WqVY/HotrSs5k9KZg6jZ8Tw/I87SMvUMJI4J5UXEZGioFQl69OpO7wCLu7wxxKY3AJ2zALDINDXkx8ea8moDjUxmWDG5uNETl7HgTOX7J1cJN9UXkREigqzC7QdC0NWQXAYpCfA3MEw82FIPoOL2cTYO2vx/aAWlC3pwb74S3SfuI45W0/YO7lIvqi8iIgUNYF14bFl1jvzmt1g70LrUZidPwLQukZZfhndljY1ypKWlcPTs7fz1KztpGZm2zm4SN6ovIiIFEUubtY78w75HYIaQNoFmPMozOoHl+Ip5+PBN48256k7a2E2wY/bTtBj0jr2xWkYSRyfyouISFEW1AAeWwHtngezK+xeAJObwdapuGAwskNNpg1uSaCvBwfOJNNj0lpmbj6GYRj2Ti5yTSajiP0LTUpKws/Pj8TERHx9fe0dR0TEcZzeAT+PglPR1teVW0P3j6BsTc4nZzB21nZW/XEWgMiw8rxxbwNKerjaMbAUJ/n5/taRFxGR4iK4IQxaBp3/B27ecHQdfNIKVr1NGU8TXw9oxnN3heJiNjE/5hQ9Jq5l16lEe6cW+RcdeRERKY4uHoVFY+HAMuvrcnWgxwQIac6WIxcYOT2a04npuLuaealbXR5uUQmTyWTfzFKk6ciLiIhcX+nK8NAc6915vcvC2T3wZSf4aRRNyxksHtWWDqEBZGZbeGn+TkZMjyYpPcveqUUAlRcRkeLLZLLenXfEZgh/GDBg2zcwqQml93zPF4+E82LXOriaTSzacZpuE9YSe0LDSGJ/Ki8iIsWdtz/cMxke/RUCG0DaRVj4JKYvO/JYtYvMfjyCCqW8OHYhlZ6frGfqusO6GknsSuVFRESsKrWEISvh7rfBw9d6VdLnHQjf/iq/DK5H53qBZOZYePXn3Tz+/VYSUzWMJPah8iIiIpe5uEKLoTByKzTqAxiwdSq+X7RgSugOXutWG3cXM7/uiqfrxDVEH7to78RSDKm8iIjIv5UMgHunwMBfIKAepF3EtOhJ+sf2Z8k9BpX8vTlxMY0HpkTxxZpDGkaSQqXyIiIi11a5FQxdDXe9BZ6lIH4n1Rb3YXmFTxkQmkO2xeCNRXt47JstXEzJtHdaKSZ0nxcREcmb1Auw6i3Y9DkYORhmN/ZU6sMjB9pxPtuL8n6eTOwbTpPK/vZOKk5I93kREZGC5+0Pd78FT0RBzU6YLFnUPfItG0s+w2i/1cQnpvDgpxv4ZOVBLJYi9f+LxcHoyIuIiNyc/cvg1xfg3D4ATrtX4cXkniy3NKZdrQDef7ARZUp62DmkOAsdeREREdur2RGGrYMu74JXaYIzj/Cl+3vM8XidS/vX0WXCGjYeOm/vlFIEFUp5mTx5MlWqVMHT05MWLVqwadOm6y4/e/ZsQkND8fT0pEGDBixevLgwYoqISH65uEHzwTAqBto8Ca6eNDXtY67Hq7yeNp4Xv/iRicv3k6NhJClANi8vM2fOZOzYsbzyyits27aNRo0a0blzZ86cOXPV5devX0+fPn0YNGgQ0dHRREZGEhkZyc6dO20dVUREbpZXKej4KoyKhsb9MExmOrtsYYnbs5T7/WnGfLaQs5cy7J1Sigibn/PSokULmjVrxqRJkwCwWCyEhIQwcuRInn/++X8t36tXL1JSUli4cGHuvJYtWxIWFsaUKVNuuD2d8yIi4gDO7oPlr8Ne6/+WpxtuzDB3JfT+l2lZr7qdw4kjcphzXjIzM9m6dSsdO3a8vEGzmY4dOxIVFXXVdaKioq5YHqBz587XXD4jI4OkpKQrJhERsbNytaH3D/Dob6QFN8fTlMUAYz51Z7Vh/ZfPkJOaYO+E4sRsWl7OnTtHTk4OgYGBV8wPDAwkLi7uquvExcXla/nx48fj5+eXO4WEhBRMeBERuXWVWuA15DcyH5hGnGc1fE2ptDr+GWnv1OPSsrcgI9neCcUJOf3VRuPGjSMxMTF3On78uL0jiYjI35lMuNfrStCzW9nc7D0OGuUpaSTjs/Z/ZL5fH9ZNgMxUe6cUJ2LT8lK2bFlcXFyIj4+/Yn58fDxBQUFXXScoKChfy3t4eODr63vFJCIiDshsplnXxzA9sYG3vZ/isCUQ94yLsPQljI8awYZPICvd3inFCdi0vLi7u9OkSROWL1+eO89isbB8+XIiIiKuuk5ERMQVywMsXbr0msuLiIhzqRbox6gn/4+p4bN4JmsIxy3lMKWcgSXPw4Qw2PgpZKXZO6Y4MJsPG40dO5bPP/+cb775hj179jBs2DBSUlIYOHAgAP369WPcuHG5y48ePZolS5bw3nvvsXfvXl599VW2bNnCiBEjbB1VREQKiaebC6/dG0b7XmPpYfqIcVmDOE0ZuHQafnkWPmwI6z6CjEv2jioOyNXWG+jVqxdnz57l5ZdfJi4ujrCwMJYsWZJ7Uu6xY8cwmy93qFatWjFt2jRefPFFXnjhBWrWrMn8+fOpX7++raOKiEgh69owmPoV2jNimh/tTt7Ggy4rebrEL5RKiYOlL8PaD6DFMGgxBLxK2zuuOAg920hEROwuIzuHN3/Zy9frjuBKNqPKRTPM5SfcEg5aF3D3sd7JN2I4lChr37BiE/n5/lZ5ERERh/Hrrjiemb2dpPRsSnma+ablaRod+gLO7LIu4OoFTQdCq5HgW96+YaVAqbyovIiIOK3jF1IZOT2amOMJAAxsVYkXahzFbe17cGqbdSGzGzTsZS0xAaH2CysFRuVF5UVExKllZlt459e9fL7mMAANK/oxqXc4lRI3wup34ei6ywvX7AytR0Hl1mAy2Smx3CqVF5UXEZEiYfmeeJ6avZ2E1Cx8PFx56/6GdGkQDCe2WK9G2vMz8OfXWPnG1hJTpweYXeyaW/JP5UXlRUSkyDiVkMao6dFsOXoRgEdaVub/utbB080Fzh+EqMkQ8wNk/3mDu9JVIGIEhD0E7t72Cy75ovKi8iIiUqRk5Vj4YOkffLzSevVR3WBfJj/UmKplS1gXSDkHmz6DTZ9D2gXrPC9/aPaYdfIJvMYni6NQeVF5EREpklb9cZYnZ8ZwISWTEu4u/O++BtwTVuHyApmp1qMwUZPg4hHrPLMb1L8PWjwOFRrbJbfcmMqLyouISJEVn5TOqOnRbDxsPcLSp3kIr3SvZx1G+oslB/b8ZH1e0vGNl+eHtLCWmDrdwcWtkJPL9ai8qLyIiBRp2TkWJqw4wMQV+zEMCA3yYVLfxtQIKPnvhU9ug41TYOdcsGRZ5/lWsA4nNRkA3v6Fml2uTuVF5UVEpFhYd+Aco2fEcC45Ay83F96IrE/PJhWvvvClONjylXVKOWud5+oFDR+0Ho0JrFt4weVfVF5UXkREio0zl9J5cmYM6w6cB+D+JhV5/Z56eLtf4/F92Rmw80frkFLcjsvzK7eBZo9CaHdwdS+E5PJ3Ki8qLyIixUqOxWDy7wf4cNkfWAyoGVCSyQ81plagz7VXMgw4FmUdUtrzMxgW6/wSAdC4n3VIqVRIoeQXlReVFxGRYmrDofOMnhFNfFIGnm5mXutRjwebhmC60Z13E0/Ctm9g6zeQHGedZzJb797bbBBU7wBms+13oBhTeVF5EREpts4nZ/DkrO2s/sN6XktkWHneuLcBJT2uMYz0dzlZsG8xbP4CDq++PL9UZWj6KIQ/rKda24jKi8qLiEixZrEYTFl9kPd++4Mci0G1siWY1Lcxdcvn43vh3H7ryb0xP0B6onWeizvUjYQm/fUspQKm8qLyIiIiwJYjFxg5PZrTiem4u5p5pXtd+javdONhpL/LTLWe4LvlSzgVfXm+f3Vo/Ag06qs7+BYAlReVFxER+dPFlEyenr2d5XvPANC1YTBv3tcAH8+buEndyW3Wc2Ni50BmsnWeyQVq3WU9ybdGR3DJw/CU/IvKi8qLiIj8jWEYfLn2MG/+spdsi0HlMt5M7tuY+hX8bu4DM5Jh93zY9u2Vd/D1CYawvtZzY/yrFUj24kLlReVFRESuIvrYRUZMi+ZkQhruLmb+r2sd+kVUzt8w0j+d2QvR38H26ZB6/vL8qrdBeD+o0w3cvG49fBGn8qLyIiIi15CYmsUzc7bz2+54AO6qF8Rb9zfEz+sWn3WUnWm9Umnbt3BwBfDn16uHH9SLhEZ9oFJLneR7DSovKi8iInIdhmEwdf0R/rd4D1k5BhVLezGpb2PCQkoVzAYSjkH0D9YrlRKPX55fuqq1xDTqBaWrFMy2igiVF5UXERHJgx0nEhgxLZpjF1JxNZt4/u5QBrWpemvDSH9nscDRtbB9BuyaD1kpl9+r3NpaZOreA576vlJ5UXkREZE8SkrPYtyPsSyKPQ1AxzoBvPtAI0p5F/DzjTJTrI8h2D4dDq0id1jJ1ct6XkyjPlCtPZhdCna7TkLlReVFRETywTAMvt94jP8s3E1mtoXyfp5M7BtOk8r+ttlg4gnYMRNipsP5/Zfn+wRDg/uhwQMQ1LBYnR+j8qLyIiIiN2HXqURGTIvm8LkUXMwmnulcmyFtq2E226hEGIb13jHbp1nvHZOecPm9srWsJaZ+TyhT3TbbdyAqLyovIiJyk5Izsvm/ebEsiDkFQPva5XjvgUaUKelh2w1nZ8D+3yB2NvzxK2SnX36vfOM/i8x94BNk2xx2ovKi8iIiIrfAMAxmbj7OKz/tIiPbQqCvBxN6h9OiWpnCCZCeBHsXWYvMoZVg5Pz5hgmqtrUWmTrdwat04eQpBCovKi8iIlIA9sYlMfyHbRw8m4LZBGPvrMUT7WvYbhjpapLPWu/mGzv7yrv5urhDjTut58jU6gzuJQovkw2ovKi8iIhIAUnNzOal+bv4cdsJANrWLMv7D4ZRzsfGw0hXc/Go9SGRsXPgzK7L8928oead1ideO2mRUXlReRERkQI2Z+sJXpq/k7SsHMr5ePBRrzBa1Shrv0Dxu2HnHGuZuXjk8nxXL6jVyemKjMqLyouIiNjA/vhLjJgWzb74S5hMMPKOmozuUBOXwhxG+ifDgNPbYdc86/DSP4tMzTutjyeo2Rk8Stop5I3l5/vbbMsgFy5c4KGHHsLX15dSpUoxaNAgkpOTr7tO+/btMZlMV0yPP/64LWOKiIjkSc1AH+YPb03vZiEYBkxYvp+HvthAfFL6jVe2FZMJyofBna/BqBgYsgraPGl9/EB2Guz5CeY8Cu/UgJkPW4/UZFz/u9jR2fTIy913383p06f59NNPycrKYuDAgTRr1oxp06Zdc5327dtTq1YtXn/99dx53t7eeT6KoiMvIiJSGBbEnOSFubGkZOZQpoQ7H/QK47Za5ewd67K/jsjsnm99NMHFw5ffc/WE6h0gtCvUvhu8bXQzvnxwiGGjPXv2ULduXTZv3kzTpk0BWLJkCV26dOHEiROUL1/+quu1b9+esLAwPvzww5varsqLiIgUlkNnkxk+LZo9p5MwmeCJ9tV5smMtXF1sOrCRf9crMiYXqNzKeul1aFfwq2iXiA5RXr766iueeuopLl68mDsvOzsbT09PZs+ezb333nvV9dq3b8+uXbswDIOgoCC6d+/OSy+9hLe391WXz8jIICMjI/d1UlISISEhKi8iIlIo0rNyeGPRbr7fcAyAZlVKM6FPOMF+XnZOdg2GAXGxsHeh9V4y8TuvfL98OIR2s5aZcrULLVZ+yourrULExcUREBBw5cZcXfH39ycuLu6a6/Xt25fKlStTvnx5duzYwXPPPce+ffuYO3fuVZcfP348r732WoFmFxERyStPNxfeiGxAy2pleP7HWDYfuUiXj9bw/oNh3B4acOMPKGwmEwQ3tE63vwAXDllLzJ6F1vvInIq2Tiv+A2VqWh8aGdrdWmrMjnFEKd9HXp5//nneeuut6y6zZ88e5s6dyzfffMO+ffuueC8gIIDXXnuNYcOG5Wl7K1asoEOHDhw4cIDq1f/9bAcdeREREUdx9HwKI6ZFE3syEYCht1Xj6c61cXO0YaRruRQP+xZby8yhlWDJuvyeT3kI7WIdWqrcBlwL9qnbNh02Onv2LOfPn7/uMtWqVeP777+/qWGjf0pJSaFkyZIsWbKEzp0733B5nfMiIiL2lJGdw/jFe5m6/ggA4ZVKMbFPOBVLX/30B4eVnmR91tLehbB/KWT+7QqlEuVg7B5wcSuwzdl02KhcuXKUK3fjs6kjIiJISEhg69atNGnSBLAeRbFYLLRo0SLP24uJiQEgODg4v1FFREQKnYerC6/2qEfLamV4ds52oo8l0HXCWt65vyGd6jnRQxU9fa2PHmhwP2Slw+HVsPdn2LcEghsVaHHJL5tfKh0fH8+UKVNyL5Vu2rRp7qXSJ0+epEOHDnz77bc0b96cgwcPMm3aNLp06UKZMmXYsWMHTz75JBUrVmTVqlV52qaOvIiIiKM4fiGVEdOj2X48AYBHW1fl+btDcXd1kmGkq7FYIO0ilCjYh1Q6zE3qfvjhB0JDQ+nQoQNdunShTZs2fPbZZ7nvZ2VlsW/fPlJTUwFwd3dn2bJldOrUidDQUJ566il69uzJzz//bMuYIiIiNhHi783soREMblsVgK/WHeb+Kes5dj7Vzslugdlc4MUlv/R4ABERkUKwbHc8T8/ZTkJqFj4errx9f0PubqBTIv7iMEdeRERExKpj3UAWj2pLk8qluZSRzbAftvHygp2kZ+XYO5rTUXkREREpJOVLeTFjSEuGtbfe+uPbqKP0/GQ9h8+l2DmZc1F5ERERKURuLmaeuyuUqQOb4V/CnV2nkug+cS0/bT9l72hOQ+VFRETEDtrXDmDxqLY0r+pPckY2o6ZHM25urIaR8kDlRURExE6C/DyZ9lgLRt1RA5MJpm86RuTkdRw4k3zjlYsxlRcRERE7cnUxM7ZTbb57tAVlS3qwN+4SPSatZe62E/aO5rBUXkRERBxAm5plWTy6Da2qlyE1M4exs7bzzOztpGZm2zuaw1F5ERERcRABPp58N6gFY++shdkEs7ee4J5J6/gj/pK9ozkUlRcREREH4mI2MapDTX54rCUBPh7sP5NMj0lrmbXlOEXsvrI3TeVFRETEAUVUL8Pi0W1pW7Ms6VkWnp2zg7GztpOSoWEklRcREREHVbakB98MbM6zd9XGxWxiXvRJuk9cy57TSfaOZlcqLyIiIg7MbDbxRPsazBjSkmA/Tw6dS+Geyev4YePRYjuMpPIiIiLiBJpV8WfRqLbcERpAZraF/5u3k5HTo7mUnmXvaIVO5UVERMRJ+Jdw54t+Tfm/LnVwNZtYuOM03SeuZefJRHtHK1QqLyIiIk7EbDYx+LZqzHo8ggqlvDhyPpX7Pl7PN+uPFJthJJUXERERJ9S4UmkWj2pLp7qBZOZYeOWnXTzxwzYS04r+MJLKi4iIiJPy83bj00ea8Er3uri5mPhlZxzdJq5h+/EEe0ezKZUXERERJ2YymRjYuio/DmtFJX9vjl9I4/4p6/ly7eEiO4yk8iIiIlIENKxYioWj2tClQRBZOQb/Wbibwd9uJSE1097RCpzKi4iISBHh6+nG5L6N+U9kfdxdzSzbE0/XCWvZevSivaMVKJUXERGRIsRkMvFIy8rMe6IVVcuW4GRCGg9+GsWUVQexWIrGMJLKi4iISBFUr7wfP49sQ49G5cmxGLz5y14e/WYzF1KcfxhJ5UVERKSIKunhyke9w3jzvgZ4uJpZue8sXT5aw6bDF+wd7ZaovIiIiBRhJpOJ3s0rsWBEa6qXK0FcUjq9P4ti0or9TjuMpPIiIiJSDIQG+fLTiDbc17gCFgPe/e0P+n+9ibOXMuwdLd9UXkRERIqJEh6uvP9gGO/c3xAvNxfW7D9HlwlrWH/wnL2j5YvKi4iISDHzQNMQfhrRmlqBJTl7KYOHv9jIh8v+IMdJhpFUXkRERIqhmoE+LBjehl5NQ7AY8OGy/Tz8xUbOJKXbO9oNqbyIiIgUU17uLrx1f0M+7BWGt7sLUYfO02XCGtbsP2vvaNel8iIiIlLMRYZXYOHINtQJ9uVccib9vtrEu7/uIzvHYu9oV6XyIiIiIlQrV5J5T7TioRaVMAyY9PsB+n6+kdOJafaO9i82Ky///e9/adWqFd7e3pQqVSpP6xiGwcsvv0xwcDBeXl507NiR/fv32yqiiIiI/I2nmwv/vbcBk/qGU9LDlU1HLtDlozX8vveMvaNdwWblJTMzkwceeIBhw4bleZ23336bCRMmMGXKFDZu3EiJEiXo3Lkz6emOf/KQiIhIUdGtYXkWjWpD/Qq+XEzNYuDUzYxfvIcsBxlGMhmGYdProqZOncqYMWNISEi47nKGYVC+fHmeeuopnn76aQASExMJDAxk6tSp9O7dO0/bS0pKws/Pj8TERHx9fW81voiISLGVkZ3D+MV7mbr+CACNK5ViYt/GVCjlVeDbys/3t8Oc83L48GHi4uLo2LFj7jw/Pz9atGhBVFTUNdfLyMggKSnpiklERERunYerC6/2qMeUhxvj4+nKtmMJdPloDUt3x9s1l8OUl7i4OAACAwOvmB8YGJj73tWMHz8ePz+/3CkkJMSmOUVERIqbu+oHs3hUWxqFlCIxLYuxM2NISLXf06nzVV6ef/55TCbTdae9e/faKutVjRs3jsTExNzp+PHjhbp9ERGR4iDE35vZQyN4rE1V/ndfA0p5u9sti2t+Fn7qqacYMGDAdZepVq3aTQUJCgoCID4+nuDg4Nz58fHxhIWFXXM9Dw8PPDw8bmqbIiIiknfurmZe7FbX3jHyV17KlStHuXLlbBKkatWqBAUFsXz58tyykpSUxMaNG/N1xZKIiIgUbTY75+XYsWPExMRw7NgxcnJyiImJISYmhuTk5NxlQkNDmTdvHgAmk4kxY8bwxhtv8NNPPxEbG0u/fv0oX748kZGRtoopIiIiTiZfR17y4+WXX+abb77JfR0eHg7A77//Tvv27QHYt28fiYmJucs8++yzpKSkMGTIEBISEmjTpg1LlizB09PTVjFFRETEydj8Pi+FTfd5ERERcT5OeZ8XERERkbxQeRERERGnovIiIiIiTkXlRURERJyKyouIiIg4FZUXERERcSoqLyIiIuJUVF5ERETEqai8iIiIiFOx2eMB7OWvGwYnJSXZOYmIiIjk1V/f23m58X+RKy+XLl0CICQkxM5JREREJL8uXbqEn5/fdZcpcs82slgsnDp1Ch8fH0wmU4F+dlJSEiEhIRw/frxIPjepqO8fFP191P45v6K+j9o/52erfTQMg0uXLlG+fHnM5uuf1VLkjryYzWYqVqxo0234+voW2X+UUPT3D4r+Pmr/nF9R30ftn/OzxT7e6IjLX3TCroiIiDgVlRcRERFxKiov+eDh4cErr7yCh4eHvaPYRFHfPyj6+6j9c35FfR+1f87PEfaxyJ2wKyIiIkWbjryIiIiIU1F5EREREaei8iIiIiJOReVFREREnIrKy3UcOXKEQYMGUbVqVby8vKhevTqvvPIKmZmZ110vPT2d4cOHU6ZMGUqWLEnPnj2Jj48vpNT589///pdWrVrh7e1NqVKl8rTOgAEDMJlMV0x33XWXbYPepJvZP8MwePnllwkODsbLy4uOHTuyf/9+2wa9BRcuXOChhx7C19eXUqVKMWjQIJKTk6+7Tvv27f/1O3z88ccLKfH1TZ48mSpVquDp6UmLFi3YtGnTdZefPXs2oaGheHp60qBBAxYvXlxISW9efvZx6tSp//pdeXp6FmLa/Fm9ejXdu3enfPnymEwm5s+ff8N1Vq5cSePGjfHw8KBGjRpMnTrV5jlvVn73b+XKlf/6/ZlMJuLi4goncD6NHz+eZs2a4ePjQ0BAAJGRkezbt++G6xX236HKy3Xs3bsXi8XCp59+yq5du/jggw+YMmUKL7zwwnXXe/LJJ/n555+ZPXs2q1at4tSpU9x3332FlDp/MjMzeeCBBxg2bFi+1rvrrrs4ffp07jR9+nQbJbw1N7N/b7/9NhMmTGDKlCls3LiREiVK0LlzZ9LT022Y9OY99NBD7Nq1i6VLl7Jw4UJWr17NkCFDbrje4MGDr/gdvv3224WQ9vpmzpzJ2LFjeeWVV9i2bRuNGjWic+fOnDlz5qrLr1+/nj59+jBo0CCio6OJjIwkMjKSnTt3FnLyvMvvPoL1TqZ//10dPXq0EBPnT0pKCo0aNWLy5Ml5Wv7w4cN07dqV22+/nZiYGMaMGcNjjz3Gr7/+auOkNye/+/eXffv2XfE7DAgIsFHCW7Nq1SqGDx/Ohg0bWLp0KVlZWXTq1ImUlJRrrmOXv0ND8uXtt982qlates33ExISDDc3N2P27Nm58/bs2WMARlRUVGFEvClff/214efnl6dl+/fvb9xzzz02zVPQ8rp/FovFCAoKMt55553ceQkJCYaHh4cxffp0Gya8Obt37zYAY/PmzbnzfvnlF8NkMhknT5685nrt2rUzRo8eXQgJ86d58+bG8OHDc1/n5OQY5cuXN8aPH3/V5R988EGja9euV8xr0aKFMXToUJvmvBX53cf8/G06GsCYN2/edZd59tlnjXr16l0xr1evXkbnzp1tmKxg5GX/fv/9dwMwLl68WCiZCtqZM2cMwFi1atU1l7HH36GOvORTYmIi/v7+13x/69atZGVl0bFjx9x5oaGhVKpUiaioqMKIWChWrlxJQEAAtWvXZtiwYZw/f97ekQrE4cOHiYuLu+L35+fnR4sWLRzy9xcVFUWpUqVo2rRp7ryOHTtiNpvZuHHjddf94YcfKFu2LPXr12fcuHGkpqbaOu51ZWZmsnXr1iv+25vNZjp27HjN//ZRUVFXLA/QuXNnh/xdwc3tI0BycjKVK1cmJCSEe+65h127dhVG3ELhbL/DmxUWFkZwcDB33nkn69ats3ecPEtMTAS47veePX6HRe7BjLZ04MABJk6cyLvvvnvNZeLi4nB3d//X+RWBgYEOO8aZX3fddRf33XcfVatW5eDBg7zwwgvcfffdREVF4eLiYu94t+Sv31FgYOAV8x319xcXF/evw8+urq74+/tfN2/fvn2pXLky5cuXZ8eOHTz33HPs27ePuXPn2jryNZ07d46cnJyr/rffu3fvVdeJi4tzmt8V3Nw+1q5dm6+++oqGDRuSmJjIu+++S6tWrdi1a5fNH0JbGK71O0xKSiItLQ0vLy87JSsYwcHBTJkyhaZNm5KRkcEXX3xB+/bt2bhxI40bN7Z3vOuyWCyMGTOG1q1bU79+/WsuZ4+/w2J55OX555+/6glUf5/++T8kJ0+e5K677uKBBx5g8ODBdkqeNzezf/nRu3dvevToQYMGDYiMjGThwoVs3ryZlStXFtxOXIet988R2HofhwwZQufOnWnQoAEPPfQQ3377LfPmzePgwYMFuBdSECIiIujXrx9hYWG0a9eOuXPnUq5cOT799FN7R5M8qF27NkOHDqVJkya0atWKr776ilatWvHBBx/YO9oNDR8+nJ07dzJjxgx7R/mXYnnk5amnnmLAgAHXXaZatWq5P586dYrbb7+dVq1a8dlnn113vaCgIDIzM0lISLji6Et8fDxBQUG3EjvP8rt/t6patWqULVuWAwcO0KFDhwL73Gux5f799TuKj48nODg4d358fDxhYWE39Zk3I6/7GBQU9K8TPbOzs7lw4UK+/r21aNECsB5drF69er7zFoSyZcvi4uLyryvzrve3ExQUlK/l7e1m9vGf3NzcCA8P58CBA7aIWOiu9Tv09fV1+qMu19K8eXPWrl1r7xjXNWLEiNwLAG50hM8ef4fFsryUK1eOcuXK5WnZkydPcvvtt9OkSRO+/vprzObrH6xq0qQJbm5uLF++nJ49ewLWs8yPHTtGRETELWfPi/zsX0E4ceIE58+fv+LL3pZsuX9Vq1YlKCiI5cuX55aVpKQkNm7cmO8rsm5FXvcxIiKChIQEtm7dSpMmTQBYsWIFFoslt5DkRUxMDECh/Q6vxt3dnSZNmrB8+XIiIyMB62Hr5cuXM2LEiKuuExERwfLlyxkzZkzuvKVLlxba31p+3cw+/lNOTg6xsbF06dLFhkkLT0RExL8uq3Xk32FBiImJsevf2vUYhsHIkSOZN28eK1eupGrVqjdcxy5/hzY7FbgIOHHihFGjRg2jQ4cOxokTJ4zTp0/nTn9fpnbt2sbGjRtz5z3++ONGpUqVjBUrVhhbtmwxIiIijIiICHvswg0dPXrUiI6ONl577TWjZMmSRnR0tBEdHW1cunQpd5natWsbc+fONQzDMC5dumQ8/fTTRlRUlHH48GFj2bJlRuPGjY2aNWsa6enp9tqNa8rv/hmGYbz55ptGqVKljAULFhg7duww7rnnHqNq1apGWlqaPXbhhu666y4jPDzc2Lhxo7F27VqjZs2aRp8+fXLf/+e/0QMHDhivv/66sWXLFuPw4cPGggULjGrVqhm33XabvXYh14wZMwwPDw9j6tSpxu7du40hQ4YYpUqVMuLi4gzDMIxHHnnEeP7553OXX7duneHq6mq8++67xp49e4xXXnnFcHNzM2JjY+21CzeU33187bXXjF9//dU4ePCgsXXrVqN3796Gp6ensWvXLnvtwnVdunQp9+8MMN5//30jOjraOHr0qGEYhvH8888bjzzySO7yhw4dMry9vY1nnnnG2LNnjzF58mTDxcXFWLJkib124bryu38ffPCBMX/+fGP//v1GbGysMXr0aMNsNhvLli2z1y5c17Bhwww/Pz9j5cqVV3znpaam5i7jCH+HKi/X8fXXXxvAVae/HD582ACM33//PXdeWlqa8cQTTxilS5c2vL29jXvvvfeKwuNI+vfvf9X9+/v+AMbXX39tGIZhpKamGp06dTLKlStnuLm5GZUrVzYGDx6c+z+8jia/+2cY1sulX3rpJSMwMNDw8PAwOnToYOzbt6/ww+fR+fPnjT59+hglS5Y0fH19jYEDB15Rzv75b/TYsWPGbbfdZvj7+xseHh5GjRo1jGeeecZITEy00x5caeLEiUalSpUMd3d3o3nz5saGDRty32vXrp3Rv3//K5afNWuWUatWLcPd3d2oV6+esWjRokJOnH/52ccxY8bkLhsYGGh06dLF2LZtmx1S581flwb/c/prn/r372+0a9fuX+uEhYUZ7u7uRrVq1a74e3Q0+d2/t956y6hevbrh6elp+Pv7G+3btzdWrFhhn/B5cK3vvL//Thzh79D0Z1gRERERp1AsrzYSERER56XyIiIiIk5F5UVEREScisqLiIiIOBWVFxEREXEqKi8iIiLiVFReRERExKmovIiIiIhTUXkRERERp6LyIiIiIk5F5UVEREScisqLiIiIOJX/ByxbAqsR2KG9AAAAAElFTkSuQmCC\",\n      \"text/plain\": [\n       \"<Figure size 640x480 with 1 Axes>\"\n      ]\n     },\n     \"metadata\": {},\n     \"output_type\": \"display_data\"\n    }\n   ],\n   \"source\": [\n    \"import numpy as np\\n\",\n    \"import matplotlib.pyplot as plt\\n\",\n    \"\\n\",\n    \"def bisymlog(x, C=None):\\n\",\n    \"    if C is None:\\n\",\n    \"        C = 1/np.log(10)\\n\",\n    \"\\n\",\n    \"    return np.sign(x)*(np.log10(1 + np.abs(x/C)))*np.log(10)\\n\",\n    \"\\n\",\n    \"x = np.linspace(-2, 2, 1000)\\n\",\n    \"y = np.ones_like(x)*1\\n\",\n    \"\\n\",\n    \"rel_err = (y-x)/y\\n\",\n    \"C = None\\n\",\n    \"log_err = bisymlog(y, C) - bisymlog(x, C)\\n\",\n    \"\\n\",\n    \"# plot log error and relative error\\n\",\n    \"plt.plot(x, rel_err)\\n\",\n    \"plt.plot(x, log_err)\\n\",\n    \"plt.show()\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": 120,\n   \"metadata\": {},\n   \"outputs\": [\n    {\n     \"data\": {\n      \"text/plain\": [\n       \"0.43429448190325187\"\n      ]\n     },\n     \"execution_count\": 120,\n     \"metadata\": {},\n     \"output_type\": \"execute_result\"\n    }\n   ],\n   \"source\": [\n    \"1/np.log(10)\"\n   ]\n  },\n  {\n   \"cell_type\": \"code\",\n   \"execution_count\": null,\n   \"metadata\": {},\n   \"outputs\": [],\n   \"source\": [\n    \"def get_cg_ratio_beam(\\n\",\n    \"    # this is coming in per hour for an individual cluster\\n\",\n    \"    obs: np.ndarray,\\n\",\n    \"    model: np.ndarray,\\n\",\n    \"    ratio_weight: str,  # TODO: this should be in the settings\\n\",\n    \"    outlier_removal: bool,\\n\",\n    \"    alpha: float,\\n\",\n    \"    outlier_std: float,\\n\",\n    \"    solar_cap: float = 3.0,  # TODO: Add to cg settings\\n\",\n    \"    model_unc_type=None,  # [\\\"CI\\\", None] TODO: Delete once the model_uncertainty below is being used\\n\",\n    \"    model_uncertainty: (\\n\",\n    \"        np.ndarray | None\\n\",\n    \"    ) = None,  # TODO: pass in the model RMSE values here.\\n\",\n    \") -> tuple[float, float, list[int]]:\\n\",\n    \"    \\\"\\\"\\\"\\n\",\n    \"    Returns the comparison group correction factor component (used later in conjunction with uncorrected counterfactual)\\n\",\n    \"    for a single hour based on all the used CG records for a given comparison group and the metering+ settings.\\n\",\n    \"\\n\",\n    \"    Inputs:\\n\",\n    \"    obs - Observed values in a single hour for all members of the comparison group\\n\",\n    \"    model - Model values in a single hour for all members of the comparison group\\n\",\n    \"    ratio_weight - Whether to do a weighted or unweighted average calculation\\n\",\n    \"    outlier removal - Whether to remove outliers from the calculation\\n\",\n    \"    alpha - (1-CI) where CI is the confidence interval desired for the\\n\",\n    \"            uncertainty calculation\\n\",\n    \"    outlier_std: The standard deviation at which to remove outliers\\n\",\n    \"    solar_cap: The maximum absolute value of the correction factor\\n\",\n    \"               (which we want to cap for solar customers with very small\\n\",\n    \"                loads). Default: 3.0. Cannot be None.\\n\",\n    \"    model_unc_type: NOTE: model_unc_type is currently ignored (March 2023).\\n\",\n    \"                    It remains for now until we are able to pass the model RMSE\\n\",\n    \"                    in to the model_uncertainty keyword when this function is called.\\n\",\n    \"                    Then the model_unc_type keyword can be removed.\\n\",\n    \"    model_uncertainty: Numpy array containint the uncorrected model RMSE of all of the\\n\",\n    \"                       meters making up the comparison group\\n\",\n    \"\\n\",\n    \"    Returns:\\n\",\n    \"    Tuple of (cg_correction_factor, cg_corr_factor_uncertainty, list_of_used_indices)\\n\",\n    \"    IMPORTANT\\n\",\n    \"    the returned used index array does not contain True/False values for which indices are used\\n\",\n    \"    but rather ONLY the indices of values that were used.\\n\",\n    \"\\n\",\n    \"    \\\"\\\"\\\"\\n\",\n    \"    # Component cap is subtracted by 1 because the component is oriented around zero, not 1.\\n\",\n    \"    component_cap = solar_cap - 1\\n\",\n    \"\\n\",\n    \"    Choice.RatioWeight.raise_error_if_value_not_in_choices_or_return(ratio_weight)\\n\",\n    \"    if model_uncertainty is None:\\n\",\n    \"        model_uncertainty = 0\\n\",\n    \"    # check same length\\n\",\n    \"    if np.shape(obs) != np.shape(model):\\n\",\n    \"        raise Exception(\\\"obs and model must be same length in 'get_cg_ratio_beam'\\\")\\n\",\n    \"\\n\",\n    \"    # drop nonfinite rows\\n\",\n    \"    idx_finite = idx_used = np.argwhere(\\n\",\n    \"        np.isfinite(obs) & np.isfinite(model) & (obs != None) & (model != None)\\n\",\n    \"    ).flatten()\\n\",\n    \"\\n\",\n    \"    obs = obs[idx_finite]\\n\",\n    \"    model = model[idx_finite]\\n\",\n    \"    cg_correction_factor_component = (obs - model) / np.abs(model)\\n\",\n    \"\\n\",\n    \"    length_err_value = _cg_correction_factor_length_check(\\n\",\n    \"        cg_correction_factor_component=cg_correction_factor_component,\\n\",\n    \"        index_valid=idx_used,\\n\",\n    \"        component_cap=component_cap,\\n\",\n    \"        model=model,\\n\",\n    \"    )\\n\",\n    \"\\n\",\n    \"    if length_err_value is not None:\\n\",\n    \"        return length_err_value\\n\",\n    \"\\n\",\n    \"    # TODO: add in weighting by treatment group?\\n\",\n    \"    if outlier_removal:\\n\",\n    \"        idx_valid = apply_cg_ratio_outlier_rejection(\\n\",\n    \"            cg_correction_factor_component, outlier_std\\n\",\n    \"        )\\n\",\n    \"        cg_correction_factor_component = cg_correction_factor_component[idx_valid]\\n\",\n    \"        obs = obs[idx_valid]\\n\",\n    \"        model = model[idx_valid]\\n\",\n    \"        idx_used = idx_used[idx_valid]\\n\",\n    \"\\n\",\n    \"    length_err_value = _cg_correction_factor_length_check(\\n\",\n    \"        cg_correction_factor_component=cg_correction_factor_component,\\n\",\n    \"        index_valid=idx_used,\\n\",\n    \"        component_cap=component_cap,\\n\",\n    \"        model=model,\\n\",\n    \"    )\\n\",\n    \"\\n\",\n    \"    if length_err_value is not None:\\n\",\n    \"        err_cf, err_unc, _ = length_err_value\\n\",\n    \"        return err_cf, err_unc, list(idx_used.astype(int))\\n\",\n    \"\\n\",\n    \"    # type hinter is saying all these are possibly unbounded further below.\\n\",\n    \"    # setting them here and checking and raising error later to avoid static analysis errors\\n\",\n    \"    solar_cap_applied = False\\n\",\n    \"    if ratio_weight == Choice.RatioWeight.WEIGHT_BY_USAGE_MAGNITUDE:\\n\",\n    \"        # ask travis about weight here, what happens with negative reads?\\n\",\n    \"        weight = np.abs(model) / np.sum(np.abs(model))\\n\",\n    \"\\n\",\n    \"        cg_correction_factor_component_mean = np.average(\\n\",\n    \"            cg_correction_factor_component, weights=weight\\n\",\n    \"        )\\n\",\n    \"        # With the weights as defined in the weighted case above, the previous line is\\n\",\n    \"        # equivalent to cg_correction_factor_mean = np.mean(obs)/np.mean(model).\\n\",\n    \"\\n\",\n    \"        # Apply the solar cap to the weighted average in places\\n\",\n    \"        # where the average of the model is \\\"small\\\" and there\\n\",\n    \"        # is a risk of a catastrophic blowup.\\n\",\n    \"\\n\",\n    \"        # TODO: figure out if solar cap still needed and if so where to apply\\n\",\n    \"        if np.abs(np.mean(model)) < _CAP_MODEL_THRESHOLD:\\n\",\n    \"            unclipped_mean = cg_correction_factor_component_mean\\n\",\n    \"            cg_correction_factor_component_mean = np.clip(\\n\",\n    \"                cg_correction_factor_component_mean, -component_cap, component_cap\\n\",\n    \"            )\\n\",\n    \"            solar_cap_applied = unclipped_mean != cg_correction_factor_component_mean\\n\",\n    \"\\n\",\n    \"        # Kish's effective sample size, weights normalized https://doi.org/10.1002/bimj.19680100122\\n\",\n    \"        n = 1 / np.sum(np.power(weight, 2))\\n\",\n    \"\\n\",\n    \"        if n < 2:\\n\",\n    \"            return (\\n\",\n    \"                cg_correction_factor_component_mean,\\n\",\n    \"                np.nan,\\n\",\n    \"                list(idx_used.astype(int)),\\n\",\n    \"            )\\n\",\n    \"\\n\",\n    \"    elif ratio_weight == Choice.RatioWeight.NO_WEIGHT:\\n\",\n    \"        weight = None\\n\",\n    \"        n = len(cg_correction_factor_component)\\n\",\n    \"\\n\",\n    \"        # Clip the individual correction factors at +/- the cap\\n\",\n    \"        # before taking the unweighted average\\n\",\n    \"        clipped = np.clip(cg_correction_factor_component, -component_cap, component_cap)\\n\",\n    \"        # Apply the cap where the model is \\\"small\\\"\\n\",\n    \"        # and there is a risk of having it blow up catastrophically.\\n\",\n    \"\\n\",\n    \"        # TODO: figure out if solar cap still needed and if so where to apply\\n\",\n    \"        # solar_cap_applied = list(clipped) != list(cg_correction_factor_component)\\n\",\n    \"        cg_correction_factor_component_mean = np.average(\\n\",\n    \"            np.where(\\n\",\n    \"                np.abs(model) < _CAP_MODEL_THRESHOLD,\\n\",\n    \"                clipped,\\n\",\n    \"                cg_correction_factor_component,\\n\",\n    \"            )\\n\",\n    \"        )\\n\",\n    \"        solar_cap_applied = cg_correction_factor_component_mean != np.average(\\n\",\n    \"            cg_correction_factor_component\\n\",\n    \"        )\\n\",\n    \"\\n\",\n    \"    else:\\n\",\n    \"        raise ValueError(\\n\",\n    \"            f\\\"ratio_weight {ratio_weight} not implemented in correction factor calc\\\"\\n\",\n    \"        )\\n\",\n    \"\\n\",\n    \"    # Calculate the uncertainty of the correction factor\\n\",\n    \"    # Note that this calculation only occurs over nonnegative weights.\\n\",\n    \"    # because the calculation isn't sensible for negative weights.\\n\",\n    \"    cg_correction_factor_mean_unc = np.nan\\n\",\n    \"    if not solar_cap_applied:\\n\",\n    \"        cg_correction_factor_std = fast_std(\\n\",\n    \"            cg_correction_factor_component,\\n\",\n    \"            mean=cg_correction_factor_component_mean,\\n\",\n    \"            weights=weight,\\n\",\n    \"        )\\n\",\n    \"        cg_correction_factor_mean_unc = np.sqrt(\\n\",\n    \"            np.average(\\n\",\n    \"                (cg_correction_factor_component * model_uncertainty / model) ** 2,\\n\",\n    \"                weights=weight,\\n\",\n    \"            )\\n\",\n    \"            + cg_correction_factor_std**2\\n\",\n    \"        ) * unc_factor(n, interval=\\\"CI\\\", alpha=alpha)\\n\",\n    \"\\n\",\n    \"    return (\\n\",\n    \"        cg_correction_factor_component_mean,\\n\",\n    \"        cg_correction_factor_mean_unc,\\n\",\n    \"        list(idx_used.astype(int)),\\n\",\n    \"    )\"\n   ]\n  }\n ],\n \"metadata\": {\n  \"kernelspec\": {\n   \"display_name\": \"Python 3\",\n   \"language\": \"python\",\n   \"name\": \"python3\"\n  },\n  \"language_info\": {\n   \"codemirror_mode\": {\n    \"name\": \"ipython\",\n    \"version\": 3\n   },\n   \"file_extension\": \".py\",\n   \"mimetype\": \"text/x-python\",\n   \"name\": \"python\",\n   \"nbconvert_exporter\": \"python\",\n   \"pygments_lexer\": \"ipython3\",\n   \"version\": \"3.11.4\"\n  }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n"
  },
  {
    "path": "opendsm/comparison_groups/savings/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\nfrom typing import Optional\n\nimport pydantic\n\nfrom opendsm.common.base_settings import BaseSettings\n\n\n\nclass TransformChoice(str, Enum):\n    STANDARDIZE = \"standardize\"\n    BISYMLOG = \"bisymlog\"\n    SCIPY_YJ = \"scipy_yj\"\n    ROBUST_SCIPY_YJ = \"robust_scipy_yj\"\n    ROBUST_YJ = \"robust_yj\"\n\n\nclass OutlierRejectionSettings(BaseSettings):\n    \"\"\"Settings for outlier rejection\"\"\"\n\n    enabled: bool = pydantic.Field(\n        default=False,\n        description=\"enables outlier rejection\"\n    )\n\n    transform: Optional[TransformChoice] = pydantic.Field(\n        default=None,\n        description=\"transformation to apply prior to outlier removal\"\n    )\n\n    std_threshold: float = pydantic.Field(\n        default = 3.0,\n        gt=0.0,\n        description=\"number of standard deviations at which outliers are defined\"\n    )\n\n    quantile: float = pydantic.Field(\n        default=0.25,\n        gt=0.0,\n        lt=0.5,\n        description=\"quantile to use for iqr outlier detection\"\n    )\n\n\nclass CorrectionCapChoice(str, Enum):\n    GLOBAL = \"global\"\n    SOLAR = \"solar\"\n\n\nclass CorrectionCapSettings(BaseSettings):\n    \"\"\"Settings for correction cap\"\"\"\n\n    enabled: bool = pydantic.Field(\n        default=True,\n        description=\"enables correction cap\"\n    )   \n\n    type: CorrectionCapChoice = pydantic.Field(\n        default=CorrectionCapChoice.SOLAR,\n        description=\"what kind of correction cap to apply\"\n    )\n\n    value: float = pydantic.Field(\n        default=3.0,\n        description=\"maximum correction as a percentage of the treatment model value\"\n    )\n\n    solar_threshold: Optional[float] = pydantic.Field(\n        default = 1/3,\n        description=\"threshold below which the cap applies for solar\"\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_solar_cap(self):\n        if self.enabled and self.type == CorrectionCapChoice.SOLAR:\n            if self.solar_threshold is None:\n                raise ValueError(\n                    \"'solar_threshold' must be specified if 'type' is 'solar'.\"\n                )\n        elif self.enabled and self.type == CorrectionCapChoice.GLOBAL:\n            if self.solar_threshold is not None:\n                raise ValueError(\n                    \"'solar_threshold' should not be specified if 'type' is 'global'.\"\n                )\n\n        return self\n\n\nclass CorrectionAlgorithm(str, Enum):\n    ODID = \"ordinary_difference_in_differences\"\n    PCTDID = \"percent_difference_in_differences\"\n    ABSPCTDID = \"absolute_percent_difference_in_differences\"\n\n\nclass WeightClusterAggChoice(str, Enum):\n    MODEL = \"model_magnitude\"\n\n\nclass CGCorrectionSettings(BaseSettings):\n    \"\"\"Settings for model correction\"\"\"\n    \n    algorithm: Optional[CorrectionAlgorithm] = pydantic.Field(\n        default=CorrectionAlgorithm.ABSPCTDID,\n        description=\"algorithm to correct treatment meter using comparison group\"\n    )\n\n    weight_cluster_aggregation: Optional[WeightClusterAggChoice] = pydantic.Field(\n        default = None,\n        description=\"how to weight cluster aggregation\"\n    )\n\n    outlier_rejection: OutlierRejectionSettings = pydantic.Field(\n        default_factory=OutlierRejectionSettings,\n        description=\"outlier rejection settings\"\n    )\n\n    correction_cap: CorrectionCapSettings = pydantic.Field(\n        default_factory=CorrectionCapSettings,\n        description=\"correction cap settings\"\n    )\n\n    alpha: float = pydantic.Field(\n        default=0.10,\n        gt=0.0,\n        lt=1.0,\n        description=\"significance level for uncertainty calculations\"\n    )\n    \n\nif __name__ == \"__main__\":\n    s = CGCorrectionSettings()\n\n    print(s.model_dump_json())\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.comparison_groups.stratified_sampling.create_comparison_groups import Stratified_Sampling\nfrom opendsm.comparison_groups.stratified_sampling.settings import (\n    StratifiedSamplingSettings as SS_Settings, \n    DistanceStratifiedSamplingSettings as DSS_Settings,\n)"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/bin_selection.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport copy\nimport itertools\nimport logging\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport numpy as np\nfrom . import equivalence\n\nlogger = logging.getLogger(__name__)\n\n__all__ = (\"StratifiedSamplingBinSelector\",)\n\n\nclass StratifiedSamplingBinSelector(object):\n    def __init__(\n        self,\n        model,\n        df_treatment,\n        df_pool,\n        equivalence_feature_ids,\n        equivalence_feature_matrix,\n        equivalence_method=\"chisquare\",\n        df_id_col=\"id\",\n        n_samples_approx=5000,\n        min_n_treatment_per_bin=0,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=0.25,\n        min_n_bins=1,\n        max_n_bins=8,\n        equivalence_quantile_size=25,\n        relax_n_samples_approx_constraint=True,\n    ):\n        \"\"\"\n        Finds an optimal stratified sampling bin configuration which minimizes\n        distance between treatmnt and comparison groups.  A bin configuration\n        is a number of bins, `n_c`, for each stratification column `c`, where\n        c is an integer between `min_n_bins` and `max_n_bin` inclusive.\n        Using a grid search, all possible bin configurations will be constructed and\n        tested, and the confiuration which minimizes treatment-comparison distance\n        will be returned.  Distance is measured on a set of features\n        provided in `df_for_equivalence` in 'long' format, i.e. multiple rows\n        per meter, one column holds feature name, one column holds feature value.\n\n        Distance is computed as follows. First cut treatment and comparison groups\n        into quantiles, with the number of quantiles chosen such that the\n        treatment group has quantiles of size `equivalence_quantile_size`.\n        Then compute the distance between each treatment-comparison quantile pair\n        according to the method in `equivalence_method`, either `euclidean` or\n        `chisquare` distance; then sum the distances across all quantiles.  For\n        example, if the treatment group is size 1000 and `equivalence_quantile_size`\n        is 100, then treatment and comparison groups will each be cut into ten quantiles,\n        and ten distances will be computed and summed.\n\n        Example usage:\n\n            m = StratifiedSampling()\n            m.add_column('annual_usage', min_value=0, max_value=20000)\n            m.add_column('summer_usage', min_value=0, max_value=1000)\n            s = StratifiedSamplingBinSelector(m, df_treatment, df_pool,\n                equivalence_feature_ids, equivalence_feature_matrix, equivalence_method=\"chisquare\")\n            results = s.results_as_json()\n            df_comparison = m.data_sample.df\n\n\n        Attributes\n        ==========\n        model: eemeter.gridmeter.StratifiedSampling\n            Model with stratification columns added.\n        df_treatment: pandas.DataFrame\n            dataframe to use for constructing the stratified sampling bins.\n        df_pool: pandas.DataFrame\n            dataframe to sample from according to the constructed stratified sampling bins.\n        df_for_equivalence: pandas.DataFrame\n            dataframe with featues to use for computing equivalence, in 'long' form\n        equivalence_feature_ids: str\n            Array of meter IDs which maps to row indices in equivalence_feature_matrix.\n        equivalence_feature_matrix: pandas.DataFrame or numpy.ndarray\n            dataframe or array with featues to use for computing equivalence, in 'wide' form,\n            i.e. one row per meter, one column per feature.  Must contain only\n            numeric values, no ID column.\n        equivalence_method: str\n            Method for computing distance -- either 'euclidean' or 'chisquare'.\n        df_id_col: str\n            Name of column in df_treatment and df_pool which contains meter ID.\n        n_samples_aprox: int\n            approximate number of total samples from df_pool which are used to construct\n            the comparison group. It is approximate because\n            there may be some slight discrepencies around the total count to ensure\n            that each bin has the correct percentage of the total.\n            A None value means that it will take as many samples as it has available.\n        min_n_treatment_per_bin: int\n            Minimum number of treatment samples that must exist in a given bin for\n            it to be considered a non-outlier bin (only applicable if there are\n            cols with fixed_width=True)\n        min_n_sampled_to_n_treatment_ratio: int\n            Minimum number samples that must exist in each bin per treatment datapoint in that bin.\n        min_n_bins: int\n            Minimum number of bins to use in stratified sampling.\n        max_n_bins: int\n            Maximum number of bins to use in stratified sampling.\n        equivalence_quantile_size: int\n            Number of samples per quantile when computing distances quantile-by-quantile.\n        relax_n_samples_approx_constraint: bool\n            If True, treats n_samples_approx as an upper bound, but gets as many comparison group\n            meters as available up to n_samples_approx. If False, it raises an exception\n            if there are not enough comparison pool meters to reach n_samples_approx.\n        \"\"\"\n        # Settings\n        self.n_samples_approx = n_samples_approx\n        self.min_n_treatment_per_bin = min_n_treatment_per_bin\n        self.random_seed = random_seed\n        self.min_n_sampled_to_n_treatment_ratio = min_n_sampled_to_n_treatment_ratio\n        self.min_n_bins = min_n_bins\n        self.max_n_bins = max_n_bins\n        self.df_id_col = df_id_col\n        self.equivalence_feature_ids = equivalence_feature_ids\n        self.equivalence_feature_matrix = equivalence_feature_matrix\n        self.equivalence_method = equivalence_method\n        self.equivalence_quantile_size = equivalence_quantile_size\n\n        self.model = model\n        self.df_treatment = df_treatment\n        self.df_pool = df_pool\n        self.n_bin_options_df = None\n        self.equiv_treatment = None\n        self.equiv_samples = []\n\n        if len(self.model.columns) == 0:\n            raise ValueError(\"You must add at least one column before fitting.\")\n        if any([not col[\"auto_bin\"] for name, col in self.model.columns.items()]):\n            raise ValueError(\"This form of fitting only works n_bins is not set\")\n        logger.debug(self.model.columns)\n        min_distance = float(\"Inf\")\n        min_columns = None\n\n        column_names = list(self.model.columns.keys())\n        n_bin_results = []\n        self.n_bin_options_df = pd.DataFrame(\n            [\n                {column_names[i - 1]: c for i, c in enumerate(comb)}\n                for comb in itertools.product(\n                    range(min_n_bins, max_n_bins + 1), repeat=len(column_names)\n                )\n            ]\n        )\n        disqualified_n_bin_options = []\n        for n_bin_option in self.n_bin_options_df.to_dict(\"records\"):\n            [\n                self.model.set_n_bins(name, n_bins)\n                for name, n_bins in n_bin_option.items()\n            ]\n            bins_selected_str = self.model.get_all_n_bins_as_str()\n\n            if n_bin_option in disqualified_n_bin_options:\n                logger.debug(f\"Skipping {bins_selected_str} (disqualified)\")\n                continue\n\n            self.model.fit(\n                self.df_treatment,\n                min_n_treatment_per_bin=min_n_treatment_per_bin,\n                random_seed=random_seed,\n            )\n\n            self.model.sample(\n                self.df_pool,\n                n_samples_approx=n_samples_approx,\n                random_seed=random_seed,\n                relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n            )\n            n_sampled_to_n_treatment_ratio = (\n                self.model.diagnostics().n_sampled_to_n_treatment_ratio()\n            )\n            if (\n                not self.model.relax_ratio_constraint\n                and n_sampled_to_n_treatment_ratio < min_n_sampled_to_n_treatment_ratio\n            ):\n                logger.info(\n                    f\"Insufficient pool data for {bins_selected_str}:\"\n                    f\"found {n_sampled_to_n_treatment_ratio}:1 but need \"\n                    f\"{min_n_sampled_to_n_treatment_ratio}:1.\"\n                )\n                disqualified_options = self.n_bin_options_df.loc[\n                    (\n                        self.n_bin_options_df[list(n_bin_option)]\n                        >= pd.Series(n_bin_option)\n                    ).all(axis=1)\n                ].to_dict(\"records\")\n                disqualified_n_bin_options.extend(disqualified_options)\n                n_bin_results.append(\n                    dict(\n                        **n_bin_option,\n                        **{\n                            \"distance\": None,\n                            \"status\": \"FAILED\",\n                            \"bins_selected_str\": bins_selected_str,\n                        },\n                    )\n                )\n                continue\n\n            # todo set up equivalence_feature_matrix and equivalence_feature_ids\n\n            treatment_ids = self.model.data_treatment.df[df_id_col].unique()\n            comparison_ids = self.model.data_sample.df[df_id_col].unique()\n            if len(treatment_ids) != len(pd.Series(treatment_ids).unique()):\n                raise ValueError(\"Duplicate IDs found in treatment group.\")\n            if len(comparison_ids) != len(pd.Series(comparison_ids).unique()):\n                raise ValueError(\"Duplicate IDs found in comparison group.\")\n\n            ix_x = equivalence.ids_to_index(treatment_ids, equivalence_feature_ids)\n            ix_y = equivalence.ids_to_index(comparison_ids, equivalence_feature_ids)\n\n            (\n                equiv_treatment,\n                equiv_sample,\n                equivalence_distance,\n            ) = equivalence.Equivalence(\n                ix_x,\n                ix_y,\n                equivalence_feature_matrix,\n                n_quantiles=equivalence_quantile_size,\n                how=equivalence_method,\n            ).compute()\n\n            n_bin_results.append(\n                dict(\n                    **n_bin_option,\n                    **{\n                        \"distance\": equivalence_distance,\n                        \"status\": \"SUCCEEDED\",\n                        \"bins_selected_str\": bins_selected_str,\n                    },\n                )\n            )\n\n            # build a dataframe with the equivalence vectors so we can plot them\n            equiv_sample[\"bin_str\"] = bins_selected_str\n            self.equiv_samples.append(equiv_sample.copy(deep=True))\n\n            logging.info(\n                f\"Computing bins: {bins_selected_str} distance: \"\n                f\"{equivalence_distance:.2f}, \"\n                #  f\"pct: {100*equivalence_distance/sum(equiv_treatment[equivalence_value_col]):.2f}\"\n            )\n            if equivalence_distance < min_distance:\n                min_distance = equivalence_distance\n                min_columns = copy.deepcopy(self.model.columns)\n\n        self.n_bin_results = pd.DataFrame(n_bin_results)\n        if not min_columns:\n            raise ValueError(\"No valid bin configurations were discovered\")\n\n        # same for all of them anyway\n        # TODO (ssuffian): Calculate this cleaner\n        equiv_treatment.name = self.model.treatment_label\n        self.equiv_treatment = equiv_treatment\n\n        self.model.columns = min_columns\n        bins_selected_str = self.model.get_all_n_bins_as_str()\n        logging.info(\n            f\"Selected bin: {bins_selected_str} distance: \"\n            f\"{min_distance:.2f}, \"\n            # f\"pct: {100*min_distance/sum(equiv_treatment[equivalence_value_col]):.2f}, \"\n            f\"random_seed: {random_seed}\"\n        )\n        self.model.fit(\n            self.df_treatment,\n            min_n_treatment_per_bin=min_n_treatment_per_bin,\n            random_seed=random_seed,\n        )\n        # if n_samples_approx is None, use the maximum available.\n        self.model.sample(\n            self.df_pool,\n            n_samples_approx=n_samples_approx,\n            random_seed=random_seed,\n            relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n        )\n        self.n_samples_approx = n_samples_approx\n\n        # get averages that can be accessed later\n        self.equiv_treatment_avg = self.equiv_treatment.groupby(\"feature_index\")[\n            \"value\"\n        ].mean()\n        self.equiv_treatment_avg = self.equiv_treatment_avg.rename(\"treatment\")\n\n        self.equiv_pool_avg = (\n            pd.DataFrame(equivalence_feature_matrix)\n            .mean()\n            .to_frame()\n            .rename(columns={0: \"comparison pool\"})\n            .reset_index(drop=True)\n        )\n\n        self.equiv_samples_avg = (\n            pd.concat(self.equiv_samples)\n            .groupby([\"bin_str\", \"feature_index\"])[\"value\"]\n            .mean()\n            .reset_index()\n            .pivot(index=\"feature_index\", columns=\"bin_str\", values=\"value\")\n        )\n        self.bins_selected_str = self.model.get_all_n_bins_as_str()\n\n        # get distances for comparison pool\n        treatment_ids = self.model.data_treatment.df[df_id_col].unique()\n        comparison_pool_ids = self.model.data_pool.df[df_id_col].unique()\n        ix_x = equivalence.ids_to_index(treatment_ids, equivalence_feature_ids)\n        ix_y = equivalence.ids_to_index(comparison_pool_ids, equivalence_feature_ids)\n        equiv_treatment, equiv_pool, equivalence_distance = equivalence.Equivalence(\n            ix_x,\n            ix_y,\n            equivalence_feature_matrix,\n            n_quantiles=equivalence_quantile_size,\n            how=equivalence_method,\n        ).compute()\n        self.equiv_pool = equiv_pool\n\n    def kwargs_as_json(self):\n        return {\n            \"equivalence_method\": self.equivalence_method,\n            \"n_samples_approx\": self.n_samples_approx,\n            \"min_n_treatment_per_bin\": self.min_n_treatment_per_bin,\n            \"random_seed\": self.random_seed,\n            \"min_n_sampled_to_n_treatment_ratio\": self.min_n_sampled_to_n_treatment_ratio,\n            \"min_n_bins\": self.min_n_bins,\n            \"max_n_bins\": self.max_n_bins,\n            \"equivalence_quantile_size\": self.equivalence_quantile_size,\n        }\n\n    def results_as_json(self):\n        equiv_samples_df = pd.concat(self.equiv_samples)\n        selected_sample_df = equiv_samples_df[\n            equiv_samples_df[\"bin_str\"] == self.bins_selected_str\n        ]\n\n        return {\n            \"bins_selected\": self.bins_selected_str,\n            \"random_seed\": self.random_seed,\n            \"n_bin_results\": self.n_bin_results.to_dict(\"records\"),\n            \"chisquare_averages\": {\n                \"selected_sample\": selected_sample_df.to_dict(\"records\"),\n                self.model.treatment_label: self.equiv_treatment.to_dict(\"records\"),\n                \"comparison_pool\": self.equiv_pool.to_dict(\"records\"),\n            },\n            \"averages\": {\n                \"samples\": self.equiv_samples_avg.reset_index().to_dict(\"records\"),\n                \"selected_sample\": self.equiv_samples_avg[self.bins_selected_str]\n                .reset_index()\n                .to_dict(\"records\"),\n                self.model.treatment_label: self.equiv_treatment_avg.reset_index().to_dict(\n                    \"records\"\n                ),\n                self.model.pool_label: self.equiv_pool_avg.reset_index().to_dict(\n                    \"records\"\n                ),\n            },\n        }\n\n    def plot_records_based_equiv_average(self, plot=True):\n        equiv_df = pd.concat(\n            [self.equiv_treatment_avg, self.equiv_pool_avg, self.equiv_samples_avg],\n            axis=1,\n        )\n\n        wrong_models = [\n            m for m in self.equiv_samples_avg.columns if m != self.bins_selected_str\n        ]\n\n        if plot:\n            fig, ax = plt.subplots()\n            for wm in wrong_models:\n                plt.plot(self.equiv_samples_avg[wm], alpha=0.1, color=\"b\")\n            equiv_df[[self.bins_selected_str, \"treatment\", \"comparison pool\"]].plot(\n                color=[\"k\", \"r\", \"k\"], style=[\"-\", \"-\", \".\"], ax=ax\n            )\n            plt.legend(loc=\"center left\", bbox_to_anchor=(1.0, 0.5))\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/bins.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport itertools\nfrom operator import and_\nfrom functools import reduce\n\n__all__ = (\"BinnedData\", \"Binning\", \"Bin\", \"MultiBin\")\n\n\nclass ModelSamplingException(Exception):\n    pass\n\n\nclass BinnedData:\n    def __init__(self, df, binning, min_n_treatment_per_bin=0):\n        self.binning = binning\n        self.df = self._map_bins(df)\n        self.min_n_treatment_per_bin = min_n_treatment_per_bin\n        self.outlier_bins = self._outlier_bins()\n        self._flag_outliers()\n\n    def _map_bins(self, df):\n        \"\"\"Add '_bin' column to df indicating which bin each row maps to.\"\"\"\n        df.loc[:, \"_bin\"] = None\n\n        for b in self.binning.multibins:\n            df.loc[b.filter_expr()(df), \"_bin\"] = b\n            df.loc[b.filter_expr()(df), \"_bin_label\"] = b.label\n        return df\n\n    def count_bins_1d(self, column):\n        \"\"\"Count number of elements within each 1-dimensional bin associated with\n        column.\"\"\"\n        bins = self.binning.bins[column]\n        df = pd.DataFrame(\n            [\n                {\n                    \"column\": column,\n                    \"index\": b.index,\n                    \"min\": b.min,\n                    \"max\": b.max,\n                    \"n\": len(self.df[b.filter_expr(self.df)]),\n                }\n                for b in bins\n            ]\n        )\n        df[\"n_pct\"] = df[\"n\"] / df[\"n\"].sum()\n        return df\n\n    def count_bins(self, skip_outliers=False):\n        \"\"\"Count number of elements within each multi-dimensional bin.\"\"\"\n        df = self.df\n        if skip_outliers:\n            df = df[~df._outlier_bin & ~df._outlier_value]\n        df = (\n            df._bin.value_counts()\n            .reset_index()\n            .rename(columns={\"_bin\": \"bin\", \"count\": \"n\"})\n        )\n        df[\"n_pct\"] = df[\"n\"] / df[\"n\"].sum()\n        return df\n\n    def _outlier_bins(self):\n        df_bins = (\n            self.df._bin.value_counts()\n            .reset_index()\n            .rename(columns={\"_bin\": \"bin\", \"count\": \"n\"})\n        )\n        df_bins[\"outlier\"] = df_bins[\"n\"] < self.min_n_treatment_per_bin\n        return df_bins\n\n    def _flag_outliers(self):\n        \"\"\"Flag elements that fall in bins that are too small.\"\"\"\n        df = self.outlier_bins\n        self.df.loc[:, \"_outlier_bin\"] = self.df._bin.isin(\n            df[df[\"outlier\"]][\"bin\"].values\n        )\n\n\nclass Binning(object):\n    \"\"\"Contains list of multidimensional bins\"\"\"\n\n    def __init__(self):\n        self.bins = {}  # 1-dimensional bins for each column\n        self.edges_1d = {}  # array of bin edges\n        self.multibins = []  # list of n-dimensional bin\n\n    def edges(self):\n        return pd.concat([b.edges() for b in self.multibins])\n\n    def edges_xy(self, col_x, col_y):\n        df = self.edges()\n        df_x = df[df.column == col_x]\n        df_x = df_x.rename(\n            columns={\"column\": \"column_x\", \"min\": \"x_min\", \"max\": \"x_max\"}\n        )\n        df_y = df[df.column == col_y]\n        df_y = df_y.rename(\n            columns={\"column\": \"column_y\", \"min\": \"y_min\", \"max\": \"y_max\"}\n        )\n        return df_x.merge(df_y)\n\n    def bin(self, values, column_name, n_bins, fixed_width):\n        \"\"\"Generate and store  1-dimensional binning for the specific column\"\"\"\n        if fixed_width:\n            bins, edges = pd.cut(values, n_bins, retbins=True, duplicates=\"drop\")\n        else:\n            bins, edges = pd.qcut(values, q=n_bins, retbins=True, duplicates=\"drop\")\n\n        if len(edges) < n_bins + 1:\n            raise ValueError(\n                f\"Duplicate bins were created for {column_name} -- this usually occurs if a large number of data points have the same value, i.e. zero. Try using fewer bins. Set n_bins to 1 and run model.diagnostics() to view data.  \\nStats: \\n{values.describe()}\"\n            )\n        self._add_column(column=column_name, edges=edges)\n\n    def _add_column(self, column, edges):\n        \"\"\"Add a new 1-demsnsional bin to internal data structure.\"\"\"\n        this_bins = []\n        for i in range(len(edges) - 1):\n            this_bins.append(Bin(column, edges[i], edges[i + 1], i))\n        self.bins[column] = this_bins\n        self.edges_1d[column] = edges\n        self._update_multibins()\n        return self\n\n    def _update_multibins(self):\n        \"\"\"Update internal data structure.\"\"\"\n        bins = [b for column, b in self.bins.items()]\n        self.multibins = [MultiBin(b) for b in itertools.product(*bins)]\n\n\nclass Bin:\n    \"\"\"Single-dimensional bin\"\"\"\n\n    def __init__(self, column, min, max, index):\n        self.column = column\n        self.min = min\n        self.max = max\n        self.index = index\n\n    def filter_expr(self):\n        \"\"\"Make  a function that filters a dataframe to keep only values witihn this\n        bin.\"\"\"\n        return lambda df: (df[self.column] >= self.min) & (df[self.column] <= self.max)\n\n    def __str__(self):\n        return f\"Bin: {self.column} {self.index} - [{self.min}, {self.max})\"\n\n    def __repr__(self):\n        return str(self)\n\n\nclass MultiBin:\n    \"\"\"Multi-dimensional bin -- intersection of n Bins\"\"\"\n\n    def __init__(self, bins):\n        self.bins = bins\n        self.label = \"__\".join(\n            [f\"{b.column}_{str(b.index).zfill(3)}\" for b in self.bins]\n        )\n\n    def filter_expr(self):\n        \"\"\"Make  a function that filters a dataframe to keep only values witihn\n        each dimension of this bin.\"\"\"\n        return lambda df: reduce(and_, [(b.filter_expr()(df)) for b in self.bins])\n\n    def get_max_n_target(self, df):\n        return len(df[self.filter_expr()(df)])\n\n    def sample(self, df, n_target, min_n_treatment_per_bin, random_seed=1):\n        \"\"\"Sample n_target elements from dataframe df that fall\n        within each dimension of this bin.\"\"\"\n\n        d1 = df[self.filter_expr()(df)]\n\n        if n_target < min_n_treatment_per_bin:\n            raise ModelSamplingException(\n                f\"Bin {self} has target of {n_target} control meters which is less than minimum of {min_n_treatment_per_bin}.  Try increasing n_outputs. \\n\\nBins: {chr(10).join([str(b) for b in self.bins])}\"\n            )\n\n        if len(d1) < n_target:\n            raise ModelSamplingException(\n                f\"Bin {self} has target of {n_target} control meters, but only {len(d1)} available.  Try reducing n_outputs or decreasing number of bins.  Run diagnostics.scatter_2d(), diagnostics.quantile_plot(), or diagnostics.histogram() to visualize data. \\n\\nBins: {chr(10).join([str(b) for b in self.bins])}\"\n            )\n        return d1.sample(n_target, replace=False, random_state=random_seed)\n\n    def edges(self):\n        return pd.DataFrame(\n            [\n                {\n                    \"bin\": self,\n                    \"label\": self.label,\n                    \"column\": b.column,\n                    \"min\": b.min,\n                    \"max\": b.max,\n                }\n                for b in self.bins\n            ]\n        )\n\n    def __str__(self):\n        return f\"MultBin: {self.label}\"\n\n    def __repr__(self):\n        return str(self)\n\n\ndef sample_bins(\n    binned_data_treatment,\n    binned_data_pool,\n    random_seed,\n    n_samples_approx,\n    counts=None,\n    skip_outliers=True,\n    relax_n_samples_approx_constraint=False,\n):\n    if not counts:\n        counts = binned_data_treatment.count_bins(skip_outliers=True)\n        counts[\"n_target\"] = np.floor(counts[\"n_pct\"] * n_samples_approx).astype(int)\n\n    if len(counts) == 0:\n        raise ValueError(\"No non-outlier treatment data remaining.\")\n    df = pd.concat(\n        [\n            row[\"bin\"].sample(\n                binned_data_pool.df,\n                n_target=row[\"n_target\"],\n                min_n_treatment_per_bin=binned_data_treatment.min_n_treatment_per_bin,\n                random_seed=random_seed,\n            )\n            for index, row in counts.iterrows()\n        ]\n    )\n    return df\n\n\ndef get_counts_and_update_n_samples_approx(\n    binned_data_treatment,\n    binned_data_pool,\n    n_samples_approx,\n    relax_n_samples_approx_constraint,\n):\n    counts = binned_data_treatment.count_bins(skip_outliers=True)\n\n    # Scenario 1: n_samples_approx = None\n    # a way to ensure you get the max number of samples if n_samples_approx=None\n    counts[\"n_samples_available\"] = [\n        row[\"bin\"].get_max_n_target(binned_data_pool.df)\n        for index, row in counts.iterrows()\n    ]\n    max_possible_n_samples_approx = int(\n        min(counts[\"n_samples_available\"] / counts[\"n_pct\"])\n    )\n    n_samples_approx = (\n        n_samples_approx if n_samples_approx else max_possible_n_samples_approx\n    )\n    # needs to be floor to ensure rounding errors don't leave one less than exists\n    counts[\"n_target\"] = np.floor(counts[\"n_pct\"] * n_samples_approx).astype(int)\n\n    # if you want to treat n_samples_approx as a max, but get as many as you can\n    # if you can't reach that, then set relax_n_samples_approx_constraint=True\n    has_enough_for_n_samples_approx = not any(\n        counts[\"n_samples_available\"] < counts[\"n_target\"]\n    )\n    relax_ratio_constraint = False\n    if has_enough_for_n_samples_approx:\n        # Scenario 2: n_samples_approx=value so we want to ignore the ratio constraint\n        relax_ratio_constraint = True\n    elif relax_n_samples_approx_constraint:\n        # Scenario 3: n_samples_approx=value and that value but that value can not\n        # be met, so we want as many as possible and it is valid as long as it\n        # meets the ratio constraint.\n        n_samples_approx = max_possible_n_samples_approx\n        counts[\"n_target\"] = np.floor(counts[\"n_pct\"] * n_samples_approx).astype(int)\n    # else:\n    # Scenario 4: It will fail during sampling because it can not meet\n    # n_samples_approx and we did not relax that constraint.\n    return n_samples_approx, relax_ratio_constraint, counts\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/const.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\n\n\nclass DistanceMetric(str, Enum):\n    EUCLIDEAN = \"euclidean\"\n    CHISQUARE = \"chisquare\""
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/create_comparison_groups.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.common.base_comparison_group import Comparison_Group_Algorithm\n\nfrom opendsm.comparison_groups.stratified_sampling.model import StratifiedSampling\nfrom opendsm.comparison_groups.stratified_sampling.bins import ModelSamplingException\nfrom opendsm.comparison_groups.stratified_sampling.diagnostics import StratifiedSamplingDiagnostics\nfrom opendsm.comparison_groups.stratified_sampling.bin_selection import StratifiedSamplingBinSelector\n\nfrom opendsm.comparison_groups.stratified_sampling.settings import Settings\n\n\nclass Stratified_Sampling(Comparison_Group_Algorithm):\n    def __init__(self, settings: Optional[Settings] = None):\n        if settings is None:\n            settings = Settings()\n\n        self.settings = settings\n        self.df_raw = None\n\n        self.model = StratifiedSampling()\n        self.model_bin_selector = None\n\n        for settings in self.settings.stratification_column:\n            self.model.add_column(\n                settings.column_name,\n                n_bins=settings.n_bins,\n                min_value_allowed=settings.min_value_allowed,\n                max_value_allowed=settings.max_value_allowed,\n                fixed_width=settings.is_fixed_width,\n                auto_bin_require_equivalence=settings.auto_bin_equivalence,\n            )\n\n        self._diagnostics = None\n\n\n    def _create_clusters_df(self, ids):\n        clusters = pd.DataFrame(ids, columns=[\"id\"])\n        clusters[\"cluster\"] = 0\n        clusters[\"weight\"] = 1.0\n\n        clusters = clusters.reset_index().set_index(\"id\")\n        clusters = clusters[[\"cluster\", \"weight\"]]\n\n        return clusters\n\n\n    def _create_treatment_weights_df(self, ids):\n        coeffs = np.ones(len(ids))\n\n        treatment_weights = pd.DataFrame(coeffs, index=ids, columns=[\"pct_cluster_0\"])\n        treatment_weights.index.name = \"id\"\n\n        return treatment_weights\n    \n    def _create_output_dfs(self, t_ids):\n        self.df_raw = self.model.data_sample.df\n\n        # Create comparison group\n        df_cg = self.df_raw[self.df_raw[\"_outlier_bin\"] == False]\n        clusters = self._create_clusters_df(df_cg[\"meter_id\"].unique())\n\n        # Create treatment_weights\n        \n        treatment_weights = self._create_treatment_weights_df(t_ids)\n\n        # Assign dfs to self\n        self.clusters = clusters\n        self.treatment_weights = treatment_weights\n\n        return clusters, treatment_weights\n\n\n    def get_comparison_group(self, treatment_data, comparison_pool_data):\n        settings = self.settings\n\n        self.treatment_data = treatment_data\n        self.comparison_pool_data = comparison_pool_data\n\n        t_ids = treatment_data.ids\n        t_features = treatment_data.features\n        t_features = t_features.reset_index().rename(columns={\"id\": \"meter_id\"})\n\n        cp_features = comparison_pool_data.features\n        cp_features = cp_features.reset_index().rename(columns={\"id\": \"meter_id\"})\n\n        if settings.equivalence_method is None:\n            self.model.fit_and_sample(\n                t_features, \n                cp_features,\n                n_samples_approx=settings.n_samples_approx,\n                relax_n_samples_approx_constraint=settings.relax_n_samples_approx_constraint,\n                min_n_treatment_per_bin=settings.min_n_treatment_per_bin,\n                min_n_sampled_to_n_treatment_ratio=settings.min_n_sampled_to_n_treatment_ratio,\n                random_seed=settings.seed,\n            )\n        else:\n            self.treatment_ids = t_ids\n            self.treatment_loadshape = treatment_data.loadshape\n            self.comparison_pool_loadshape = comparison_pool_data.loadshape\n            t_loadshape = self.treatment_loadshape\n            cp_loadshape = self.comparison_pool_loadshape\n\n            df_equiv = pd.concat([t_loadshape, cp_loadshape])\n            df_equiv.index.name = \"meter_id\"\n\n            self.model_bin_selector = StratifiedSamplingBinSelector(\n                self.model,\n                t_features, \n                cp_features,\n                equivalence_feature_ids=df_equiv.index,\n                equivalence_feature_matrix=df_equiv,\n                df_id_col=\"meter_id\",\n                equivalence_method=settings.equivalence_method,\n                equivalence_quantile_size=settings.equivalence_quantile,\n                \n                n_samples_approx=settings.n_samples_approx,\n                relax_n_samples_approx_constraint=settings.relax_n_samples_approx_constraint,\n\n                min_n_bins=settings.min_n_bins,\n                max_n_bins=settings.max_n_bins,\n                min_n_treatment_per_bin=settings.min_n_treatment_per_bin,\n                min_n_sampled_to_n_treatment_ratio=settings.min_n_sampled_to_n_treatment_ratio,\n                random_seed=settings.seed,\n            )\n\n        clusters, treatment_weights = self._create_output_dfs(t_ids)\n\n        return clusters, treatment_weights\n\n\n    def diagnostics(self):\n        if self.df_raw is None:\n            raise RuntimeError(\"Must run get_comparison_group() before calling diagnostics()\")\n        \n        if self._diagnostics is None:\n            self._diagnostics = StratifiedSamplingDiagnostics(model=self.model)\n        \n        return self._diagnostics"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/diagnostics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nfrom itertools import combinations\nfrom scipy.stats import ttest_ind, ks_2samp\nfrom scipy.spatial.distance import pdist\nfrom scipy.stats import chisquare\nimport warnings\n\n# Flag to check if plotnine is available\nplotnine_available = True\ntry:\n    import plotnine\n    from plotnine import *\nexcept ModuleNotFoundError:\n    plotnine_available = False\n\n\ndef t_and_ks_test(x, y, thresh=0.05):\n    t_p = \"{:,.3f}\".format(ttest_ind(x, y).pvalue)\n    ks_p = \"{:,.3f}\".format(ks_2samp(x, y).pvalue)\n\n    t_p = ttest_ind(x, y).pvalue\n    ks_p = ks_2samp(x, y).pvalue\n    t_ok = t_p > thresh\n    ks_ok = ks_p > thresh\n    t_p_str = \"t pval: {:,.3f}\".format(t_p)\n    ks_p_str = \"KS pval: {:,.3f}\".format(ks_p)\n\n    return pd.Series(\n        {\n            \"ks_ok\": ks_ok,\n            \"t_ok\": t_ok,\n            \"ks_p\": ks_p_str,\n            \"t_p\": t_p_str,\n            \"t_value\": t_p,\n            \"ks_value\": ks_p,\n        }\n    )\n\n\nclass DiagnosticPlotter:\n    def quantile(self, df, df_equiv, cols=None):\n        if cols is None:\n            cols = self.default_cols\n\n        df_quantile = df[[\"population\"] + cols].melt(id_vars=[\"population\"])\n        quantile_range = np.arange(0.005, 1.0, 0.01)\n        df_quantile = (\n            df_quantile.groupby([\"population\", \"variable\"])\n            .apply(\n                lambda x: pd.DataFrame(\n                    {\n                        \"quantile\": quantile_range,\n                        \"value\": x[\"value\"].quantile(quantile_range),\n                    }\n                )\n            )\n            .reset_index()\n        )\n        \n        if plotnine_available:\n            plotnine.options.figure_size = (6, 3 * df_quantile.variable.nunique())\n\n            base_plot = (\n                ggplot(df_quantile, aes(x=\"quantile\", y=\"value\", color=\"population\"))\n                + geom_point()\n                + facet_wrap(\"~variable\", scales=\"free_y\", ncol=1)\n                + theme_bw()\n            )\n\n            df_range = (\n                df_quantile.groupby(\"variable\")\n                .apply(lambda df: pd.Series({\"min\": df.value.min(), \"max\": df.value.max()}))\n                .reset_index()\n            )\n\n            df_equiv = df_equiv.merge(df_range)\n            df_equiv[\"x\"] = 0\n            df_equiv[\"y\"] = (df_equiv[\"max\"] - df_equiv[\"min\"]) * 0.95 + df_equiv[\"min\"]\n            df_equiv[\"y2\"] = (df_equiv[\"max\"] - df_equiv[\"min\"]) * 0.8 + df_equiv[\"min\"]\n\n            p = (\n                base_plot\n                + geom_label(\n                    aes(label=\"t_p\", fill=\"t_ok\", x=\"x\", y=\"y\"),\n                    data=df_equiv,\n                    ha=\"left\",\n                    va=\"top\",\n                    color=\"black\",\n                    size=10,\n                )\n                + geom_label(\n                    aes(label=\"ks_p\", fill=\"ks_ok\", x=\"x\", y=\"y2\"),\n                    data=df_equiv,\n                    ha=\"left\",\n                    va=\"top\",\n                    color=\"black\",\n                    size=10,\n                )\n                + scale_fill_manual({True: \"lightgreen\", False: \"orange\"}, guide=None)\n                + scale_color_discrete()\n            )\n\n            return p\n        else:\n            warnings.warn(\"Plotnine is not installed. Diagnostic functionality will not be available. Use 'pip install plotnine' to address this.\")\n            return None\n\n    def scatter(self, df, cols=None):\n        if cols is None:\n            cols = self.default_cols\n        col_pairs = combinations(cols, 2)\n        plots = [self._scatter(df, p[0], p[1]) for p in col_pairs]\n        \n        return [p for p in plots]\n\n    def _scatter(self, df, col_x, col_y):\n        def sample_if_too_big(df):\n            if len(df) > 2000:\n                df = df.sample(2000)\n            return df\n\n        df = (\n            df.groupby(\"population\", group_keys=False)\n            .apply(sample_if_too_big)\n            .reset_index()\n        )\n\n        if plotnine_available:\n            plotnine.options.figure_size = (12, 5)\n            base_plot = (\n                ggplot(df, aes(x=col_x, y=col_y, color=\"population\"))\n                + geom_point()\n                + facet_wrap(\"~population\", nrow=1)\n                + theme_bw()\n            )\n            outlier_bins = self.data_treatment.outlier_bins\n            outlier_bins = outlier_bins[outlier_bins[\"outlier\"]][\"bin\"].values\n            df_rects = self.binning.edges_xy(col_x, col_y)\n            df_rects = df_rects[~df_rects[\"bin\"].isin(outlier_bins)]\n\n            # due to plotnine bug\n            df_rects[col_x] = np.nan\n            df_rects[col_y] = np.nan\n            p = base_plot + geom_rect(\n                aes(xmin=\"x_min\", xmax=\"x_max\", ymin=\"y_min\", ymax=\"y_max\"),\n                data=df_rects,\n                color=\"black\",\n                fill=None,\n                size=0.2,\n            )\n\n            return p\n        else:\n            warnings.warn(\"Plotnine is not installed. Diagnostic functionality will not be available. Use 'pip install plotnine' to address this.\")\n            return None\n\n    def histogram(self, df, cols=None):\n        if cols is None:\n            cols = self.default_cols\n\n        return [self._histogram(df, c) for c in cols]\n\n    def _histogram(self, df, col):\n        if plotnine_available:\n            plotnine.options.figure_size = (12, 5)\n            p = (\n                ggplot(df, aes(x=col, fill=\"population\"))\n                + geom_histogram(bins=30)\n                + facet_wrap(\"~population\", nrow=1, scales=\"free_y\")\n                + theme_bw()\n            )\n\n            outlier_bins = self.data_treatment.outlier_bins\n            outlier_bins = outlier_bins[outlier_bins[\"outlier\"]][\"bin\"].values\n            df_rects = self.binning.edges_xy(col, col)\n            df_rects = df_rects[~df_rects[\"bin\"].isin(outlier_bins)]\n\n            # due to plotnine bug\n            df_rects[col] = np.nan\n            p = p + geom_rect(\n                aes(xmin=\"x_min\", xmax=\"x_max\", ymin=-np.inf, ymax=np.inf),\n                data=df_rects,\n                color=\"black\",\n                fill=None,\n                size=0.2,\n            )\n\n            return p\n        else:\n            warnings.warn(\"Plotnine is not installed. Diagnostic functionality will not be available. Use 'pip install plotnine' to address this.\")\n            return None\n\n\nclass StratifiedSamplingDiagnostics(DiagnosticPlotter):\n    \"\"\"\n    Construct plots and tables summarizing results of stratified sampling.\n    Operates on a StratifiedSamplingModel.  Plots will show treatment,\n    pool, and comparison group meters on the same axes to allow for easy comparisons.\n    If fitting failed, plots will be available with treatment and pool meters only.\n\n    Methods\n    =======\n\n    scatter():\n        Construct 2-D scatter plots of all stratification columns with bins superimposed.\n\n    histogram():\n        Construct 1-D histogram plots of all stratification columns with bins superimposed.\n\n    quantile_equivalence():\n        Construct quantile plots to compare distributions; include t-test and ks-test\n        p-values.\n\n    count_bins():\n        Construct a table of pins and relative densities for treatment, pool, and comparison.\n\n\n    Attributes\n    ==========\n\n    model:\n        A StratifiedSamplingModel, after fit() or fit_and_sample() have been run.\n\n    \"\"\"\n\n    def __init__(self, model):\n        self.model = model\n        self.binning = self.model.binning\n        self.data_treatment = self.model.data_treatment\n        self.data_pool = self.model.data_pool\n        self.sampled = self.model.sampled\n\n        self.treatment_label = self.model.treatment_label\n        self.pool_label = self.model.pool_label\n        self.data_sample = self.model.data_sample\n\n        # these two will always exist\n        df_treatment = self.model.data_treatment.df\n        df_pool = self.model.data_pool.df\n        self.default_cols = self.model.col_names\n\n        self.available_equiv_labels = [\n            self.treatment_label,\n            self.pool_label,\n            \"sample\",\n        ]\n        df_sample = (\n            self.data_sample.df if self.data_sample is not None else pd.DataFrame()\n        )\n        self.labeled_dfs = [df_treatment, df_pool, df_sample]\n\n        def _concat_dfs(dfs_to_concat, concat_col, concat_values):\n            if len(dfs_to_concat) != len(concat_values):\n                raise ValueError(\n                    \"dfs_to_concat should be the same length as concat_values\"\n                )\n            return pd.concat(\n                [\n                    df.assign(**{concat_col: value})\n                    for df, value in zip(dfs_to_concat, concat_values)\n                ],\n                sort=False,\n            )\n\n        self.df_all = _concat_dfs(\n            self.labeled_dfs, \"population\", self.available_equiv_labels\n        )\n\n    def histogram(self, cols=None):\n        return super().histogram(self.df_all, cols)\n\n    def scatter(self, cols=None):\n        return super().scatter(self.df_all, cols)\n\n    def quantile_equivalence(self, cols=None):\n        df_equiv = self.equivalence(cols)\n        return super().quantile(self.df_all, df_equiv, cols=cols)\n\n    def _check_equiv_labels(self, equiv_label_x, equiv_label_y):\n        if (\n            equiv_label_x is not None\n            and equiv_label_x not in self.available_equiv_labels\n        ):\n            raise ValueError(\n                f\"equiv_label_x must be one of: {self.available_equiv_labels}\"\n            )\n        if (\n            equiv_label_y is not None\n            and equiv_label_y not in self.available_equiv_labels\n        ):\n            raise ValueError(\n                f\"equiv_label_y must be one of: {self.available_equiv_labels}\"\n            )\n        equiv_label_x = equiv_label_x if equiv_label_x else self.treatment_label\n        equiv_label_y = (\n            equiv_label_y\n            if equiv_label_y\n            else (\"sample\" if self.data_sample else self.pool_label)\n        )\n        return equiv_label_x, equiv_label_y\n\n    def equivalence(self, cols=None, equiv_label_x=None, equiv_label_y=None):\n        \"\"\"\n        Attributes\n        ----------\n        cols: str\n            Columns to plot and calculate equivalence for. Defaults to all available cols.\n        equiv_label_x: str\n            First label to measure equivalence against (defaults to treatment label)\n        equiv_label_y: str\n            Second label to measure equivalence against (defaults to sample if available,\n             otherwise defaults to full pool set)\n        \"\"\"\n        if (\n            equiv_label_x is not None\n            and equiv_label_x not in self.available_equiv_labels\n        ):\n            raise ValueError(\n                f\"equiv_label_x must be one of: {self.available_equiv_labels}\"\n            )\n        if (\n            equiv_label_y is not None\n            and equiv_label_y not in self.available_equiv_labels\n        ):\n            raise ValueError(\n                f\"equiv_label_y must be one of: {self.available_equiv_labels}\"\n            )\n        equiv_label_x = equiv_label_x if equiv_label_x else self.treatment_label\n        equiv_label_y = (\n            equiv_label_y\n            if equiv_label_y\n            else (\"sample\" if self.data_sample else self.pool_label)\n        )\n        cols = cols if cols else self.default_cols\n\n        df = self.df_all[[\"population\"] + cols].melt(id_vars=[\"population\"])\n        return (\n            df.groupby(\"variable\")\n            .apply(\n                lambda x: t_and_ks_test(\n                    x[x[\"population\"] == equiv_label_x].value.dropna(),\n                    x[x[\"population\"] == equiv_label_y].value.dropna(),\n                ), \n                include_groups=False,\n            )\n            .reset_index()\n        )\n\n    def equivalence_passed(self, cols=None):\n        df = self.equivalence(cols=cols)\n        return all(df[\"ks_ok\"]) & all(df[\"t_ok\"])\n\n    def count_bins(self):\n\n        df_treatment = self.data_treatment.count_bins(skip_outliers=True).rename(\n            columns={\n                \"n\": f\"n_{self.treatment_label}\",\n                \"n_pct\": f\"n_pct_{self.treatment_label}\",\n            }\n        )\n\n        df_pool = self.data_pool.count_bins(skip_outliers=False).rename(\n            columns={\n                \"n\": f\"n_{self.pool_label}\",\n                \"n_pct\": f\"n_pct_{self.pool_label}\",\n            }\n        )\n\n        df = df_treatment.merge(df_pool)\n\n        if self.sampled:\n            df_sample = self.data_sample.count_bins(skip_outliers=False).rename(\n                columns={\"n\": f\"n_sampled\", \"n_pct\": f\"n_pct_sampled\"}\n            )\n            df = df.merge(df_sample)\n        return df\n\n    def n_sampled_to_n_treatment_ratio(self):\n        bin_df = self.count_bins()\n        if bin_df.empty:\n            return 0\n        else:\n            return (\n                (bin_df[\"n_sampled\"] / bin_df[f\"n_{self.treatment_label}\"])\n                .min()\n                .astype(int)\n            )\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/equivalence.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\nimport numpy as np\nfrom scipy.spatial.distance import pdist\nfrom scipy.stats import chisquare\n\n\ndef ids_to_index(subset_ids, all_ids):\n    \"\"\"Convert an array of ids to an array of indexes relative to a superset of ids.\"\"\"\n    \n    df_1 = pd.DataFrame({'a': subset_ids}).reset_index()\n    df_2 = pd.DataFrame({'a': all_ids}).reset_index().rename(columns={'index': 'x'})\n\n    df_out = df_1.merge(df_2)\n    diff = len(df_1) - len(df_out)\n    if diff > 0:\n        raise ValueError(f\"{diff} IDs present in subset are missing in pool\")\n\n    return df_out.x.values\n\n\nclass Equivalence:\n    \"\"\" Computes equivalence between two sets of features, by cutting into \n    quantiles, computing distance between each quantile, and summing distances.\n\n    Parameters:\n    ------------\n\n    ix_x: List or array\n        Array of indices which map to the first row-set of features in features_matrix.\n    ix_y: List or array\n        Array of indices which map to the second row-set of features in features_matrix.\n    features_matrix: pd.DataFrame or numpy.ndarray\n        Dataframe or array of features, one row per item, one column per feature.\n    n_quantiles: int\n        Number of quantiles to cut eachset into.\n    how: str\n        Distance metric, either 'euclidean' or 'chisquare'\n\n\n    \"\"\"\n    def __init__(self, ix_x, ix_y, features_matrix, n_quantiles=1, how='euclidean'):\n        self.ix_x = ix_x\n        self.ix_y = ix_y\n        self.n_quantiles = n_quantiles\n        self.how = how\n\n        if type(features_matrix) == pd.DataFrame:\n            features_matrix = features_matrix.to_numpy() # pragma: no cover\n        elif type(features_matrix) == np.ndarray:\n            pass\n        else:\n            raise ValueError(\"features_matrix must be a pandas DataFrame or numpy ndarray.\") # pragma: no cover\n\n        self.features_matrix = features_matrix\n        self.X = self.features_matrix[ix_x].transpose()\n        self.Y = self.features_matrix[ix_y].transpose()\n\n\n    def compute(self):\n        means_x, means_y, quantiles_x, quantiles_y = quantile_means_population(self.X, self.Y, self.n_quantiles)\n        distance = sum_column_distance(means_x, means_y, how=self.how)\n        equiv_x = reshape_outputs(means_x, quantiles_x)\n        equiv_y = reshape_outputs(means_y, quantiles_y)\n        return equiv_x, equiv_y, distance\n\n\n\ndef reshape_outputs(means, quantiles):\n    out = []\n    for feature in range(len(means)):\n        for q in range(len(quantiles[0])-1):\n            bin_label = f\"[{quantiles[feature][q]}, {quantiles[feature][q+1]}]\" \n            mean = means[feature][q]\n            out.append({'_bin_label': bin_label, 'value': mean, 'feature_index': feature})\n    return pd.DataFrame(out)\n\n\n\ndef get_quantile_indexes(n_quantiles):\n    return np.linspace(0, 1, n_quantiles + 1)\n\ndef get_quantiles(col, n_quantiles):\n    return np.quantile(col, get_quantile_indexes(n_quantiles))\n\ndef cut_column(col, q_this, q_next):\n    # return slice of an array between q_this and q_next inclusive\n    # inclusive means we include 0th and 100th percentiles, at \n    # the expense of possible duplication of middle quantiles \n    return col[(col >= q_this) & (col <= q_next)]\n\ndef quantile_means_array(col, n_quantiles):\n    # slice an array into n quantiles and compute the mean value for each \n    if type(col) != np.ndarray:\n        col = np.array(col)\n    quantiles = get_quantiles(col, n_quantiles)\n    means = np.ndarray(n_quantiles)\n    for i in range(len(quantiles) - 1):\n        means[i] = np.mean(cut_column(col, quantiles[i], quantiles[i+1]))\n    return means, quantiles\n\n\ndef quantile_means_population(X, Y, n_quantiles):\n    # compute means per quantile, for each column in X and Y\n    n_cols = len(X)\n    if not len(X) == len(Y):\n        raise ValueError(\"Matrices must have the same number of columns.\") # pragma: no cover\n\n    means_x = np.ndarray((n_cols, n_quantiles))\n    means_y = np.ndarray((n_cols, n_quantiles))\n    quantiles_x = np.ndarray((n_cols, n_quantiles + 1))\n    quantiles_y = np.ndarray((n_cols, n_quantiles + 1))\n\n    for i in range(n_cols):\n        col_x = X[i]\n        col_y = Y[i]\n        means_x[i], quantiles_x[i] = quantile_means_array(col_x, n_quantiles)\n        means_y[i], quantiles_y[i] = quantile_means_array(col_y, n_quantiles)\n\n    return means_x, means_y, quantiles_x, quantiles_y\n\ndef chisquare_dist(X,Y):\n    distance = 0\n    for i in range(len(X)):\n        distance = distance + ((X[i] - Y[i])**2 / (X[i] + Y[i]))\n    return distance \n\ndef get_distance_func(how=\"euclidean\"):\n    if how == \"euclidean\":\n        return lambda x, y: pdist([x,y])[0]\n    elif how == \"chisquare\":\n        return chisquare_dist \n    else:\n        raise ValueError(f\"Unsupported distance metric: {how}\") # pragma: no cover\n\n\ndef sum_column_distance(means_x, means_y, how=\"euclidean\"):\n    column_distances = np.ndarray(len(means_x))\n    distance_func = get_distance_func(how)\n    for i in range(len(means_x)):\n        column_distances[i] = distance_func(means_x[i], means_y[i])\n    return np.sum(column_distances)\n\n\n\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport copy\nimport pandas as pd\nimport itertools\nimport logging\nimport numpy as np\nfrom .diagnostics import StratifiedSamplingDiagnostics\nfrom .bins import (\n    Binning,\n    BinnedData,\n    ModelSamplingException,\n    sample_bins,\n    get_counts_and_update_n_samples_approx,\n)\n\npd.options.mode.chained_assignment = None  # suppress warnings\n\nlogger = logging.getLogger(__name__)\n\n\nclass StratifiedSampling(object):\n    \"\"\"\n    Perform stratified sampling on a treatment group and comparison pool.  \n    \n    Input data must be provided in the form of two data frames, df_treatment and df_pool, \n    which have identical columns.  These data frames should contain one row per meter, \n    one ID column, and one or more numerical feature columns.  The comparison pool\n    will be stratified (i.e. binned) along one or more of these feature columns, and \n    a comparison group will be selected such that the distribution of features in the \n    comparison group is as close as possible to that of the treatment group.\n\n    Stratification columns must be configured as follows:\n\n        m = StratifiedSampling()\n        m.add_column('annual_usage', min_value=0, max_value=20000)\n        m.add_column('summer_usage', min_value=0, max_value=1000)\n\n    In this case, `annual_usage` and `summer_usage` are feature columns that \n    are present in `df_treatment` and `df_pool`.\n    See `StratifiedSampling.add_column()` for more information on configuring columns.\n    Once columns are added, execute the model as follows:\n\n        m.fit_and_sample(df_treatment, df_pool)\n\n    See `StratifiedSampling.fit_and_sample()` for additional options, notably several\n    parameters which determine the number of meters in the comparison group.\n\n    After fitting the model, you can create a StratifiedSamplingDiagnostics object \n    which has methods for producing diagnostic plots and tables:\n\n        d = m.diagnostics()\n        d.scatter()\n        d.bin_counts()\n\n\n    \"\"\"\n\n    def __init__(\n        self, treatment_label=\"treatment\", pool_label=\"pool\", output_name=\"output\"\n    ):\n        self.columns = {}\n        self.treatment_label = treatment_label\n        self.pool_label = pool_label\n        self.output_name = output_name\n        self.trained = False\n        self.sampled = False\n        self.data_treatment = None\n        self.data_pool = None\n        self.data_sample = None\n\n    def _chop_outliers(self, df):\n        for name, c in self.columns.items():\n            if c[\"min_value_allowed\"] is not None:\n                df = df[df[c[\"name\"]] >= c[\"min_value_allowed\"]]\n            if c[\"max_value_allowed\"] is not None:\n                df = df[df[c[\"name\"]] <= c[\"max_value_allowed\"]]\n        return df\n\n    def _perturb(self, df_orig, col_names=None, random_seed=1):\n        # qcut doesn't work if the same value recurs too many times, i.e. zero.  We can add a small amount of random noise to fix this\n        np.random.seed(random_seed)\n        df_pert = df_orig.copy()\n        col_names = col_names if col_names else list(self.columns.keys())\n        for col_name in col_names:\n            df_pert[col_name] = df_pert[col_name].astype(float)\n            range = df_pert[col_name].max() - df_pert[col_name].min()\n            perturbation = (np.random.random(len(df_pert)) - 0.5) * range * 1e-6\n            df_pert.loc[:, col_name] = df_pert[col_name] + perturbation\n        return df_pert\n\n    def add_column(\n        self,\n        name: str,\n        n_bins: int = None,\n        min_value_allowed: int = None,\n        max_value_allowed: int = None,\n        fixed_width: int = True,\n        auto_bin_require_equivalence: bool = True,\n    ):\n        \"\"\"\n        Add a stratification column to the model.\n\n        Attributes\n        ----------\n        name: str\n            The name of the column to be added to the model.\n        n_bins: int\n            Fixed number of bins to stratify over for this column.\n            If set to None, automatic binning occurs. \n        min_value_allowed: int\n            Minimum treatment value used to construct bins (used to remove outliers).\n        max_value_allowed: int\n            Maximum treatment value used to construct bins (used to remove outliers).\n        auto_bin_require_equivalence: bool\n            Whether the column requires equivalence when auto-binning\n        \"\"\"\n        auto_bin = n_bins is None\n        n_bins = 1 if n_bins is None else n_bins\n\n        self.columns[name] = {\n            \"name\": name,\n            \"auto_bin\": auto_bin,\n            \"n_bins\": n_bins,\n            \"min_value_allowed\": min_value_allowed,\n            \"max_value_allowed\": max_value_allowed,\n            \"fixed_width\": fixed_width,\n            \"auto_bin_require_equivalence\": auto_bin_require_equivalence,\n        }\n\n        self.binning = None\n        self.trained = False\n        self.predicted = False\n        self.col_names = list(self.columns.keys())\n        return self\n\n    def _check_columns_present(self, df):\n        if not getattr(self, \"col_names\"):\n            raise ValueError(\n                \"No columns found in model. Use add_columns(...) to add a column.\"\n            )\n        missing_cols = list(set(self.col_names) - set(df.columns))\n        if len(missing_cols) > 0:\n            raise ValueError(\n                f\"data is missing required columns: {','.join(missing_cols)}\"\n            )\n\n    def fit_and_sample(\n        self,\n        df_treatment,\n        df_pool,\n        n_samples_approx=None,\n        min_n_treatment_per_bin=0,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=4,\n        relax_n_samples_approx_constraint=False,\n    ):\n        \"\"\"\n        Attributes\n        ----------\n        df_treatment: pandas.DataFrame\n            dataframe to use for constructing the stratified sampling bins.\n        df_pool: pandas.DataFrame\n            dataframe to sample from according to the constructed stratified sampling bins.\n        n_samples_approx: int\n            approximate number of total samples from df_pool. It is approximate because\n            there may be some slight discrepencies around the total count to ensure\n            that each bin has the correct percentage of the total.\n        min_n_treatment_per_bin: int\n            Minimum number of treatment samples that must exist in a given bin for \n            it to be considered a non-outlier bin (only applicable if there are \n            cols with fixed_width=True)\n        min_n_sampled_to_n_treatment_ratio: int\n        relax_n_samples_approx_constraint: bool\n            If True, treats n_samples_approx as an upper bound, but gets as many comparison group\n            meters as available up to n_samples_approx. If False, it raises an exception\n            if there are not enough comparison pool meters to reach n_samples_approx.\n            \n        \"\"\"\n        if len(self.columns) == 0:\n            raise ValueError(\"You must add at least one column before fitting.\")\n        logger.debug(self.columns)\n        for name, col in self.columns.items():\n            if col[\"auto_bin\"]:\n                completed = False\n                while not completed:\n                    logging.info(f\"Computing bins: {self.get_all_n_bins_as_str()} \")\n                    self.fit(\n                        df_treatment,\n                        min_n_treatment_per_bin=min_n_treatment_per_bin,\n                        random_seed=random_seed,\n                    )\n                    self.sample(\n                        df_pool,\n                        n_samples_approx=n_samples_approx,\n                        random_seed=random_seed,\n                        relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n                    )\n\n                    def _violates_ratio():\n                        n_sampled_to_n_treatment_ratio = (\n                            self.diagnostics().n_sampled_to_n_treatment_ratio()\n                        )\n                        if (\n                            n_sampled_to_n_treatment_ratio\n                            < min_n_sampled_to_n_treatment_ratio\n                        ):\n                            logger.info(\n                                f\"Insufficient pool data in one of the bins for {col['name']}:\"\n                                f\"found {n_sampled_to_n_treatment_ratio}:1 but need \"\n                                f\"{min_n_sampled_to_n_treatment_ratio}:1. Using last successful n_bins.\"\n                            )\n                            return True\n                        return False\n\n                    if col[\"auto_bin_require_equivalence\"]:\n                        if self.data_sample.df.empty:\n                            raise ValueError(\n                                \"Too many bin divisions before finding equivalence\"\n                                f\" for {col['name']} (usually occurs when several\"\n                                \" stratification params are used).\"\n                            )\n                        completed = self.diagnostics().equivalence_passed([col[\"name\"]])\n                        if min_n_sampled_to_n_treatment_ratio and _violates_ratio():\n                            completed = True\n                            self.set_n_bins(name, self.get_n_bins(name) - 1)\n                        if not completed:\n                            self.set_n_bins(name, self.get_n_bins(name) + 1)\n                    else:\n                        if min_n_sampled_to_n_treatment_ratio and _violates_ratio():\n                            self.set_n_bins(name, self.get_n_bins(name) - 1)\n                            completed = True\n                        else:\n                            self.set_n_bins(name, self.get_n_bins(name) + 1)\n\n        self.fit(\n            df_treatment,\n            min_n_treatment_per_bin=min_n_treatment_per_bin,\n            random_seed=random_seed,\n        )\n        n_treatment = len(df_treatment)\n        # if n_samples_approx is None, use the maximum available.\n        df_sample = self.sample(\n            df_pool,\n            n_samples_approx=n_samples_approx,\n            random_seed=random_seed,\n            relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n        )\n        self.n_samples_approx = n_samples_approx\n        return df_sample\n\n    def print_n_bins(self):\n        logger.info(self.get_all_n_bins_as_str())\n\n    def get_all_n_bins_as_str(self):\n        return \",\".join(\n            [f\"{col}:{self.get_n_bins(col)} bins\" for col in self.columns.keys()]\n        )\n\n    def get_n_bins(self, col_name):\n        col = self.columns[col_name]\n        return col[\"n_bins\"]\n\n    def set_n_bins(self, col_name, n_bins):\n        col = self.columns[col_name]\n        col[\"n_bins\"] = n_bins\n        self.columns[col_name] = col\n\n    def fit(self, df_treatment, min_n_treatment_per_bin=0, random_seed=1):\n        self._check_columns_present(df_treatment)\n        df_treatment = self._perturb(\n            self._chop_outliers(df_treatment), random_seed=random_seed\n        )\n        self.df_treatment = df_treatment.copy()\n        self.binning = Binning()\n\n        self.df_treatment[\"_outlier_value\"] = False\n        for name, col in self.columns.items():\n\n            if col[\"min_value_allowed\"] is not None:\n                self.df_treatment.loc[\n                    self.df_treatment[col[\"name\"]] < col[\"min_value_allowed\"],\n                    \"_outlier_value\",\n                ] = True\n            if col[\"max_value_allowed\"] is not None:\n                self.df_treatment.loc[\n                    self.df_treatment[col[\"name\"]] > col[\"max_value_allowed\"],\n                    \"_outlier_value\",\n                ] = True\n\n        for name, col in self.columns.items():\n            values = (\n                self.df_treatment.loc[~self.df_treatment._outlier_value, col[\"name\"]]\n                .dropna()\n                .astype(float)\n            )\n            self.binning.bin(\n                values, col[\"name\"], col[\"n_bins\"], fixed_width=col[\"fixed_width\"]\n            )\n\n        self.data_treatment = BinnedData(\n            self.df_treatment,\n            self.binning,\n            min_n_treatment_per_bin=min_n_treatment_per_bin,\n        )\n        self.trained = True\n\n    # what kinds of diagnostics?\n    # - explore raw treatment data\n    # - explore raw pool data\n    # - compare treatment data vs pool data, pre-fit\n    # - compare treatment data vs pool data, post-fit\n    # - compare treatment data vs pool data, post-sampled\n\n    def diagnostics(self):\n        return StratifiedSamplingDiagnostics(model=self)\n\n    def sample(\n        self,\n        df_pool,\n        n_samples_approx=None,\n        random_seed=1,\n        relax_n_samples_approx_constraint=False,\n    ):\n        if not self.trained and self.data_treatment is not None:\n            raise ValueError(\"No model found; please run fit()\")\n        \n        self._check_columns_present(df_pool)\n        df_pool = self._perturb(self._chop_outliers(df_pool), random_seed=random_seed)\n        self.data_pool = BinnedData(df_pool, self.binning)\n        (\n            n_samples_approx,\n            relax_ratio_constraint,\n            counts,\n        ) = get_counts_and_update_n_samples_approx(\n            self.data_treatment,\n            self.data_pool,\n            n_samples_approx=n_samples_approx,\n            relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n        )\n        self.relax_ratio_constraint = relax_ratio_constraint\n        df_sample = sample_bins(\n            self.data_treatment,\n            self.data_pool,\n            n_samples_approx=n_samples_approx,\n            relax_n_samples_approx_constraint=relax_n_samples_approx_constraint,\n            random_seed=random_seed,\n        )\n        self.data_sample = BinnedData(df_sample, self.binning)\n        self.sampled = True\n        return self.data_sample\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/param_selection.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\nimport numpy as np\n\n\ndef get_prob_bins(df, df_comparison, col, bins=20, fixed_count=True):\n    # we only care about the range within the treatment group\n    min_dat = df[col].min()\n    max_dat = df[col].max()\n    # do it for the treatment\n    if fixed_count:\n        cuts, bins = pd.qcut(df[col], q=bins, duplicates=\"drop\", retbins=True)\n    else:\n        bins = np.linspace(min_dat, max_dat, bins)\n        cuts = pd.cut(df[col], bins=bins, include_lowest=True)\n    vals = cuts.value_counts(sort=False)\n    vals = vals.values / sum(vals)\n    # do it for the comparison\n    cuts_c = pd.cut(df_comparison[col], bins=bins, include_lowest=True)\n    vals_c = cuts_c.value_counts(sort=False)\n    vals_c = vals_c.values / sum(vals_c)\n    return [vals, vals_c, min_dat, max_dat]\n\n\ndef get_kl_divs(df_treat, df_compare, **kwargs):\n    # Kullback-Leibler divergence\n    # bin the targeting params\n    vcs = [\n        get_prob_bins(df_treat, df_compare, col, **kwargs) for col in df_treat.columns\n    ]\n    # define the kl divergence\n    def kl_(x):\n        p = x[0]\n        q = x[1]\n        # note: q!=0 is an assumption that does not fit with the traditional\n        # use of KL. It assumes that we have any lack of bins in q are due to\n        # outliers in p. (where p is non zero)\n        # it could be worth writing this out further\n        # if there are more than 1 missing value raising more flags\n        return np.sum(np.where((p != 0) & (q != 0), p * np.log(p / q), 0))\n\n    d = pd.DataFrame(vcs, index=df_treat.columns, columns=[0, 1, \"min\", \"max\"])\n    d[\"kl_divergence\"] = d.apply(kl_, axis=1)\n\n    return d[[\"kl_divergence\", \"min\", \"max\"]].sort_values(\n        \"kl_divergence\", ascending=False\n    )\n\n\n# choose parameters based on the correlation matrix\ndef choose_params(difs, corr_matrix, thresh=0.75, num_params=3):\n    ordered_list = difs.index[1:]\n    chosen = [difs.index[0]]\n    for i in ordered_list:\n        if len(chosen) == num_params:\n            break\n        cor = corr_matrix.loc[i]\n        if any(abs(cor.loc[chosen]) > thresh):\n            pass\n        else:\n            chosen.append(i)\n    return chosen\n\n\ndef get_params(treatment, comparison, thresh=0.75, num_params=3, **kwargs):\n    df = get_kl_divs(treatment, comparison, **kwargs)\n    corr_m = treatment.corr()\n    params = choose_params(df, corr_m, thresh, num_params)\n    return params, df\n"
  },
  {
    "path": "opendsm/comparison_groups/stratified_sampling/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport pydantic\n\nimport opendsm.comparison_groups.stratified_sampling.const as _const\nfrom opendsm.common.base_settings import BaseSettings\n\nfrom typing import Optional, Literal, Union\n\n\nclass StratificationColumnSettings(BaseSettings):\n    \"\"\"column name to use for stratification\"\"\"\n    column_name: str = pydantic.Field()\n\n    \"\"\"fixed number of bins to use for stratification\"\"\"\n    n_bins: Optional[int] = pydantic.Field(\n        default=8, \n        ge=2, \n        validate_default=True,\n    )\n\n    \"\"\"minimum treatment value used to construct bins (used to remove outliers)\"\"\"\n    min_value_allowed: int = pydantic.Field(\n        default=3000, \n        ge=0, \n        validate_default=True,\n    )\n\n    \"\"\"maximum treatment value used to construct bins (used to remove outliers)\"\"\"\n    max_value_allowed: int = pydantic.Field(\n        default=6000, \n        ge=0, \n        validate_default=True,\n    )\n\n    \"\"\"whether to use fixed width bins or fixed proportion bins\"\"\"\n    is_fixed_width: bool = pydantic.Field(\n        default=False, \n    )\n\n    \"\"\"column requires equivalence when auto-binning\"\"\"\n    auto_bin_equivalence: Literal[False] = False\n\n\nclass DSS_StratificationColumnSettings(StratificationColumnSettings):\n    \"\"\"fixed number of bins to use for stratification\"\"\"\n    n_bins: Literal[None] = None\n\n    \"\"\"column requires equivalence when auto-binning\"\"\"\n    auto_bin_equivalence: Literal[True] = True\n\n\nclass Settings(BaseSettings):\n    \"\"\"\n    min_n_sampled_to_n_treatment_ratio: int\n        TODO: FILL THIS OUT\n    seed: int\n        Seed for random number generator\n    \"\"\"\n\n    min_n_treatment_per_bin: int = pydantic.Field(\n        default=0, \n        ge=0, \n        validate_default=True,\n    )\n\n    seed: int = pydantic.Field(\n        default=42, \n        ge=0, \n        validate_default=True,\n    )\n\n\nclass StratifiedSamplingSettings(Settings):\n    \"\"\"\n    n_samples_approx: int\n        approximate number of total samples from df_pool. It is approximate because\n        there may be some slight discrepancies around the total count to ensure\n        that each bin has the correct percentage of the total.\n    min_n_treatment_per_bin: int\n        minimum number of treatment samples that must exist in a given bin for \n        it to be considered a non-outlier bin (only applicable if there are \n        cols with fixed_width=True)\n    min_n_sampled_to_n_treatment_ratio: int\n    relax_n_samples_approx_constraint: bool\n        If True, treats n_samples_approx as an upper bound, but gets as many comparison group\n        meters as available up to n_samples_approx. if false, it raises an exception\n        if there are not enough comparison pool meters to reach n_samples_approx.\n    \"\"\"\n\n    n_samples_approx: Optional[int] = pydantic.Field(\n        default=None, \n        ge=1, \n        validate_default=True,\n    )\n\n    relax_n_samples_approx_constraint: bool = pydantic.Field(\n        default=False, \n    )\n\n    equivalence_method: Literal[None] = None\n\n    equivalence_quantile: Literal[None] = None\n\n    min_n_bins: Literal[None] = None\n\n    max_n_bins: Literal[None] = None\n\n    min_n_sampled_to_n_treatment_ratio: float = pydantic.Field(\n        default=4, \n        ge=0, \n        validate_default=True,\n    )\n\n    stratification_column: Union[list[StratificationColumnSettings], list[dict]] = pydantic.Field(\n        default=[\n            StratificationColumnSettings(column_name=\"summer_usage\"),\n            StratificationColumnSettings(column_name=\"winter_usage\"),\n        ],\n    )\n\n    \"\"\"set stratification column classes with given dictionaries\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _set_nested_classes(self):\n        if len(self.stratification_column) > 3:\n            raise ValueError(\"a maximum of 3 stratification_column's are allowed\")\n\n        strat_settings = []\n        has_dict = False\n        for strat_item in self.stratification_column:\n            if isinstance(strat_item, dict):\n                has_dict = True\n                strat_class = StratificationColumnSettings(**strat_item)\n\n            else:\n                strat_class = strat_item\n\n            strat_settings.append(strat_class)\n\n        if has_dict:\n            self.stratification_column = strat_settings\n\n        return self\n\n\n# subclass Settings to change default values\nclass DistanceStratifiedSamplingSettings(Settings):\n    \"\"\"\n    n_samples_approx: int\n        approximate number of total samples from df_pool. It is approximate because\n        there may be some slight discrepancies around the total count to ensure\n        that each bin has the correct percentage of the total.\n    min_n_treatment_per_bin: int\n        Minimum number of treatment samples that must exist in a given bin for \n        it to be considered a non-outlier bin (only applicable if there are \n        cols with fixed_width=True)\n    min_n_sampled_to_n_treatment_ratio: int\n    relax_n_samples_approx_constraint: bool\n        If True, treats n_samples_approx as an upper bound, but gets as many comparison group\n        meters as available up to n_samples_approx. If False, it raises an exception\n        if there are not enough comparison pool meters to reach n_samples_approx.\n    \"\"\"\n    \n    n_samples_approx: Optional[int] = pydantic.Field(\n        default=5000, \n        ge=1, \n        validate_default=True,\n    )\n\n    relax_n_samples_approx_constraint: bool = pydantic.Field(\n        default=True, \n    )\n\n    equivalence_method: _const.DistanceMetric = pydantic.Field(\n        default=_const.DistanceMetric.CHISQUARE,\n        validate_default=True,\n    )\n\n    equivalence_quantile: int = pydantic.Field(\n        default=25,\n        validate_default=True,\n    )\n\n    min_n_bins: int = pydantic.Field(\n        default=1, \n        ge=1, \n        validate_default=True,\n    )\n\n    max_n_bins: int = pydantic.Field(\n        default=8, \n        ge=2, \n        validate_default=True,\n    )\n\n    min_n_sampled_to_n_treatment_ratio: float = pydantic.Field(\n        default=0.25, \n        ge=0, \n        validate_default=True,\n    )\n\n    stratification_column: Union[list[DSS_StratificationColumnSettings], list[dict]] = pydantic.Field(\n        default=[\n            DSS_StratificationColumnSettings(column_name=\"summer_usage\"),\n            DSS_StratificationColumnSettings(column_name=\"winter_usage\"),\n        ],\n    )\n\n    \"\"\"set stratification column classes with given dictionaries\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def _set_nested_classes(self):\n        if len(self.stratification_column) > 3:\n            raise ValueError(\"A maximum of 3 stratification_column's are allowed\")\n\n        strat_settings = []\n        has_dict = False\n        for strat_item in self.stratification_column:\n            if isinstance(strat_item, dict):\n                has_dict = True\n                strat_class = DSS_StratificationColumnSettings(**strat_item)\n\n            else:\n                strat_class = strat_item\n\n            strat_settings.append(strat_class)\n\n        if has_dict:\n            self.stratification_column = strat_settings\n\n        return self\n\n\nif __name__ == \"__main__\":\n    s = StratifiedSamplingSettings()\n    # s = DistanceStratifiedSamplingSettings()\n\n    print(s.model_dump_json())"
  },
  {
    "path": "opendsm/drmeter/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.drmeter.models import *\n"
  },
  {
    "path": "opendsm/drmeter/models/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .caltrack import (\n    Model as CaltrackDRModel,\n    BaselineData as CaltrackDRBaselineData,\n    ReportingData as CaltrackDRReportingData,\n)\n"
  },
  {
    "path": "opendsm/drmeter/models/caltrack/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .data import BaselineData, ReportingData\nfrom .model import Model\n"
  },
  {
    "path": "opendsm/drmeter/models/caltrack/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter.models.hourly_caltrack.data import (\n    HourlyBaselineData as BaselineData,\n    HourlyReportingData as ReportingData,\n)\n"
  },
  {
    "path": "opendsm/drmeter/models/caltrack/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter.models.hourly_caltrack import HourlyModel\n\n\nclass Model(HourlyModel):\n    def __init__(self, settings=None):\n        self.segment_type = \"single\"\n        self.alpha = 0.1\n"
  },
  {
    "path": "opendsm/eemeter/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter.models import *\nfrom opendsm.eemeter.utilities import *\nfrom opendsm.eemeter.samples import *\n"
  },
  {
    "path": "opendsm/eemeter/common/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n"
  },
  {
    "path": "opendsm/eemeter/common/data_processor_utilities.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom math import ceil\nfrom typing import Optional\n\nimport numpy as np\nimport pandas as pd\nimport pytz\nfrom pandas.tseries.offsets import MonthBegin, MonthEnd\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\ndef remove_duplicates(df_or_series):\n    \"\"\"Remove duplicate rows or values by keeping the first of each duplicate.\n\n    Parameters\n    ----------\n    df_or_series : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        Pandas object from which to drop duplicate index values.\n\n    Returns\n    -------\n    deduplicated : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        The deduplicated pandas object.\n    \"\"\"\n    # CalTrack 2.3.2.2\n    return df_or_series[~df_or_series.index.duplicated(keep=\"first\")]\n\n\ndef day_counts(index):\n    \"\"\"Days between DatetimeIndex values as a :any:`pandas.Series`.\n\n    Parameters\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        The index for which to get day counts.\n\n    Returns\n    -------\n    day_counts : :any:`pandas.Series`\n        A :any:`pandas.Series` with counts of days between periods. Counts are\n        given on start dates of periods.\n    \"\"\"\n    # dont affect the original data\n    index = index.copy()\n\n    if len(index) == 0:\n        return pd.Series([], index=index)\n\n    timedeltas = (index[1:] - index[:-1]).append(pd.TimedeltaIndex([pd.NaT]))\n    timedelta_days = timedeltas.total_seconds() / (60 * 60 * 24)\n\n    return pd.Series(timedelta_days, index=index)\n\n\ndef clean_billing_data(data, source_interval, warnings):\n    # check for empty data\n    if data[\"value\"].dropna().empty:\n        return data[:0]\n\n    if source_interval.startswith(\"billing\"):\n        diff = list((data.index[1:] - data.index[:-1]).days)\n        filter_ = pd.Series(diff + [np.nan], index=data.index)\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        if source_interval == \"billing_monthly\":\n            data = data[\n                (filter_ <= 35) & (filter_ >= 25)  # keep these, inclusive\n            ].reindex(data.index)\n\n            if len(data[(filter_ > 35) | (filter_ < 25)]) > 0:\n                warnings.append(\n                    EEMeterWarning(\n                        qualified_name=\"eemeter.sufficiency_criteria.offcycle_reads_in_billing_monthly_data\",\n                        description=(\n                            \"Off-cycle reads found in billing monthly data having a duration of less than 25 days\"\n                        ),\n                        data=[\n                            timestamp.isoformat()\n                            for timestamp in data[(filter_ > 35) | (filter_ < 25)].index\n                        ],\n                    )\n                )\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        if source_interval == \"billing_bimonthly\":\n            data = data[\n                (filter_ <= 70) & (filter_ >= 25)  # keep these, inclusive\n            ].reindex(data.index)\n\n            if len(data[(filter_ > 70) | (filter_ < 25)]) > 0:\n                warnings.append(\n                    EEMeterWarning(\n                        qualified_name=\"eemeter.sufficiency_criteria.offcycle_reads_in_billing_monthly_data\",\n                        description=(\n                            \"Off-cycle reads found in billing monthly data having a duration of less than 25 days\"\n                        ),\n                        data=[\n                            timestamp.isoformat()\n                            for timestamp in data[(filter_ > 70) | (filter_ < 25)].index\n                        ],\n                    )\n                )\n\n        # CalTRACK 2.2.3.1\n        \"\"\"\n        Adds estimate to subsequent read if there aren't more than one estimate in a row\n        and then removes the estimated row.\n\n        Input:\n        index   value   estimated\n        1       2       False\n        2       3       False\n        3       5       True\n        4       4       False\n        5       6       True\n        6       3       True\n        7       4       False\n        8       NaN     NaN\n\n        Output:\n        index   value\n        1       2\n        2       3\n        4       9\n        5       NaN\n        7       7\n        8       NaN\n        \"\"\"\n        add_estimated = []\n        remove_estimated_fixed_rows = []\n        orig_data = data.copy()\n        if \"estimated\" in data.columns:\n            data[\"unestimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated == False)].reindex(data.index)\n            )\n            data[\"estimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated)].reindex(data.index)\n            )\n            for i, (index, row) in enumerate(data[:-1].iterrows()):\n                # ensures there is a prev_row and previous row value is null\n                if i > 0 and pd.isnull(prev_row[\"unestimated_value\"]):\n                    # current row value is not null\n                    add_estimated.append(prev_row[\"estimated_value\"])\n                    if not pd.isnull(row[\"unestimated_value\"]):\n                        # get all rows that had only estimated reads that will be\n                        # added to the subsequent row meaning this row\n                        # needs to be removed\n                        remove_estimated_fixed_rows.append(prev_index)\n                else:\n                    add_estimated.append(0)\n                prev_row = row\n                prev_index = index\n            add_estimated.append(np.nan)\n            data[\"value\"] = data[\"unestimated_value\"] + add_estimated\n            data = data[~data.index.isin(remove_estimated_fixed_rows)]\n            data = data[[\"value\"]]  # remove the estimated column\n\n    # check again for empty data\n    if data.dropna().empty:\n        return data[:0]\n\n    return data[\"value\"].to_frame()\n\n\ndef as_freq(\n    data_series,\n    freq,\n    atomic_freq=\"1 Min\",\n    series_type=\"cumulative\",\n    include_coverage=False,\n):\n    \"\"\"Resample data to a different frequency.\n\n    This method can be used to upsample or downsample meter data. The\n    assumption it makes to do so is that meter data is constant and averaged\n    over the given periods. For instance, to convert billing-period data to\n    daily data, this method first upsamples to the atomic frequency\n    (1 minute freqency, by default), \"spreading\" usage evenly across all\n    minutes in each period. Then it downsamples to hourly frequency and\n    returns that result. With instantaneous series, the data is copied to all\n    contiguous time intervals and the mean over `freq` is returned.\n\n    **Caveats**:\n\n     - This method gives a fair amount of flexibility in\n       resampling as long as you are OK with the assumption that usage is\n       constant over the period (this assumption is generally broken in\n       observed data at large enough frequencies, so this caveat should not be\n       taken lightly).\n\n    Parameters\n    ----------\n    data_series : :any:`pandas.Series`\n        Data to resample. Should have a :any:`pandas.DatetimeIndex`.\n    freq : :any:`str`\n        The frequency to resample to. This should be given in a form recognized\n        by the :any:`pandas.Series.resample` method.\n    atomic_freq : :any:`str`, optional\n        The \"atomic\" frequency of the intermediate data form. This can be\n        adjusted to a higher atomic frequency to increase speed or memory\n        performance.\n    series_type : :any:`str`, {'cumulative', ‘instantaneous’},\n        default 'cumulative'\n        Type of data sampling. 'cumulative' data can be spread over smaller\n        time intervals and is aggregated using addition (e.g. meter data).\n        'instantaneous' data is copied (not spread) over smaller time intervals\n        and is aggregated by averaging (e.g. weather data).\n    include_coverage: :any:`bool`,\n        default `False`\n        Option of whether to return a series with just the resampled values\n        or a dataframe with a column that includes percent coverage of source data\n        used for each sample.\n\n    Returns\n    -------\n    resampled_data : :any:`pandas.Series` or :any:`pandas.DataFrame`\n        Data resampled to the given frequency (optionally as a dataframe with a coverage column if `include_coverage` is used.\n    \"\"\"\n    # TODO(philngo): make sure this complies with CalTRACK 2.2.2.1\n    if not isinstance(data_series, pd.Series):\n        raise ValueError(\n            \"expected series, got object with class {}\".format(data_series.__class__)\n        )\n    if data_series.empty:\n        return data_series\n    series = remove_duplicates(data_series)\n    target_freq = pd.Timedelta(atomic_freq)\n    timedeltas = (series.index[1:] - series.index[:-1]).append(\n        pd.TimedeltaIndex([pd.NaT])\n    )\n\n    if series_type == \"cumulative\":\n        spread_factor = target_freq.total_seconds() / timedeltas.total_seconds()\n        series_spread = series * spread_factor\n        atomic_series = series_spread.asfreq(atomic_freq, method=\"ffill\")\n        resampled = atomic_series.resample(freq, origin=series.index[0]).sum()\n        resampled_with_nans = atomic_series.resample(\n            freq, origin=series.index[0]\n        ).first()\n        n_coverage = atomic_series.resample(freq, origin=series.index[0]).count()\n        resampled = resampled[resampled_with_nans.notnull()].reindex(resampled.index)\n\n    elif series_type == \"instantaneous\":\n        # ffill on series.asfreq can produce unintuitive results if resampling a sparse matrix.\n        # for example, attempting to resample 2 months of hourly data to daily with a month of\n        # absent rows (not NaN, but missing from the dataframe) will ffill that month with the previous read.\n        #\n        # a similar effect can happen if you have NaNs at a different frequency appended to the end\n        # of a series. this could happen if you concat a monthly series with an hourly one at an offset.\n        # the call to asfreq() could erroneously fill in a month of data, followed by NaNs\n        atomic_series = series.asfreq(atomic_freq, method=\"ffill\")\n        resampled = atomic_series.resample(freq, origin=series.index[0]).mean()\n        n_coverage = atomic_series.resample(freq, origin=series.index[0]).count()\n\n    # Edit : Added a check so that hourly and daily frequencies don't have a null value at the end\n    if freq not in [\"h\", \"D\"] and resampled.index[-1] < series.index[-1]:\n        # this adds a null at the end using the target frequency\n        last_index = pd.date_range(resampled.index[-1], freq=freq, periods=2)[1:]\n        resampled = (\n            pd.concat([resampled, pd.Series(np.nan, index=last_index)])\n            .resample(freq)\n            .mean()\n        )\n    if include_coverage:\n        n_total = (\n            resampled.resample(atomic_freq)\n            .count()\n            .resample(freq, origin=resampled.index[0])\n            .count()\n        )\n        resampled = resampled.to_frame(\"value\")\n        resampled[\"coverage\"] = n_coverage / n_total\n\n        # TODO : hacky fix to account all occurences of last hour not being counted due to the NaN appended above.\n        # Due to above issue number of median granularity periods would end up being 1 rather than the entire 720(24 * 30), thus squashing the\n        # reported value to 1/720th the actual. Set it back to 1 like the other usual periods. Might break if the last period is uneven.\n        if resampled.coverage.iloc[-1] > 1:\n            resampled.iloc[-1, resampled.columns.get_loc(\"coverage\")] = 1\n        return resampled\n    else:\n        return resampled\n\n\ndef downsample_and_clean_daily_data(dataset, warnings):\n    dataset = as_freq(dataset, \"D\", include_coverage=True)\n\n    if not dataset[dataset.coverage <= 0.5].empty:\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_meter_data\",\n                description=(\n                    \"More than 50% of the high frequency Meter data is missing.\"\n                ),\n                data=[\n                    timestamp.isoformat()\n                    for timestamp in dataset[dataset.coverage <= 0.5].index\n                ],\n            )\n        )\n\n    # CalTRACK 2.2.2.1 - interpolate with average of non-null values\n    dataset.loc[dataset.coverage > 0.5, \"value\"] = (\n        dataset[dataset.coverage > 0.5].value / dataset[dataset.coverage > 0.5].coverage\n    )\n\n    return dataset[dataset.coverage > 0.5].reindex(dataset.index)[[\"value\"]]\n\n\ndef clean_billing_daily_data(data, source_interval, warnings):\n    # billing data is cleaned but not resampled\n    if source_interval.startswith(\"billing\"):\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        return clean_billing_data(data, source_interval, warnings)\n\n    # higher intervals like daily, hourly, 30min, 15min are\n    # resampled (daily) or downsampled (hourly, 30min, 15min)\n    elif source_interval == \"daily\":\n        return data.to_frame(\"value\")\n    else:\n        return downsample_and_clean_daily_data(data, warnings)\n\n\n# TODO : requires more testing\ndef compute_minimum_granularity(index: pd.Series, default_granularity: Optional[str]):\n    if len(index) <= 1:\n        return default_granularity\n    # Inferred frequency returns None if frequency can't be autodetected\n    index.freq = index.inferred_freq\n    if index.freq is None:\n        # max_difference = day_counts(index).max()\n        # min_difference = day_counts(index).min()\n        median_difference = day_counts(index).median()\n        # if max_difference == 1 and min_difference == 1:\n        #     min_granularity = 'daily'\n        # elif max_difference < 1:\n        #     min_granularity = 'hourly'\n        # elif max_difference >= 60:\n        #     min_granularity = 'billing_bimonthly'\n        # elif max_difference >= 30:\n        #     min_granularity = 'billing_monthly'\n        # else:\n        #     min_granularity = default_granularity\n\n        granularity_dict = {\n            median_difference < 1: \"hourly\",\n            median_difference == 1: \"daily\",\n            1 < median_difference <= 35: \"billing_monthly\",\n            35 < median_difference <= 70: \"billing_bimonthly\",\n        }\n        min_granularity = granularity_dict.get(True, default_granularity)\n        return min_granularity\n    # The other cases still result in granularity being unknown so this causes the frequency to be resampled to daily\n    if isinstance(index.freq, MonthEnd) or isinstance(\n        index.freq, MonthBegin\n    ):  # Can be MonthEnd or MonthBegin instance\n        if index.freq.n == 1:\n            min_granularity = \"billing_monthly\"\n        else:\n            min_granularity = \"billing_bimonthly\"\n    elif index.freq <= pd.Timedelta(hours=1):\n        min_granularity = \"hourly\"\n    elif index.freq <= pd.Timedelta(days=1):\n        min_granularity = \"daily\"\n    elif index.freq <= pd.Timedelta(days=30):\n        min_granularity = \"billing_monthly\"\n    else:\n        min_granularity = \"billing_bimonthly\"\n\n    return min_granularity\n"
  },
  {
    "path": "opendsm/eemeter/common/data_settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nimport pydantic\nimport datetime\n\nfrom typing import Optional, Union\n\nfrom opendsm.common.base_settings import MutableBaseSettings\n\n\n# TODO: use this in future for all columns\nclass ColumnSufficiencySettings(MutableBaseSettings):\n    min_pct_hourly_coverage: float = pydantic.Field(\n        default=0.5,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of hourly coverage.\",\n    )\n    \n    min_pct_daily_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of daily coverage.\",\n    )\n    \n    min_pct_monthly_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of monthly coverage.\",\n    )\n\n    min_pct_period_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of period coverage.\",\n    )\n\n\nclass TemperatureSufficiencySettings(ColumnSufficiencySettings):\n    pass\n\n\nclass GhiSufficiencySettings(MutableBaseSettings):\n    min_pct_monthly_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of monthly coverage.\",\n    )\n\n\nclass ObservedSufficiencySettings(MutableBaseSettings):\n    min_pct_hourly_coverage: float = pydantic.Field(\n        default=0.5,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of hourly coverage.\",\n    )\n    \n    min_pct_daily_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of daily coverage.\",\n    )\n    \n    min_pct_monthly_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of monthly coverage.\",\n    )\n\n\nclass JointSufficiencySettings(MutableBaseSettings):\n    min_pct_daily_coverage: float = pydantic.Field(\n        default=0.9,\n        gt=0,\n        le=1,\n        description=\"Minimum percentage of daily coverage.\",\n    )\n\n\nclass BaseSufficiencySettings(MutableBaseSettings):\n    requested_start: Optional[pd.Timestamp] = pydantic.Field(\n        default=None,\n        description=\"Requested start date for the data. If None, use the data start date.\"\n    )\n    \n    requested_end: Optional[pd.Timestamp] = pydantic.Field(\n        default=None,\n        description=\"Requested end date for the data. If None, use the data end date.\"\n    )\n\n    min_baseline_length: int = pydantic.Field(\n        default=np.ceil(0.9 * 365),\n        ge=1,\n        description=\"Minimum number of days in the baseline.\",\n    )\n\n    max_baseline_length: int = pydantic.Field(\n        default=366, # 366 for leap year\n        ge=2,\n        description=\"Maximum number of days in the baseline.\",\n    )\n\n    temperature: TemperatureSufficiencySettings = pydantic.Field(\n        default_factory=TemperatureSufficiencySettings,\n    )\n\n    ghi: GhiSufficiencySettings = pydantic.Field(\n        default_factory=GhiSufficiencySettings,\n    )\n\n    observed: ObservedSufficiencySettings = pydantic.Field(\n        default_factory=ObservedSufficiencySettings,\n    )\n\n    joint: JointSufficiencySettings = pydantic.Field(\n        default_factory=JointSufficiencySettings,\n    )\n\n    @pydantic.field_validator(\"min_baseline_length\", \"max_baseline_length\", mode=\"before\")\n    @classmethod\n    def convert_float_to_int(cls, v):\n        if isinstance(v, float) and v.is_integer():\n            return int(v)\n        return v\n\n    @pydantic.model_validator(mode=\"after\")\n    def check_baseline_lengths(self):\n        max_baseline_length = self.max_baseline_length\n        min_baseline_length = self.min_baseline_length\n        if max_baseline_length <= min_baseline_length:\n            raise ValueError(\n                f\"max_baseline_length ({max_baseline_length}) must be greater than min_baseline_length ({min_baseline_length})\"\n            )\n        \n        return self\n\n\nclass DailyDataSufficiencySettings(BaseSufficiencySettings):\n    ghi: None = None\n    \n\nclass BillingDataSufficiencySettings(BaseSufficiencySettings):\n    ghi: None = None\n\n    min_days_in_period: int = pydantic.Field(\n        default=25,\n        ge=1,\n        description=\"Minimum number of days in a billing period.\",\n    )\n\n    max_days_in_monthly_period: int = pydantic.Field(\n        default=70,\n        ge=1,\n        description=\"Maximum number of days in a billing period.\",\n    )\n\n    max_days_in_bimonthly_period: int = pydantic.Field(\n        default=70,\n        ge=1,\n        description=\"Maximum number of days in a billing period.\",\n    )\n\n    @pydantic.field_validator(\"min_days_in_period\", \"max_days_in_monthly_period\", \"max_days_in_bimonthly_period\", mode=\"before\")\n    @classmethod\n    def convert_float_to_int(cls, v):\n        if isinstance(v, float) and v.is_integer():\n            return int(v)\n        return v\n\n    \nclass HourlyTemperatureSufficiencySettings(TemperatureSufficiencySettings):\n    max_consecutive_hours_missing: int = pydantic.Field(\n        default=6,\n        ge=0,\n        description=\"Maximum number of consecutive missing hours to declare the day as missing.\",\n    )\n\n    @pydantic.field_validator(\"max_consecutive_hours_missing\", mode=\"before\")\n    @classmethod\n    def convert_float_to_int(cls, v):\n        if isinstance(v, float) and v.is_integer():\n            return int(v)\n        return v\n\nclass HourlyDataSufficiencySettings(BaseSufficiencySettings):\n    temperature: HourlyTemperatureSufficiencySettings = pydantic.Field(\n        default_factory=HourlyTemperatureSufficiencySettings,\n    )\n\n\nclass BaseDataSettings(MutableBaseSettings):\n    \"\"\"is electricity data\"\"\"\n    is_electricity_data: bool = pydantic.Field(\n        default=True, # TODO: if is_electricity_data removed from data, this needs to be required\n        description=\"Boolean flag to specify if the data is electricity data or not.\",\n    )\n\n    time_zone: Optional[datetime.timezone] = pydantic.Field(\n        default=None,\n        description=\"Time zone for the data, e.g., 'America/Los_Angeles'. If None, time zone is not set.\"\n    )\n\nclass DailyDataSettings(BaseDataSettings):\n    sufficiency: DailyDataSufficiencySettings = pydantic.Field(\n        default_factory=DailyDataSufficiencySettings,\n    )\n\nclass BillingDataSettings(BaseDataSettings):\n    sufficiency: BillingDataSufficiencySettings = pydantic.Field(\n        default_factory=BillingDataSufficiencySettings,\n    )\n\nclass HourlyDataSettings(BaseDataSettings):\n    pv_start: Optional[Union[datetime.date, str]] = pydantic.Field(\n        default=None,\n        description=\"Date of the solar installation. If None, assume solar status is static.\"\n    )\n\n    sufficiency: HourlyDataSufficiencySettings = pydantic.Field(\n        default_factory=HourlyDataSufficiencySettings,\n    )"
  },
  {
    "path": "opendsm/eemeter/common/exceptions.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\n__all__ = (\n    \"EEMeterError\",\n    \"NoBaselineDataError\",\n    \"NoReportingDataError\",\n    \"MissingModelParameterError\",\n    \"UnrecognizedModelTypeError\",\n    \"DataSufficiencyError\",\n    \"DisqualifiedModelError\",\n)\n\n\nclass EEMeterError(Exception):\n    \"\"\"Base class for EEmeter library errors.\"\"\"\n\n    pass\n\n\nclass NoBaselineDataError(EEMeterError):\n    \"\"\"Error indicating lack of baseline data.\"\"\"\n\n    pass\n\n\nclass NoReportingDataError(EEMeterError):\n    \"\"\"Error indicating lack of reporting data.\"\"\"\n\n    pass\n\n\nclass MissingModelParameterError(EEMeterError):\n    \"\"\"Error indicating missing model parameter.\"\"\"\n\n    pass\n\n\nclass UnrecognizedModelTypeError(EEMeterError):\n    \"\"\"Error indicating unrecognized model type.\"\"\"\n\n    pass\n\n\nclass DataSufficiencyError(EEMeterError):\n    \"\"\"Error indicating insufficient data to fit model on.\"\"\"\n\n    pass\n\n\nclass DisqualifiedModelError(EEMeterError):\n    \"\"\"Error indicating attempt to predict with disqualified or poorly fit model.\"\"\"\n\n    pass\n"
  },
  {
    "path": "opendsm/eemeter/common/features.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport statsmodels.formula.api as smf\n\nfrom ..models.hourly_caltrack.segmentation import iterate_segmented_dataset\nfrom .transform import day_counts, overwrite_partial_rows_with_nan\nfrom .warnings import EEMeterWarning\n\n__all__ = (\n    \"compute_usage_per_day_feature\",\n    \"compute_occupancy_feature\",\n    \"compute_temperature_features\",\n    \"compute_temperature_bin_features\",\n    \"compute_time_features\",\n    \"estimate_hour_of_week_occupancy\",\n    \"fit_temperature_bins\",\n    \"get_missing_hours_of_week_warning\",\n    \"merge_features\",\n)\n\n\ndef merge_features(features, keep_partial_nan_rows=False):\n    \"\"\"\n    Combine dataframes of features which share a datetime index.\n\n    Parameters\n    ----------\n    features : :any:`list` of :any:`pandas.DataFrame`\n        List of dataframes to be concatenated to share an index.\n    keep_partial_nan_rows : :any:`bool`, default False\n        If True, don't overwrite partial rows with NaN, otherwise any row with a NaN\n        value gets changed to all NaN values.\n\n    Returns\n    -------\n    merged_features : :any:`pandas.DataFrame`\n        A single dataframe with the index of the input data and all of the columns\n        in the input feature dataframes.\n    \"\"\"\n\n    def _to_frame_if_needed(df_or_series):\n        if isinstance(df_or_series, pd.Series):\n            return df_or_series.to_frame()\n        return df_or_series\n\n    df = pd.concat([_to_frame_if_needed(feature) for feature in features], axis=1)\n\n    if not keep_partial_nan_rows:\n        df = overwrite_partial_rows_with_nan(df)\n    return df\n\n\ndef compute_usage_per_day_feature(meter_data, series_name=\"usage_per_day\"):\n    \"\"\"Compute average usage per day for billing/daily data.\n\n    Parameters\n    ----------\n    meter_data : :any:`pandas.DataFrame`\n        Meter data for which to compute usage per day.\n    series_name : :any:`str`\n        Name of the output pandas series\n\n    Returns\n    -------\n    usage_per_day_feature : :any:`pandas.Series`\n        The usage per day feature.\n    \"\"\"\n    # CalTrack 3.3.1.1\n    # convert to average daily meter values.\n    usage_per_day = meter_data.value / day_counts(meter_data.index)\n    return pd.Series(usage_per_day, name=series_name)\n\n\ndef get_missing_hours_of_week_warning(hours_of_week):\n    \"\"\"Warn if any hours of week (0-167) are missing.\n\n    Parameters\n    ----------\n    hours_of_week : :any:`pandas.Series`\n        Hour of week feature as given by :any:`eemeter.compute_time_features`.\n\n    Returns\n    -------\n    warning : :any:`eemeter.EEMeterWarning`\n        Warning with qualified name \"eemeter.hour_of_week.missing\"\n    \"\"\"\n    unique = set(hours_of_week.unique())\n    total = set(range(168))\n    missing = sorted(total - unique)\n    if len(missing) == 0:\n        return None\n    else:\n        return EEMeterWarning(\n            qualified_name=\"eemeter.hour_of_week.missing\",\n            description=\"Missing some of the (zero-indexed) 168 hours of the week.\",\n            data={\"missing_hours_of_week\": missing},\n        )\n\n\ndef compute_time_features(index, hour_of_week=True, day_of_week=True, hour_of_day=True):\n    \"\"\"Compute hour of week, day of week, or hour of day features.\n\n    Parameters\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        Datetime index with hourly frequency.\n    hour_of_week : :any:`bool`\n        Include the `hour_of_week` feature.\n    day_of_week : :any:`bool`\n        Include the `day_of_week` feature.\n    hour_of_day : :any:`bool`\n        Include the `hour_of_day` feature.\n\n    Returns\n    -------\n    time_features : :any:`pandas.DataFrame`\n        A dataframe with the input datetime index and up to three columns\n\n        - hour_of_week : Label for hour of week, 0-167, 0 is 12-1am Monday\n        - day_of_week : Label for day of week, 0-6, 0 is Monday.\n        - hour_of_day : Label for hour of day, 0-23, 0 is 12-1am.\n    \"\"\"\n    if index.freq != \"h\":\n        raise ValueError(\n            \"index must have hourly frequency (freq='H').\"\n            \" Found: {}\".format(index.freq)\n        )\n\n    dow_feature = pd.Series(index.dayofweek, index=index, name=\"day_of_week\")\n    hod_feature = pd.Series(index.hour, index=index, name=\"hour_of_day\")\n    how_feature = (dow_feature * 24 + hod_feature).rename(\"hour_of_week\")\n\n    features = []\n    warnings = []\n\n    if day_of_week:\n        features.append(dow_feature.astype(\"category\"))\n    if hour_of_day:\n        features.append(hod_feature.astype(\"category\"))\n    if hour_of_week:\n        how_feature = how_feature.astype(\"category\")\n        features.append(how_feature)\n        warning = get_missing_hours_of_week_warning(how_feature)\n        if warning is not None:\n            warnings.append(warning)\n\n    if len(features) == 0:\n        raise ValueError(\"No features selected.\")\n\n    time_features = merge_features(features)\n    return time_features\n\n\ndef _matching_groups(index, df, tolerance):\n    # convert index to df for use with merge_asof\n    index_df = pd.DataFrame({\"index_col\": index}, index=index)\n\n    # get a dataframe containing mean temperature\n    #   1) merge by matching temperature to closest previous meter start date,\n    #      up to tolerance limit, using merge_asof.\n    #   2) group by meter_index, and take the mean, ignoring all columns except\n    #      the temperature column.\n    groups = pd.merge_asof(\n        left=df, right=index_df, left_index=True, right_index=True, tolerance=tolerance\n    ).groupby(\"index_col\")\n    return groups\n\n\ndef _degree_day_columns(\n    heating_balance_points,\n    cooling_balance_points,\n    degree_day_method,\n    percent_hourly_coverage_per_day,\n    percent_hourly_coverage_per_billing_period,\n    use_mean_daily_values,\n):\n    # TODO(philngo): can this be refactored to be a more general without losing\n    # on performance?\n\n    # Not used in CalTRACK 2.0\n    if degree_day_method == \"hourly\":\n\n        def _compute_columns(temps):\n            n_temps = temps.shape[0]\n            n_temps_kept = temps.count()\n            count_cols = {\n                \"n_hours_kept\": n_temps_kept,\n                \"n_hours_dropped\": n_temps - n_temps_kept,\n            }\n            if use_mean_daily_values:\n                n_days = 1\n            else:\n                n_days = n_temps / 24.0\n            cdd_cols = {\n                \"cdd_%s\" % bp: np.maximum(temps - bp, 0).mean() * n_days\n                for bp in cooling_balance_points\n            }\n            hdd_cols = {\n                \"hdd_%s\" % bp: np.maximum(bp - temps, 0).mean() * n_days\n                for bp in heating_balance_points\n            }\n\n            columns = count_cols\n            columns.update(cdd_cols)\n            columns.update(hdd_cols)\n            return columns\n\n    # CalTRACK 2.2.2.3\n    n_limit_daily = 24 * percent_hourly_coverage_per_day\n\n    if degree_day_method == \"daily\":\n\n        def _compute_columns(temps):\n            count = temps.shape[0]\n            if count > 24:\n                day_groups = np.floor(np.arange(count) / 24)\n                daily_temps = temps.groupby(day_groups).agg([\"mean\", \"count\"])\n                n_limit_period = percent_hourly_coverage_per_billing_period * count\n                n_days_total = daily_temps.shape[0]\n\n                # CalTrack 2.2.3.2\n                if temps.notnull().sum() < n_limit_period:\n                    daily_temps = daily_temps[\"mean\"].iloc[0:0]\n                else:\n                    # CalTRACK 2.2.2.3\n                    daily_temps = daily_temps[\"mean\"][\n                        daily_temps[\"count\"] > n_limit_daily\n                    ]\n                n_days_kept = daily_temps.shape[0]\n                count_cols = {\n                    \"n_days_kept\": n_days_kept,\n                    \"n_days_dropped\": n_days_total - n_days_kept,\n                }\n\n                if use_mean_daily_values:\n                    n_days = 1\n                else:\n                    n_days = n_days_total\n\n                cdd_cols = {\n                    \"cdd_%s\" % bp: np.maximum(daily_temps - bp, 0).mean() * n_days\n                    for bp in cooling_balance_points\n                }\n                hdd_cols = {\n                    \"hdd_%s\" % bp: np.maximum(bp - daily_temps, 0).mean() * n_days\n                    for bp in heating_balance_points\n                }\n            else:  # faster route for daily case, should have same effect.\n                if count > n_limit_daily:\n                    count_cols = {\"n_days_kept\": 1, \"n_days_dropped\": 0}\n                    # CalTRACK 2.2.2.3\n                    mean_temp = temps.mean()\n                else:\n                    count_cols = {\"n_days_kept\": 0, \"n_days_dropped\": 1}\n                    mean_temp = np.nan\n\n                # CalTrack 3.3.4.1.1\n                cdd_cols = {\n                    \"cdd_%s\" % bp: np.maximum(mean_temp - bp, 0)\n                    for bp in cooling_balance_points\n                }\n\n                # CalTrack 3.3.5.1.1\n                hdd_cols = {\n                    \"hdd_%s\" % bp: np.maximum(bp - mean_temp, 0)\n                    for bp in heating_balance_points\n                }\n\n            columns = count_cols\n            columns.update(cdd_cols)\n            columns.update(hdd_cols)\n            return columns\n\n    # TODO(philngo): option to ignore the count columns?\n\n    agg_funcs = [(\"degree_day_columns\", _compute_columns)]\n    return agg_funcs\n\n\ndef compute_temperature_features(\n    meter_data_index,\n    temperature_data,\n    heating_balance_points=None,\n    cooling_balance_points=None,\n    data_quality=False,\n    temperature_mean=True,\n    degree_day_method=\"daily\",\n    percent_hourly_coverage_per_day=0.5,\n    percent_hourly_coverage_per_billing_period=0.9,\n    use_mean_daily_values=True,\n    tolerance=None,\n    keep_partial_nan_rows=False,\n):\n    \"\"\"Compute temperature features from hourly temperature data using the\n    :any:`pandas.DatetimeIndex` meter data..\n\n    Creates a :any:`pandas.DataFrame` with the same index as the meter data.\n\n    .. note::\n\n        For CalTRACK compliance (2.2.2.3), must set\n        ``percent_hourly_coverage_per_day=0.5``,\n        ``cooling_balance_points=range(30,90,X)``, and\n        ``heating_balance_points=range(30,90,X)``, where\n        X is either 1, 2, or 3. For natural gas meter use data, must\n        set ``fit_cdd=False``.\n\n    .. note::\n\n        For CalTRACK compliance (2.2.3.2), for billing methods, must set\n        ``percent_hourly_coverage_per_billing_period=0.9``.\n\n    .. note::\n\n        For CalTRACK compliance (2.3.3), ``meter_data_index`` and ``temperature_data``\n        must both be timezone-aware and have matching timezones.\n\n    .. note::\n\n        For CalTRACK compliance (3.3.1.1), for billing methods, must set\n        ``use_mean_daily_values=True``.\n\n    .. note::\n\n        For CalTRACK compliance (3.3.1.2), for daily or billing methods,\n        must set ``degree_day_method=daily``.\n\n    Parameters\n    ----------\n    meter_data_index : :any:`pandas.DataFrame`\n        A :any:`pandas.DatetimeIndex` corresponding to the index over which\n        to compute temperature features.\n    temperature_data : :any:`pandas.Series`\n        Series with :any:`pandas.DatetimeIndex` with hourly (``'H'``) frequency\n        and a set of temperature values.\n    cooling_balance_points : :any:`list` of :any:`int` or :any:`float`, optional\n        List of cooling balance points for which to create cooling degree days.\n    heating_balance_points : :any:`list` of :any:`int` or :any:`float`, optional\n        List of heating balance points for which to create heating degree days.\n    data_quality : :any:`bool`, optional\n        If True, compute data quality columns for temperature, i.e.,\n        ``temperature_not_null`` and ``temperature_null``, containing for\n        each meter value\n    temperature_mean : :any:`bool`, optional\n        If True, compute temperature means for each meter period.\n    degree_day_method : :any:`str`, ``'daily'`` or ``'hourly'``\n        The method to use in calculating degree days.\n    percent_hourly_coverage_per_day : :any:`str`, optional\n        Percent hourly temperature coverage per day for heating and cooling\n        degree days to not be dropped.\n    use_mean_daily_values : :any:`bool`, optional\n        If True, meter and degree day values should be mean daily values, not\n        totals. If False, totals will be used instead.\n    tolerance : :any:`pandas.Timedelta`, optional\n        Do not merge more than this amount of temperature data beyond this limit.\n    keep_partial_nan_rows: :any:`bool`, optional\n        If True, keeps data in resultant :any:`pandas.DataFrame` that has\n        missing temperature or meter data. Otherwise, these rows are overwritten\n        entirely with ``numpy.nan`` values.\n\n    Returns\n    -------\n    data : :any:`pandas.DataFrame`\n        A dataset with the specified parameters.\n    \"\"\"\n    if temperature_data.index.freq != \"h\":\n        raise ValueError(\n            \"temperature_data.index must have hourly frequency (freq='H').\"\n            \" Found: {}\".format(temperature_data.index.freq)\n        )\n\n    if not temperature_data.index.tz:\n        raise ValueError(\n            \"temperature_data.index must be timezone-aware. You can set it with\"\n            \" temperature_data.tz_localize(...).\"\n        )\n\n    if meter_data_index.freq is None and meter_data_index.inferred_freq == \"h\":\n        raise ValueError(\n            \"If you have hourly data explicitly set the frequency\"\n            \" of the dataframe by setting\"\n            \"``meter_data_index.freq =\"\n            \" pd.tseries.frequencies.to_offset('H').\"\n        )\n\n    if not meter_data_index.tz:\n        raise ValueError(\n            \"meter_data_index must be timezone-aware. You can set it with\"\n            \" meter_data.tz_localize(...).\"\n        )\n\n    if meter_data_index.duplicated().any():\n        raise ValueError(\"Duplicates found in input meter trace index.\")\n\n    temp_agg_funcs = []\n    temp_agg_column_renames = {}\n\n    if heating_balance_points is None:\n        heating_balance_points = []\n    if cooling_balance_points is None:\n        cooling_balance_points = []\n\n    if meter_data_index.freq is not None:\n        try:\n            freq_timedelta = pd.Timedelta(meter_data_index.freq)\n        except ValueError:  # freq cannot be converted to timedelta\n            freq_timedelta = None\n    else:\n        freq_timedelta = None\n\n    if tolerance is None:\n        tolerance = freq_timedelta\n\n    if not (heating_balance_points == [] and cooling_balance_points == []):\n        if degree_day_method == \"hourly\":\n            pass\n        elif degree_day_method == \"daily\":\n            if meter_data_index.freq == \"h\":\n                raise ValueError(\n                    \"degree_day_method='daily' must be used with\"\n                    \" daily meter data. Found: 'hourly'\".format(degree_day_method)\n                )\n        else:\n            raise ValueError(\"method not supported: {}\".format(degree_day_method))\n\n    if freq_timedelta == pd.Timedelta(\"1h\"):\n        # special fast route for hourly data.\n        df = temperature_data.to_frame(\"temperature_mean\").reindex(meter_data_index)\n\n        if use_mean_daily_values:\n            n_days = 1\n        else:\n            n_days = 1.0 / 24.0\n\n        df = df.assign(\n            **{\n                \"cdd_{}\".format(bp): np.maximum(df.temperature_mean - bp, 0) * n_days\n                for bp in cooling_balance_points\n            }\n        )\n        df = df.assign(\n            **{\n                \"hdd_{}\".format(bp): np.maximum(bp - df.temperature_mean, 0) * n_days\n                for bp in heating_balance_points\n            }\n        )\n        df = df.assign(\n            n_hours_dropped=df.temperature_mean.isnull().astype(int),\n            n_hours_kept=df.temperature_mean.notnull().astype(int),\n        )\n        # TODO(philngo): bad interface or maybe this is just wrong for some reason?\n        if data_quality:\n            df = df.assign(\n                temperature_null=df.n_hours_dropped,\n                temperature_not_null=df.n_hours_kept,\n            )\n        if not temperature_mean:\n            del df[\"temperature_mean\"]\n    else:\n        # daily/billing route\n        # heating/cooling degree day aggregations. Needed for n_days fields as well.\n        temp_agg_funcs.extend(\n            _degree_day_columns(\n                heating_balance_points=heating_balance_points,\n                cooling_balance_points=cooling_balance_points,\n                degree_day_method=degree_day_method,\n                percent_hourly_coverage_per_day=percent_hourly_coverage_per_day,\n                percent_hourly_coverage_per_billing_period=percent_hourly_coverage_per_billing_period,\n                use_mean_daily_values=use_mean_daily_values,\n            )\n        )\n        temp_agg_column_renames.update(\n            {(\"temp\", \"degree_day_columns\"): \"degree_day_columns\"}\n        )\n\n        if data_quality:\n            temp_agg_funcs.extend(\n                [(\"not_null\", \"count\"), (\"null\", lambda x: x.isnull().sum())]\n            )\n            temp_agg_column_renames.update(\n                {\n                    (\"temp\", \"not_null\"): \"temperature_not_null\",\n                    (\"temp\", \"null\"): \"temperature_null\",\n                }\n            )\n\n        if temperature_mean:\n            temp_agg_funcs.extend([(\"mean\", \"mean\")])\n            temp_agg_column_renames.update({(\"temp\", \"mean\"): \"temperature_mean\"})\n\n        # aggregate temperatures\n        temp_df = temperature_data.to_frame(\"temp\")\n        temp_groups = _matching_groups(meter_data_index, temp_df, tolerance)\n        temp_aggregations = temp_groups.agg({\"temp\": temp_agg_funcs})\n\n        # expand temp aggregations by faking and deleting the `meter_value` column.\n        # I haven't yet figured out a way to avoid this and get the desired\n        # structure and behavior. (philngo)\n        meter_value = pd.DataFrame({\"meter_value\": 0}, index=meter_data_index)\n        df = pd.concat([meter_value, temp_aggregations], axis=1).rename(\n            columns=temp_agg_column_renames\n        )\n        del df[\"meter_value\"]\n\n        if \"degree_day_columns\" in df:\n            if df[\"degree_day_columns\"].dropna().empty:\n                column_defaults = {\n                    column: np.full(df[\"degree_day_columns\"].shape, np.nan)\n                    for column in [\"n_days_dropped\", \"n_days_kept\"]\n                }\n                df = df.drop([\"degree_day_columns\"], axis=1).assign(**column_defaults)\n            else:\n                df = pd.concat(\n                    [\n                        df.drop([\"degree_day_columns\"], axis=1),\n                        df[\"degree_day_columns\"].dropna().apply(pd.Series),\n                    ],\n                    axis=1,\n                )\n\n    if not keep_partial_nan_rows:\n        df = overwrite_partial_rows_with_nan(df)\n\n    if df.dropna(how='all').empty:\n        raise ValueError(\"All rows are NaN.\")\n\n    # nan last row\n    df = df.iloc[:-1].reindex(df.index)\n    return df\n\n\ndef _estimate_hour_of_week_occupancy(model_data, threshold):\n    index = pd.CategoricalIndex(range(168))\n    if model_data.dropna().empty:\n        return pd.Series(np.nan, index=index, name=\"occupancy\")\n\n    usage_model = smf.wls(\n        formula=\"meter_value ~ cdd_65 + hdd_50\",\n        data=model_data,\n        weights=model_data.weight,\n    )\n\n    model_data_with_residuals = model_data.merge(\n        pd.DataFrame({\"residuals\": usage_model.fit().resid}),\n        left_index=True,\n        right_index=True,\n    )\n\n    def _is_high_usage(df):\n        if df.empty:\n            return np.nan\n        n_positive_residuals = sum(df.residuals > 0)\n        n_residuals = float(len(df.residuals))\n        ratio_positive_residuals = n_positive_residuals / n_residuals\n        return int(ratio_positive_residuals > threshold)\n\n    return (\n        model_data_with_residuals.groupby([\"hour_of_week\"], observed=False)[\n            [\"residuals\"]\n        ]\n        .apply(_is_high_usage)\n        .rename(\"occupancy\")\n        .reindex(index)\n        .astype(bool)\n    )  # guarantee an index value for all hours\n\n\ndef estimate_hour_of_week_occupancy(data, segmentation=None, threshold=0.65):\n    \"\"\"Estimate occupancy features for each segment.\n\n    Parameters\n    ----------\n    data : :any:`pandas.DataFrame`\n        Input data for the weighted least squares (\"meter_value ~ cdd_65 + hdd_50\")\n        used to estimate occupancy. Must contain meter_value, hour_of_week, cdd_65, and\n        hdd_50 columns with an hourly :any:`pandas.DatetimeIndex`.\n    segmentation : :any:`pandas.DataFrame`, default None\n        A segmentation expressed as a dataframe which shares the timeseries index of\n        the data and has named columns of weights, which are of the form returned by\n        :any:`eemeter.segment_time_series`.\n    threshold : :any:`float`, default 0.65\n        To be marked as unoccupied, the ratio of points with negative residuals in the\n        weighted least squares in a particular hour of week must exceed this threshold.\n        Said another way, in the default case, if more than 35% of values are greater\n        than the basic degree day model for any particular hour of the week, that hour\n        of week is marked as being occupied.\n\n    Returns\n    -------\n    occupancy_lookup : :any:`pandas.DataFrame`\n        The occupancy lookup has a categorical index with values from 0 to 167 - one\n        for each hour of the week, and boolean values indicating an occupied (1, True)\n        or unoccupied (0, False) for each of the segments. Each segment has a column\n        labeled by its segment name.\n    \"\"\"\n\n    occupancy_lookups = {}\n    segmented_datasets = iterate_segmented_dataset(data, segmentation)\n    for segment_name, segmented_data in segmented_datasets:\n        hour_of_week_occupancy = _estimate_hour_of_week_occupancy(\n            segmented_data, threshold\n        )\n        column = \"occupancy\" if segment_name is None else segment_name\n        occupancy_lookups[column] = hour_of_week_occupancy\n    # make sure columns stay in same order\n    columns = [\"occupancy\"] if segmentation is None else segmentation.columns\n    return pd.DataFrame(occupancy_lookups, columns=columns)\n\n\ndef _fit_temperature_bins(temperature_data, default_bins, min_temperature_count):\n    def _compute_temp_summary(bins):\n        bins = [-np.inf] + bins + [np.inf]\n        bin_intervals = [\n            pd.Interval(bin_left, bin_right, closed=\"right\")\n            for bin_left, bin_right in zip(bins, bins[1:])\n        ]\n        temp_bins = pd.cut(temperature_data, bins=bins).cat.set_categories(\n            bin_intervals\n        )\n        return (\n            pd.DataFrame({\"temp\": temperature_data, \"bin\": temp_bins})\n            .groupby(\"bin\", observed=False)[\"temp\"]\n            .count()\n            .rename(\"count\")\n            .sort_index()\n        )\n\n    def _find_endpoints_to_remove(temp_summary):\n        if len(temp_summary) == 1:\n            return set()\n\n        def _bin_count_invalid(i):\n            count = temp_summary.iloc[i]\n            return count < min_temperature_count or np.isnan(count)\n\n        # work from outside in assuming less density at distribution edges\n        endpoints = set()\n\n        if _bin_count_invalid(0):  # first\n            endpoints.add(temp_summary.index[0].right)\n\n        if _bin_count_invalid(-1):  # last\n            endpoints.add(temp_summary.index[-1].left)\n\n        if len(endpoints) == 0:\n            # try points in middle\n            for i in range(1, len(temp_summary) - 1):\n                if _bin_count_invalid(i):\n                    endpoints.add(temp_summary.index[i].right)\n\n        return endpoints\n\n    test_bins = set(default_bins)\n\n    while True:\n        temp_summary = _compute_temp_summary(sorted(test_bins))\n        endpoints_to_remove = _find_endpoints_to_remove(temp_summary)\n\n        if len(endpoints_to_remove) == 0:\n            break\n        for endpoint in endpoints_to_remove:\n            test_bins.discard(endpoint)\n\n    return sorted(test_bins)\n\n\ndef fit_temperature_bins(\n    data,\n    segmentation=None,\n    occupancy_lookup=None,\n    default_bins=[30, 45, 55, 65, 75, 90],\n    min_temperature_count=20,\n):\n    \"\"\"Determine appropriate temperature bins for a particular set of temperature\n    data given segmentation and occupancy.\n\n    Parameters\n    ----------\n    data : :any:`pandas.Series`\n        Input temperature data with an hourly :any:`pandas.DatetimeIndex`\n    segmentation : :any:`pandas.DataFrame`, default None\n        A dataframe containing segment weights with one column per segment. If\n        left off, segmentation will not be considered.\n    occupancy_lookup : :any:`pandas.DataFrame`, default None\n        A dataframe of the form returned by :any:`eemeter.estimate_hour_of_week_occupancy`\n        containing occupancy for each segment. If None, occupancy will not be\n        considered.\n    default_bins : :any:`list` of :any:`float` or :any:`int`\n        A list of candidate bin endpoints to begin the search with.\n    min_temperature_count : :any:`int`\n        The minimum number of temperatre values that must be included in any bin. If\n        this threshold is not met, bins are dropped from the outside in following the\n        algorithm described in the CalTRACK documentation.\n\n    Returns\n    -------\n    temperature_bins : :any:`pandas.DataFrame` or, if occupancy_lookup is provided a\n    two :any:`tuple` of :any:`pandas.DataFrame`\n        A dataframe with boolean values indicating whether or not a bin was kept, with a\n        categorical index for each candidate bin endpoint and a column for each segment.\n    \"\"\"\n\n    if occupancy_lookup is None:\n        segmented_bins = {}\n        segmented_datasets = iterate_segmented_dataset(data, segmentation)\n        for segment_name, segmented_data in segmented_datasets:\n            segmented_bins[segment_name] = _fit_temperature_bins(\n                segmented_data.temperature_mean, default_bins, min_temperature_count\n            )\n\n        if segmentation is None:\n            bins = segmented_bins[None]\n            return pd.DataFrame(\n                {\"keep_bin_endpoint\": [endpoint in bins for endpoint in default_bins]},\n                index=pd.Series(default_bins, name=\"bin_endpoints\"),\n            )\n\n        return pd.DataFrame(\n            {\n                segment_name: [endpoint in bins for endpoint in default_bins]\n                for segment_name, bins in segmented_bins.items()\n            },\n            columns=segmentation.columns,\n            index=pd.Series(default_bins, name=\"bin_endpoints\"),\n        )\n    else:\n        occupied_segmented_bins = {}\n        unoccupied_segmented_bins = {}\n        segmented_datasets = iterate_segmented_dataset(data, segmentation)\n        for segment_name, segmented_data in segmented_datasets:\n            hourly_segmented_data = segmented_data.resample(\"h\").mean(numeric_only=True)\n            time_features = compute_time_features(\n                hourly_segmented_data.index,\n                hour_of_week=True,\n                day_of_week=False,\n                hour_of_day=False,\n            )\n            if segment_name is None:\n                occupancy = occupancy_lookup[\"occupancy\"]\n            else:\n                occupancy = occupancy_lookup[segment_name]\n            occupancy_features = compute_occupancy_feature(\n                time_features.hour_of_week, occupancy\n            )\n            occupied_temperatures = segmented_data.temperature_mean[occupancy_features]\n            unoccupied_temperatures = segmented_data.temperature_mean[\n                ~occupancy_features\n            ]\n            occupied_segmented_bins[segment_name] = _fit_temperature_bins(\n                occupied_temperatures, default_bins, min_temperature_count\n            )\n            unoccupied_segmented_bins[segment_name] = _fit_temperature_bins(\n                unoccupied_temperatures, default_bins, min_temperature_count\n            )\n\n        if segmentation is None:\n            occupied_bins = occupied_segmented_bins[None]\n            unoccupied_bins = unoccupied_segmented_bins[None]\n            return (\n                pd.DataFrame(\n                    {\n                        \"keep_bin_endpoint\": [\n                            endpoint in occupied_bins for endpoint in default_bins\n                        ]\n                    },\n                    index=pd.Series(default_bins, name=\"bin_endpoints\"),\n                ),\n                pd.DataFrame(\n                    {\n                        \"keep_bin_endpoint\": [\n                            endpoint in unoccupied_bins for endpoint in default_bins\n                        ]\n                    },\n                    index=pd.Series(default_bins, name=\"bin_endpoints\"),\n                ),\n            )\n\n        return (\n            pd.DataFrame(\n                {\n                    segment_name: [endpoint in bins for endpoint in default_bins]\n                    for segment_name, bins in occupied_segmented_bins.items()\n                },\n                columns=segmentation.columns,\n                index=pd.Series(default_bins, name=\"bin_endpoints\"),\n            ),\n            pd.DataFrame(\n                {\n                    segment_name: [endpoint in bins for endpoint in default_bins]\n                    for segment_name, bins in unoccupied_segmented_bins.items()\n                },\n                columns=segmentation.columns,\n                index=pd.Series(default_bins, name=\"bin_endpoints\"),\n            ),\n        )\n\n\n# TODO(philngo): combine with compute_temperature_features?\ndef compute_temperature_bin_features(temperatures, bin_endpoints):\n    \"\"\"Compute temperature bin features.\n\n    Parameters\n    ----------\n    temperatures : :any:`pandas.Series`\n        Hourly temperature data.\n    bin_endpoints : :any:`list` of :any:`int` or :any:`float`\n        List of bin endpoints to use when assigning features.\n\n    Returns\n    -------\n    temperature_bin_features : :any:`pandas.DataFrame`\n        A datafame with the input index and one column per bin. The sum of each\n        row (with all of the temperature bins) equals the input temperature. More\n        details on this bin feature are available in the CalTRACK documentation.\n    \"\"\"\n    bin_endpoints = [-np.inf] + bin_endpoints + [np.inf]\n\n    bins = {}\n\n    for i, (left_bin, right_bin) in enumerate(zip(bin_endpoints, bin_endpoints[1:])):\n        bin_name = \"bin_{}\".format(i)\n\n        in_bin = (temperatures > left_bin) & (temperatures <= right_bin)\n        gt_bin = temperatures > right_bin\n\n        not_in_bin_index = temperatures.index[~in_bin]\n        gt_bin_index = temperatures.index[gt_bin]\n\n        def _expand_and_fill(partial_temp_series):\n            return partial_temp_series.reindex(temperatures.index, fill_value=0)\n\n        def _mask_nans(temp_series):\n            return temp_series[temperatures.notnull()].reindex(temperatures.index)\n\n        if i == 0:\n            temps_in_bin = _expand_and_fill(temperatures[in_bin])\n            temps_out_of_bin = _expand_and_fill(\n                pd.Series(right_bin, index=not_in_bin_index)\n            )\n            bin_values = temps_in_bin + temps_out_of_bin\n        else:\n            temps_in_bin = _expand_and_fill(temperatures[in_bin] - left_bin)\n            temps_gt_bin = _expand_and_fill(\n                pd.Series(right_bin - left_bin, index=gt_bin_index)\n            )\n            bin_values = temps_in_bin + temps_gt_bin\n        bins[bin_name] = _mask_nans(bin_values)\n    return pd.DataFrame(bins)\n\n\ndef compute_occupancy_feature(hour_of_week, occupancy):\n    \"\"\"Given an hour of week feature, determine the occupancy for that hour of week.\n\n    Parameters\n    ----------\n    hour_of_week : :any:`pandas.Series`\n        Hour of week feature as given by :any:`eemeter.compute_time_features`.\n    occupancy : :any:`pandas.Series`\n        Boolean occupancy assignents for each hour of week as determined by\n        :any:`eemeter.estimate_hour_of_week_occupancy`\n\n    Returns\n    -------\n    occupancy_feature : :any:`pandas.Series`\n        Occupancy labels for the timeseries.\n    \"\"\"\n    return pd.merge(\n        hour_of_week.dropna().to_frame(),\n        occupancy.to_frame(\"occupancy\"),\n        how=\"left\",\n        left_on=\"hour_of_week\",\n        right_index=True,\n    ).occupancy.reindex(hour_of_week.index)\n"
  },
  {
    "path": "opendsm/eemeter/common/sufficiency_criteria.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom typing import Literal\n\nimport numpy as np\nimport pandas as pd\nimport pytz\n\nimport pydantic\n\nfrom opendsm.common.base_settings import BaseSettings\nfrom opendsm.common.pydantic_utils import computed_field_cached_property\n\nfrom opendsm.eemeter.common.data_processor_utilities import day_counts\nfrom opendsm.eemeter.common.data_settings import BaseSufficiencySettings\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\n# TODO implement as registered functions rather than needing to call everything manually\n# probably easiest to use two decorators, can be stacked, for baseline/reporting\n\n\nclass SufficiencyCriteria(BaseSettings):\n    model_config = pydantic.ConfigDict(\n        frozen = False,\n        arbitrary_types_allowed=True,\n        str_to_lower = True,\n        str_strip_whitespace = True,\n    )\n\n    data: pd.DataFrame\n    is_electricity_data: bool\n    is_reporting_data: bool\n    settings: BaseSufficiencySettings\n\n    _n_valid_observed_days = None\n    _n_valid_days = None\n    _n_valid_temperature_days = None\n\n    disqualification: list = []\n    warnings: list = []\n\n    @computed_field_cached_property()\n    def _has_ghi(self) -> bool:\n        return \"ghi\" in self.data.columns\n\n    @computed_field_cached_property()\n    def n_days_total(self) -> int:\n        requested_start = self.settings.requested_start\n        requested_end = self.settings.requested_end\n\n        data_end = self.data.dropna().index.max()\n        data_start = self.data.dropna().index.min()\n        n_days_data = (\n            data_end - data_start\n        ).days + 1  # TODO confirm. no longer using last row nan\n\n        if requested_start is not None:\n            # check for gap at beginning\n            requested_start = requested_start.astimezone(pytz.UTC)\n            n_days_start_gap = (data_start - requested_start).days\n        else:\n            n_days_start_gap = 0\n\n        if requested_end is not None:\n            # check for gap at end\n            requested_end = requested_end.astimezone(pytz.UTC)\n            n_days_end_gap = (requested_end - data_end).days\n        else:\n            n_days_end_gap = 0\n\n        n_days_total = n_days_data + n_days_start_gap + n_days_end_gap\n\n        return n_days_total\n\n    def _compute_valid_day_counts(self):\n        min_pct = self.settings.temperature.min_pct_period_coverage\n        valid_temperature_rows = (\n            self.data.temperature_not_null\n            / (self.data.temperature_not_null + self.data.temperature_null)\n        ) > min_pct\n\n        # get number of days per period - for daily this should be a series of ones\n        row_day_counts = day_counts(self.data.index)\n\n        # get valid rows\n        valid_rows = valid_temperature_rows\n\n        if not self.is_reporting_data:\n            valid_observed_rows = self.data.observed.notnull()\n            valid_rows = valid_rows & valid_observed_rows\n\n            n_valid_observed_days = (valid_observed_rows * row_day_counts).sum()\n            self._n_valid_observed_days = int(n_valid_observed_days)\n        else:\n            self._n_valid_observed_days = None\n\n        self._n_valid_temperature_days = int((valid_temperature_rows * row_day_counts).sum())\n        self._n_valid_days = int((valid_rows * row_day_counts).sum())\n    \n    @computed_field_cached_property()\n    def n_valid_temperature_days(self) -> int:\n        if self._n_valid_temperature_days is None:\n            self._compute_valid_day_counts()\n\n        return self._n_valid_temperature_days\n\n    @computed_field_cached_property()\n    def n_valid_observed_days(self) -> int:\n        if self._n_valid_observed_days is None:\n            self._compute_valid_day_counts()\n\n        return self._n_valid_observed_days\n\n    @computed_field_cached_property()\n    def n_valid_days(self) -> int:\n        if self._n_valid_days is None:\n            self._compute_valid_day_counts()\n\n        return self._n_valid_days\n\n    def _check_no_data(self):\n        if self.data.dropna().empty:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.no_data\",\n                    description=(\"No data available.\"),\n                    data={},\n                )\n            )\n            return False\n        return True\n\n    def _check_n_days_boundary_gap(self, gap_type: Literal[\"start\", \"end\"]):\n        gap = 0\n        if gap_type == \"start\":\n            user_boundary = self.settings.requested_start\n            \n            if user_boundary is not None:\n                data_boundary = self.data.dropna().index.min()\n                gap = (data_boundary - user_boundary).days\n        else:\n            if user_boundary is not None:\n                data_boundary = self.data.dropna().index.max()\n                gap = (user_boundary - data_boundary).days\n\n        if gap < 0:\n            # CalTRACK 2.2.4\n            if gap_type == \"start\":\n                err = \"before\"\n            else:\n                err = \"after\"\n\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=(\n                        \"eemeter.sufficiency_criteria\"\n                        f\".extra_data_{err}_requested_{gap_type}_date\"\n                    ),\n                    description=(f\"Extra data found {err} requested {gap_type} date.\"),\n                    data={\n                        f\"requested_{gap_type}\": user_boundary.isoformat(),\n                        f\"data_{gap_type}\": data_boundary.isoformat(),\n                    },\n                )\n            )\n\n    def _check_baseline_day_length(self):\n        min_length = self.settings.min_baseline_length\n        max_length = self.settings.max_baseline_length\n\n        if self.is_reporting_data:\n            return\n        \n        if self.n_days_total < min_length or self.n_days_total > max_length:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=(\n                        \"eemeter.sufficiency_criteria\" \".incorrect_number_of_total_days\"\n                    ),\n                    description=(\n                        f\"Baseline length is not within the expected range of {min_length}-{max_length} days.\"\n                    ),\n                    data={\"n_days_total\": self.n_days_total},\n                )\n            )\n\n    def _check_negative_observed_values(self):\n        if self.is_reporting_data:\n            return\n        elif self.is_electricity_data:\n            return\n\n        n_negative_observed_values = self.data.observed[self.data.observed < 0].shape[0]\n\n        if n_negative_observed_values > 0:\n            # CalTrack 2.3.5\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=(\n                        \"eemeter.sufficiency_criteria\" \".negative_observed_values\"\n                    ),\n                    description=(\"Found negative Observed values\"),\n                    data={\"n_negative_observed_values\": n_negative_observed_values},\n                )\n            )\n    \n    def _check_valid_days_percentage(self, col: Literal[\"temperature\", \"ghi\", \"observed\", \"joint\"]):\n        if self.is_reporting_data and col == \"observed\":\n            return\n        elif col == \"ghi\" and not self._has_ghi:\n            return\n        \n        n_days_total = float(self.n_days_total)\n\n        if col == \"temperature\":\n            name = col.capitalize()\n            valid_days = self.n_valid_temperature_days\n            min_pct = self.settings.temperature.min_pct_daily_coverage\n        elif col == \"ghi\":\n            name = col.upper()\n            raise NotImplementedError(\"GHI valid days percentage check not implemented yet\")\n            valid_days = self.n_valid_ghi_days\n            min_pct = self.settings.ghi.min_pct_daily_coverage\n        elif col == \"observed\":\n            name = col.capitalize()\n            valid_days = self.n_valid_observed_days\n            min_pct = self.settings.observed.min_pct_daily_coverage\n        elif col == \"joint\":\n            name = col.capitalize()\n            valid_days = self.n_valid_days\n            min_pct = self.settings.joint.min_pct_daily_coverage\n\n        valid_pct = 0\n        if n_days_total > 0:\n            valid_pct = valid_days / n_days_total\n\n        if valid_pct < min_pct:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=(\n                        \"eemeter.sufficiency_criteria\"\n                        f\".too_many_days_with_missing_{col}_data\"\n                    ),\n                    description=(\n                        f\"Too many days in data have missing {name} data.\"\n                    ),\n                    data={\n                        f\"n_valid_{col}_data_days\": valid_days,\n                        \"n_days_total\": n_days_total,\n                    },\n                )\n            )\n\n    def _check_valid_monthly_coverage(self, col: Literal[\"temperature\", \"ghi\", \"observed\", \"joint\"]):\n        if self.is_reporting_data and col == \"observed\":\n            return\n        elif col == \"ghi\" and not self._has_ghi:\n            return\n\n        if col == \"temperature\":\n            name = col.capitalize()\n            min_pct = self.settings.temperature.min_pct_monthly_coverage\n        elif col == \"ghi\":\n            name = col.upper()\n            min_pct = self.settings.ghi.min_pct_monthly_coverage\n        elif col == \"observed\":\n            name = col.capitalize()\n            min_pct = self.settings.observed.min_pct_monthly_coverage\n        elif col == \"joint\":\n            name = col.capitalize()\n            raise NotImplementedError(\"Joint monthly coverage check not implemented yet\")\n            min_pct = self.settings.joint.min_pct_monthly_coverage\n        \n        non_null_pct_per_month = (\n            self.data[col]\n            .groupby(self.data.index.month)\n            .apply(lambda x: x.notna().mean())\n        )\n\n        if (non_null_pct_per_month < min_pct).any():\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=f\"eemeter.sufficiency_criteria.missing_monthly_{col}_data\",\n                    description=(\n                        f\"More than {(1-min_pct)*100}% of the monthly {name} data is missing.\"\n                    ),\n                    data={\n                        \"lowest_monthly_coverage\": non_null_pct_per_month.min(),\n                    },\n                )\n            )\n\n    def _check_season_weekday_weekend_availability(self):\n        raise NotImplementedError(\n            \"90% of season and weekday/weekend check not implemented yet\"\n        )\n\n    def _check_extreme_values(self):\n        if self.is_reporting_data:\n            return    \n        elif self.data[\"observed\"].dropna().empty:\n            return\n        \n        if not self.is_reporting_data:\n            median = self.data.observed.median()\n            lower_quantile = self.data.observed.quantile(0.25)\n            upper_quantile = self.data.observed.quantile(0.75)\n            iqr = upper_quantile - lower_quantile\n            lower_bound = lower_quantile - (3 * iqr)\n            upper_bound = upper_quantile + (3 * iqr)\n            n_extreme_values = self.data.observed[\n                (self.data.observed < lower_bound) | (self.data.observed > upper_bound)\n            ].shape[0]\n            min_value = float(self.data.observed.min())\n            max_value = float(self.data.observed.max())\n\n            if n_extreme_values > 0:\n                # Inspired by CalTRACK 2.3.6\n                self.warnings.append(\n                    EEMeterWarning(\n                        qualified_name=(\n                            \"eemeter.sufficiency_criteria\" \".extreme_values_detected\"\n                        ),\n                        description=(\n                            \"Extreme values (outside 3x IQR) must be flagged for manual review.\"\n                        ),\n                        data={\n                            \"n_extreme_values\": n_extreme_values,\n                            \"median\": median,\n                            \"upper_quantile\": upper_quantile,\n                            \"lower_quantile\": lower_quantile,\n                            \"lower_bound\": lower_bound,\n                            \"upper_bound\": upper_bound,\n                            \"min_value\": min_value,\n                            \"max_value\": max_value,\n                        },\n                    )\n                )\n\n    def _check_high_frequency_temperature_values(self):\n        # TODO broken as written\n        # If high frequency data check for 50% data coverage in rollup\n        min_pct = self.settings.temperature.min_pct_hourly_coverage\n        if len(temperature_features[temperature_features.coverage <= min_pct]) > 0:\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\",\n                    description=(\n                        f\"More than {(1-min_pct)*100}% of the high frequency Temperature data is missing.\"\n                    ),\n                    data={\n                        \"high_frequency_data_missing_count\": len(\n                            temperature_features[\n                                temperature_features.coverage <= min_pct\n                            ].index.to_list()\n                        )\n                    },\n                )\n            )\n\n        # Set missing high frequency data to NaN\n        temperature_features.value[temperature_features.coverage > min_pct] = (\n            temperature_features[temperature_features.coverage > min_pct].value\n            / temperature_features[temperature_features.coverage > min_pct].coverage\n        )\n\n        temperature_features = (\n            temperature_features[temperature_features.coverage > min_pct]\n            .reindex(temperature_features.index)[[\"value\"]]\n            .rename(columns={\"value\": \"temperature_mean\"})\n        )\n\n        if \"coverage\" in temperature_features.columns:\n            temperature_features = temperature_features.drop(columns=[\"coverage\"])\n\n    def _check_high_frequency_observed_values(self):\n        min_pct = self.settings.observed.min_pct_hourly_coverage\n        if not self.data[self.data.coverage <= min_pct].empty:\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_observed_data\",\n                    description=(\n                        f\"More than {(1-min_pct)*100}% of the high frequency Observed data is missing.\"\n                    ),\n                    data=(self.data[self.data.coverage <= min_pct].index.to_list()),\n                )\n            )\n\n        # CalTRACK 2.2.2.1 - interpolate with average of non-null values\n        self.data.value[self.data.coverage > min_pct] = (\n            self.data[self.data.coverage > min_pct].value\n            / self.data[self.data.coverage > min_pct].coverage\n        )\n\n    def check_sufficiency_baseline(self):\n        self._check_no_data()\n        self._check_baseline_day_length()\n        self._check_negative_observed_values()\n\n        self._check_valid_days_percentage(col=\"temperature\")\n        self._check_valid_days_percentage(col=\"observed\")\n        self._check_valid_days_percentage(col=\"joint\")\n        self._check_valid_monthly_coverage(col=\"temperature\")\n\n        self._check_extreme_values()\n\n    def check_sufficiency_reporting(self):\n        self._check_no_data()\n\n        self._check_valid_days_percentage(col=\"temperature\")\n        self._check_valid_days_percentage(col=\"joint\")\n        self._check_valid_monthly_coverage(col=\"temperature\")\n        # self._check_high_frequency_temperature_values()\n\n\nclass DailySufficiencyCriteria(SufficiencyCriteria):\n    \"\"\"\n    Sufficiency Criteria class for daily models\n    \"\"\"\n\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n\n    def check_sufficiency_baseline(self):\n        super().check_sufficiency_baseline()\n\n        # self._check_n_days_boundary_gap(\"start\")\n        # self._check_n_days_boundary_gap(\"end\")\n        # TODO : Maybe make these checks static? To work with the current data class\n        # self._check_high_frequency_meter_values()\n        # self._check_high_frequency_temperature_values()\n\n    def check_sufficiency_reporting(self):\n        super().check_sufficiency_reporting()\n\n\nclass BillingSufficiencyCriteria(SufficiencyCriteria):\n    \"\"\"\n    Sufficiency Criteria class for billing models - monthly / bimonthly\n    \"\"\"\n\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n\n    def _check_observed_data_billing_monthly(self):\n        if self.data[\"value\"].dropna().empty:\n            return\n\n        diff = list((data.index[1:] - data.index[:-1]).days)\n        filter_ = pd.Series(diff + [np.nan], index=data.index)\n\n        min_days = self.settings.min_days_in_period\n        max_days = self.settings.max_days_in_monthly_period\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        # Billing Monthly data frequency check\n        data = data[(min_days <= filter_) & (filter_ <= max_days)].reindex(  # keep these, inclusive\n            data.index\n        )\n\n        if len(data[(max_days < filter_) | (filter_ < min_days)]) > 0:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.offcycle_reads_in_billing_monthly_data\",\n                    description=(\n                        f\"Off-cycle reads found in billing monthly data having a duration less than {min_days} days or greater than {max_days} days\"\n                    ),\n                    data=(data[(max_days < filter_) | (filter_ < min_days)].index.to_list()),\n                )\n            )\n\n    def _check_observed_data_billing_bimonthly(self):\n        if self.data[\"value\"].dropna().empty:\n            return\n\n        diff = list((data.index[1:] - data.index[:-1]).days)\n        filter_ = pd.Series(diff + [np.nan], index=data.index)\n\n        min_days = self.settings.min_days_in_period\n        max_days = self.settings.max_days_in_bimonthly_period\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        data = data[(min_days <= filter_) & (filter_ <= max_days)].reindex(  # keep these, inclusive\n            data.index\n        )\n\n        if len(data[(max_days < filter_) | (filter_ < min_days)]) > 0:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.offcycle_reads_in_billing_bimonthly_data\",\n                    description=(\n                        f\"Off-cycle reads found in billing bimonthly data having a duration less than {min_days} days or greater than {max_days} days\"\n                    ),\n                    data=(data[(max_days < filter_) | (filter_ < min_days)].index.to_list()),\n                )\n            )\n\n    def _check_estimated_observed_values(self):\n        # CalTRACK 2.2.3.1\n        \"\"\"\n        Adds estimate to subsequent read if there aren't more than one estimate in a row\n        and then removes the estimated row.\n\n        Input:\n        index   value   estimated\n        1       2       False\n        2       3       False\n        3       5       True\n        4       4       False\n        5       6       True\n        6       3       True\n        7       4       False\n        8       NaN     NaN\n\n        Output:\n        index   value\n        1       2\n        2       3\n        4       9\n        5       NaN\n        7       7\n        8       NaN\n        \"\"\"\n        add_estimated = []\n        remove_estimated_fixed_rows = []\n        data = self.data\n        if \"estimated\" in data.columns:\n            data[\"unestimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated == False)].reindex(data.index)\n            )\n            data[\"estimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated)].reindex(data.index)\n            )\n            for i, (index, row) in enumerate(data[:-1].iterrows()):\n                # ensures there is a prev_row and previous row value is null\n                if i > 0 and pd.isnull(prev_row[\"unestimated_value\"]):\n                    # current row value is not null\n                    add_estimated.append(prev_row[\"estimated_value\"])\n                    if not pd.isnull(row[\"unestimated_value\"]):\n                        # get all rows that had only estimated reads that will be\n                        # added to the subsequent row meaning this row\n                        # needs to be removed\n                        remove_estimated_fixed_rows.append(prev_index)\n                else:\n                    add_estimated.append(0)\n                prev_row = row\n                prev_index = index\n            add_estimated.append(np.nan)\n            data[\"value\"] = data[\"unestimated_value\"] + add_estimated\n            data = data[~data.index.isin(remove_estimated_fixed_rows)]\n            data = data[[\"value\"]]  # remove the estimated column\n\n    def check_sufficiency_baseline(self):\n        super().check_sufficiency_baseline()\n\n        # self._check_n_days_boundary_gap(\"start\")\n        # self._check_n_days_boundary_gap(\"end\")\n        # if self.median_granularity == \"billing_monthly\":\n        #     self._check_observed_data_billing_monthly()\n        # else :\n        #     self._check_observed_data_billing_bimonthly()\n        self._check_estimated_observed_values()\n        # self._check_high_frequency_temperature_values()\n\n    def check_sufficiency_reporting(self):\n        super().check_sufficiency_reporting()\n\n\nclass HourlySufficiencyCriteria(SufficiencyCriteria):\n    \"\"\"\n    Sufficiency Criteria class for hourly models\n    \"\"\"\n\n    def __init__(self, *args, **kwargs):\n        super().__init__(*args, **kwargs)\n\n    def _check_baseline_length_hourly_model(self):\n        pass\n\n    def _check_hourly_consecutive_temperature_data(self):\n        # TODO : Check implementation wrt Caltrack 2.2.4.1\n        # Resample to hourly by taking the first non NaN value\n        hourly_data = self.data[\"temperature\"].resample(\"H\").first()\n        mask = hourly_data.isna().any(axis=1)\n        grouped = mask.groupby((mask != mask.shift()).cumsum())\n        max_consecutive_nans = grouped.sum().max()\n        allowed_consecutive_nans = self.settings.temperature.max_consecutive_hours_missing\n        if max_consecutive_nans > allowed_consecutive_nans:\n            self.disqualification.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.too_many_consecutive_hours_temperature_data_missing\",\n                    description=(\n                        f\"More than {allowed_consecutive_nans} hours of consecutive hourly Temperature data is missing.\"\n                    ),\n                    data={\"Max_consecutive_hours_missing\": int(max_consecutive_nans)},\n                )\n            )\n\n    def check_sufficiency_baseline(self):\n        super().check_sufficiency_baseline()\n        \n        # TODO : add caltrack check number on top of each method\n        # self._check_n_days_boundary_gap(\"start\")\n        # self._check_n_days_boundary_gap(\"end\")\n        self._check_valid_monthly_coverage(col=\"ghi\")\n        self._check_valid_monthly_coverage(col=\"observed\")\n        # TODO these will only apply to legacy, and currently do not work\n        # self._check_high_frequency_observed_values()\n        # self._check_high_frequency_temperature_values()\n        # self._check_hourly_consecutive_temperature_data()\n\n    def check_sufficiency_reporting(self):\n        super().check_sufficiency_reporting()\n        \n        self._check_valid_monthly_coverage(col=\"ghi\")"
  },
  {
    "path": "opendsm/eemeter/common/transform.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom datetime import timedelta\n\nimport numpy as np\nimport pandas as pd\nfrom pandas.tseries.offsets import MonthEnd\nimport pytz\n\nfrom .exceptions import NoBaselineDataError, NoReportingDataError\nfrom .warnings import EEMeterWarning\n\n__all__ = (\n    \"Term\",\n    \"as_freq\",\n    \"day_counts\",\n    \"get_baseline_data\",\n    \"get_reporting_data\",\n    \"get_terms\",\n    \"remove_duplicates\",\n    \"overwrite_partial_rows_with_nan\",\n    \"clean_caltrack_billing_data\",\n    \"clean_caltrack_billing_daily_data\",\n    \"add_freq\",\n    \"trim\",\n    \"format_energy_data_for_caltrack\",\n    \"format_temperature_data_for_caltrack\",\n)\n\n\ndef overwrite_partial_rows_with_nan(df):\n    return df.dropna().reindex(df.index)\n\n\ndef remove_duplicates(df_or_series):\n    \"\"\"Remove duplicate rows or values by keeping the first of each duplicate.\n\n    Parameters\n    ----------\n    df_or_series : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        Pandas object from which to drop duplicate index values.\n\n    Returns\n    -------\n    deduplicated : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        The deduplicated pandas object.\n    \"\"\"\n    # CalTrack 2.3.2.2\n    return df_or_series[~df_or_series.index.duplicated(keep=\"first\")]\n\n\ndef as_freq(\n    data_series,\n    freq,\n    atomic_freq=\"1 Min\",\n    series_type=\"cumulative\",\n    include_coverage=False,\n):\n    \"\"\"Resample data to a different frequency.\n\n    This method can be used to upsample or downsample meter data. The\n    assumption it makes to do so is that meter data is constant and averaged\n    over the given periods. For instance, to convert billing-period data to\n    daily data, this method first upsamples to the atomic frequency\n    (1 minute freqency, by default), \"spreading\" usage evenly across all\n    minutes in each period. Then it downsamples to hourly frequency and\n    returns that result. With instantaneous series, the data is copied to all\n    contiguous time intervals and the mean over `freq` is returned.\n\n    **Caveats**:\n\n     - This method gives a fair amount of flexibility in\n       resampling as long as you are OK with the assumption that usage is\n       constant over the period (this assumption is generally broken in\n       observed data at large enough frequencies, so this caveat should not be\n       taken lightly).\n\n    Parameters\n    ----------\n    data_series : :any:`pandas.Series`\n        Data to resample. Should have a :any:`pandas.DatetimeIndex`.\n    freq : :any:`str`\n        The frequency to resample to. This should be given in a form recognized\n        by the :any:`pandas.Series.resample` method.\n    atomic_freq : :any:`str`, optional\n        The \"atomic\" frequency of the intermediate data form. This can be\n        adjusted to a higher atomic frequency to increase speed or memory\n        performance.\n    series_type : :any:`str`, {'cumulative', ‘instantaneous’},\n        default 'cumulative'\n        Type of data sampling. 'cumulative' data can be spread over smaller\n        time intervals and is aggregated using addition (e.g. meter data).\n        'instantaneous' data is copied (not spread) over smaller time intervals\n        and is aggregated by averaging (e.g. weather data).\n    include_coverage: :any:`bool`,\n        default `False`\n        Option of whether to return a series with just the resampled values\n        or a dataframe with a column that includes percent coverage of source data\n        used for each sample.\n\n    Returns\n    -------\n    resampled_data : :any:`pandas.Series` or :any:`pandas.DataFrame`\n        Data resampled to the given frequency (optionally as a dataframe with a coverage column if `include_coverage` is used.\n    \"\"\"\n    # TODO(philngo): make sure this complies with CalTRACK 2.2.2.1\n    if not isinstance(data_series, pd.Series):\n        raise ValueError(\n            \"expected series, got object with class {}\".format(data_series.__class__)\n        )\n    if data_series.empty:\n        return data_series\n    series = remove_duplicates(data_series)\n    target_freq = pd.Timedelta(atomic_freq)\n    timedeltas = (series.index[1:] - series.index[:-1]).append(\n        pd.TimedeltaIndex([pd.NaT])\n    )\n\n    if series_type == \"cumulative\":\n        spread_factor = target_freq.total_seconds() / timedeltas.total_seconds()\n        series_spread = series * spread_factor\n        atomic_series = series_spread.asfreq(atomic_freq, method=\"ffill\")\n        resampled = atomic_series.resample(freq).sum()\n        resampled_with_nans = atomic_series.resample(freq).first()\n        n_coverage = atomic_series.resample(freq).count()\n        resampled = resampled[resampled_with_nans.notnull()].reindex(resampled.index)\n\n    elif series_type == \"instantaneous\":\n        atomic_series = series.asfreq(atomic_freq, method=\"ffill\")\n        resampled = atomic_series.resample(freq).mean()\n\n    if resampled.index[-1] < series.index[-1]:\n        # this adds a null at the end using the target frequency\n        last_index = pd.date_range(resampled.index[-1], freq=freq, periods=2)[1:]\n        resampled = (\n            pd.concat([resampled, pd.Series(np.nan, index=last_index)])\n            .resample(freq)\n            .mean()\n        )\n    if include_coverage:\n        n_total = resampled.resample(atomic_freq).count().resample(freq).count()\n        resampled = resampled.to_frame(\"value\")\n        resampled[\"coverage\"] = n_coverage / n_total\n        return resampled\n    else:\n        return resampled\n\n\ndef day_counts(index):\n    \"\"\"Days between DatetimeIndex values as a :any:`pandas.Series`.\n\n    Parameters\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        The index for which to get day counts.\n\n    Returns\n    -------\n    day_counts : :any:`pandas.Series`\n        A :any:`pandas.Series` with counts of days between periods. Counts are\n        given on start dates of periods.\n    \"\"\"\n    # dont affect the original data\n    index = index.copy()\n\n    if len(index) == 0:\n        return pd.Series([], index=index)\n\n    timedeltas = (index[1:] - index[:-1]).append(pd.TimedeltaIndex([pd.NaT]))\n    timedelta_days = timedeltas.total_seconds() / (60 * 60 * 24)\n\n    return pd.Series(timedelta_days, index=index)\n\n\ndef _make_baseline_warnings(\n    end_inf, start_inf, data_start, data_end, start_limit, end_limit\n):\n    warnings = []\n    # warn if there is a gap at end\n    if not end_inf and data_end < end_limit:\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.get_baseline_data.gap_at_baseline_end\",\n                description=(\n                    \"Data does not have coverage at requested baseline end date.\"\n                ),\n                data={\n                    \"requested_end\": end_limit.isoformat(),\n                    \"data_end\": data_end.isoformat(),\n                },\n            )\n        )\n    # warn if there is a gap at start\n    if not start_inf and start_limit < data_start:\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.get_baseline_data.gap_at_baseline_start\",\n                description=(\n                    \"Data does not have coverage at requested baseline start date.\"\n                ),\n                data={\n                    \"requested_start\": start_limit.isoformat(),\n                    \"data_start\": data_start.isoformat(),\n                },\n            )\n        )\n    return warnings\n\n\ndef get_baseline_data(\n    data,\n    start=None,\n    end=None,\n    max_days=365,\n    allow_billing_period_overshoot=False,\n    n_days_billing_period_overshoot=None,\n    ignore_billing_period_gap_for_day_count=False,\n):\n    \"\"\"Filter down to baseline period data.\n\n    .. note::\n\n        For compliance with CalTRACK, set ``max_days=365`` (section 2.2.1.1).\n\n    Parameters\n    ----------\n    data : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        The data to filter to baseline data. This data will be filtered down\n        to an acceptable baseline period according to the dates passed as\n        `start` and `end`, or the maximum period specified with `max_days`.\n    start : :any:`datetime.datetime`\n        A timezone-aware datetime that represents the earliest allowable moment\n        for the baseline data. The stricter of this or `max_days` is used\n        to determine the earliest allowable baseline period timestamp.\n    end : :any:`datetime.datetime`\n        A timezone-aware datetime that represents the latest allowable end\n        moment for the baseline data, i.e., the latest moment for which data is\n        available before the intervention begins.\n    max_days : :any:`int`, default 365\n        The maximum length of the period. Ignored if `end` is not set.\n        The stricter of this or `start` is used to determine the earliest\n        allowable moment for the baseline period.\n    allow_billing_period_overshoot : :any:`bool`, default False\n        If True, count `max_days` from the end of the last billing data period\n        that ends before the `end` date, rather than from the exact `end` date.\n        Otherwise use the exact `end` date as the cutoff.\n    n_days_billing_period_overshoot: :any:`int`, default None\n        If `allow_billing_period_overshoot` is set to True, this determines\n        the number of days of overshoot that will be tolerated. A value of\n        None implies that any number of days is allowed.\n    ignore_billing_period_gap_for_day_count : :any:`bool`, default False\n        If True, instead of going back `max_days` from either the\n        `end` date or end of the last billing period before that date (depending\n        on the value of the `allow_billing_period_overshoot` setting) and\n        excluding the last period that began before that date, first check to\n        see if excluding or including that period gets closer to a total of\n        `max_days` of data.\n\n        For example, with `max_days=365`, if an exact 365 period would targeted\n        Feb 15, but the billing period went from Jan 20 to Feb 20, exclude that\n        period for a total of ~360 days of data, because that's closer to 365\n        than ~390 days, which would be the total if that period was included.\n        If, on the other hand, if that period started Feb 10 and went to Mar 10,\n        include the period, because ~370 days of data is closer to than ~340.\n\n    Returns\n    -------\n    baseline_data, warnings : :any:`tuple` of (:any:`pandas.DataFrame` or :any:`pandas.Series`, :any:`list` of :any:`eemeter.EEMeterWarning`)\n        Data for only the specified baseline period and any associated warnings.\n    \"\"\"\n    if max_days is not None:\n        if start is not None:\n            raise ValueError(  # pragma: no cover\n                \"If max_days is set, start cannot be set: start={}, max_days={}.\".format(\n                    start, max_days\n                )\n            )\n\n    start_inf = False\n    if start is None:\n        # py datetime min/max are out of range of pd.Timestamp min/max\n        start_target = pytz.UTC.localize(pd.Timestamp.min) + timedelta(days=1)\n        start_inf = True\n    else:\n        start_target = start\n\n    end_inf = False\n    if end is None:\n        end_limit = pytz.UTC.localize(pd.Timestamp.max) - timedelta(days=1)\n        end_inf = True\n    else:\n        end_limit = end\n\n    # copying prevents setting on slice warnings\n    data_before_end_limit = data[:end_limit].copy()\n    data_end = data_before_end_limit.index.max()\n\n    if ignore_billing_period_gap_for_day_count and (\n        n_days_billing_period_overshoot is None\n        or end_limit - timedelta(days=n_days_billing_period_overshoot) < data_end\n    ):\n        end_limit = data_before_end_limit.index.max()\n\n    if not end_inf and max_days is not None:\n        start_target = end_limit - timedelta(days=max_days)\n\n    if allow_billing_period_overshoot:\n        # adjust start limit to get a selection closest to max_days\n        # also consider ffill for get_loc method - always picks previous\n        try:\n            loc = data_before_end_limit.index.get_indexer(\n                [start_target], method=\"nearest\"\n            )[0]\n        except (KeyError, IndexError):  # pragma: no cover\n            baseline_data = data_before_end_limit\n            start_limit = start_target\n        else:\n            start_limit = data_before_end_limit.index[loc]\n            baseline_data = data_before_end_limit[start_limit:].copy()\n\n    else:\n        # use hard limit for baseline start\n        start_limit = start_target\n        baseline_data = data_before_end_limit[start_limit:].copy()\n\n    if baseline_data.dropna().empty:\n        raise NoBaselineDataError()\n\n    baseline_data.iloc[-1] = np.nan\n\n    data_end = data.index.max()\n    data_start = data.index.min()\n    return (\n        baseline_data,\n        _make_baseline_warnings(\n            end_inf, start_inf, data_start, data_end, start_limit, end_limit\n        ),\n    )\n\n\ndef _make_reporting_warnings(\n    end_inf, start_inf, data_start, data_end, start_limit, end_limit\n):\n    warnings = []\n    # warn if there is a gap at end\n    if not end_inf and data_end < end_limit:\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.get_reporting_data.gap_at_reporting_end\",\n                description=(\n                    \"Data does not have coverage at requested reporting end date.\"\n                ),\n                data={\n                    \"requested_end\": end_limit.isoformat(),\n                    \"data_end\": data_end.isoformat(),\n                },\n            )\n        )\n    # warn if there is a gap at start\n    if not start_inf and start_limit < data_start:\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.get_reporting_data.gap_at_reporting_start\",\n                description=(\n                    \"Data does not have coverage at requested reporting start date.\"\n                ),\n                data={\n                    \"requested_start\": start_limit.isoformat(),\n                    \"data_start\": data_start.isoformat(),\n                },\n            )\n        )\n    return warnings\n\n\ndef get_reporting_data(\n    data,\n    start=None,\n    end=None,\n    max_days=365,\n    allow_billing_period_overshoot=False,\n    ignore_billing_period_gap_for_day_count=False,\n):\n    \"\"\"Filter down to reporting period data.\n\n    Parameters\n    ----------\n    data : :any:`pandas.DataFrame` or :any:`pandas.Series`\n        The data to filter to reporting data. This data will be filtered down\n        to an acceptable reporting period according to the dates passed as\n        `start` and `end`, or the maximum period specified with `max_days`.\n    start : :any:`datetime.datetime`\n        A timezone-aware datetime that represents the earliest allowable moment\n        for the reporting data, i.e., the earliest moment for which data is\n        available after the intervention begins.\n    end : :any:`datetime.datetime`\n        A timezone-aware datetime that represents the latest allowable end\n        moment for the reporting data. The stricter of this or `max_days` is used\n        to determine the latest allowable reporting period timestamp.\n    max_days : :any:`int`, default 365\n        The maximum length of the period. Ignored if `start` is not set.\n        The stricter of this or `end` is used to determine the latest\n        allowable reporting period timestamp.\n    allow_billing_period_overshoot : :any:`bool`, default False\n        If True, count `max_days` from the start of the first billing data period\n        that starts after the `start` date, rather than from the exact `start` date.\n        Otherwise use the exact `start` date as the cutoff.\n    ignore_billing_period_gap_for_day_count : :any:`bool`, default False\n        If True, instead of going forward `max_days` from either the\n        `start` date or the `start` of the first billing period after that date\n        (depending on the value of the `allow_billing_period_overshoot` setting)\n        and excluding the first period that ended after that date, first check\n        to see if excluding or including that period gets closer to a total of\n        `max_days` of data.\n\n        For example, with `max_days=365`, if an exact 365 period would targeted\n        Feb 15, but the billing period went from Jan 20 to Feb 20, include that\n        period for a total of ~370 days of data, because that's closer to 365\n        than ~340 days, which would be the total if that period was excluded.\n        If, on the other hand, if that period started Feb 10 and went to Mar 10,\n        exclude the period, because ~360 days of data is closer to than ~390.\n\n    Returns\n    -------\n    reporting_data, warnings : :any:`tuple` of (:any:`pandas.DataFrame` or\n    :any:`pandas.Series`, :any:`list` of :any:`eemeter.EEMeterWarning`)\n        Data for only the specified reporting period and any associated warnings.\n    \"\"\"\n    if max_days is not None:\n        if end is not None:\n            raise ValueError(  # pragma: no cover\n                \"If max_days is set, end cannot be set: end={}, max_days={}.\".format(\n                    end, max_days\n                )\n            )\n\n    start_inf = False\n    if start is None:\n        # py datetime min/max are out of range of pd.Timestamp min/max\n        start_limit = pytz.UTC.localize(pd.Timestamp.min) + timedelta(days=1)\n        start_inf = True\n    else:\n        start_limit = start\n\n    end_inf = False\n    if end is None:\n        end_target = pytz.UTC.localize(pd.Timestamp.max) - timedelta(days=1)\n        end_inf = True\n    else:\n        end_target = end\n\n    # copying prevents setting on slice warnings\n    data_after_start_limit = data[start_limit:].copy()\n\n    if ignore_billing_period_gap_for_day_count:\n        start_limit = data_after_start_limit.index.min()\n\n    if not start_inf and max_days is not None:\n        end_target = start_limit + timedelta(days=max_days)\n\n    if allow_billing_period_overshoot:\n        # adjust start limit to get a selection closest to max_days\n        # also consider bfill for get_loc method - always picks next\n        try:\n            loc = data_after_start_limit.index.get_indexer(\n                [end_target], method=\"nearest\"\n            )[0]\n        except (KeyError, IndexError):  # pragma: no cover\n            reporting_data = data_after_start_limit\n            end_limit = end_target\n        else:\n            end_limit = data_after_start_limit.index[loc]\n            reporting_data = data_after_start_limit[:end_limit].copy()\n\n    else:\n        # use hard limit for baseline start\n        end_limit = end_target\n        reporting_data = data_after_start_limit[:end_limit].copy()\n\n    if reporting_data.dropna().empty:\n        raise NoReportingDataError()\n\n    reporting_data.iloc[-1] = np.nan\n\n    data_end = data.index.max()\n    data_start = data.index.min()\n    return (\n        reporting_data,\n        _make_reporting_warnings(\n            end_inf, start_inf, data_start, data_end, start_limit, end_limit\n        ),\n    )\n\n\nclass Term(object):\n    \"\"\"\n    The term object represents a subset of an index.\n\n    Attributes\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        The index of the term. Includes a period at the end meant to be NaN-value.\n    label : :any:`str`\n        The label for the term.\n    target_start_date : :any:`pandas.Timestamp` or :any:`datetime.datetime`\n        The start date inferred for this term from the start date and target term\n        lenths.\n    target_end_date : :any:`pandas.Timestamp` or :any:`datetime.datetime`\n        The end date inferred for this term from the start date and target term\n        lenths.\n    target_term_length_days : :any:`int`\n        The number of days targeted for this term.\n    actual_start_date : :any:`pandas.Timestamp`\n        The first date in the index.\n    actual_end_date : :any:`pandas.Timestamp`\n        The last date in the index.\n    actual_term_length_days : :any:`int`\n        The number of days between the actual start date and actual end date.\n    complete : :any:`bool`\n        True if this term is conclusively complete, such that additional data added\n        to the series would not add more data to this term.\n\n    \"\"\"\n\n    def __init__(\n        self,\n        index,\n        label,\n        target_start_date,\n        target_end_date,\n        target_term_length_days,\n        actual_start_date,\n        actual_end_date,\n        actual_term_length_days,\n        complete,\n    ):\n        self.index = index\n        self.label = label\n        self.target_start_date = target_start_date\n        self.target_end_date = target_end_date\n        self.target_term_length_days = target_term_length_days\n        self.actual_start_date = actual_start_date\n        self.actual_end_date = actual_end_date\n        self.actual_term_length_days = actual_term_length_days\n        self.complete = complete\n\n    def __repr__(self):\n        return (\n            \"Term(label={}, target_term_length_days={}, actual_term_length_days={},\"\n            \" complete={})\"\n        ).format(\n            self.label,\n            self.target_term_length_days,\n            self.actual_term_length_days,\n            self.complete,\n        )\n\n\ndef get_terms(index, term_lengths, term_labels=None, start=None, method=\"strict\"):\n    \"\"\"Breaks a :any:`pandas.DatetimeIndex` into consecutive terms of specified\n    lengths.\n\n    Parameters\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        The index to split into terms, generally `meter_data.index`\n        or `temperature_data.index`.\n    term_lengths : :any:`list` of :any:`int`\n        The lengths (in days) of the terms into which to split the data.\n    term_labels : :any:`list` of :any:`str`, default None\n        Labels to use for each term. List must be the same length as the\n        `term_lengths` list.\n    start : :any:`datetime.datetime`, default None\n        A timezone-aware datetime that represents the earliest allowable start\n        date for the terms. If None, use the first element of the index.\n    method: one of ['strict', 'nearest'], default 'strict'\n        The method to use to get terms.\n\n        - \"strict\": Ensures that the term end will come on or before the length of\n\n    Returns\n    -------\n    terms : :any:`list` of :any:`eemeter.Term`\n        A dataframe of term labels with the same :any:`pandas.DatetimeIndex`\n        given as `index`. This can be used to filter the original data into\n        terms of approximately the desired length.\n\n\n    \"\"\"\n    if method == \"strict\":\n        get_loc_method = \"pad\"\n    elif method == \"nearest\":\n        get_loc_method = \"nearest\"\n    else:\n        raise ValueError(\n            \"method {} not supported - use either 'strict' or 'closest'\".format(method)\n        )\n\n    if not index.is_monotonic_increasing:\n        raise ValueError(\"get_terms requires a sorted index\")\n\n    if term_labels is None:\n        term_labels = [\n            \"term_{:03d}\".format(i + 1) for i, term_length in enumerate(term_lengths)\n        ]\n\n    elif len(term_labels) != len(term_lengths):\n        raise ValueError(\n            \"term_labels (len {}) must be the same length as term_length (len {})\".format(\n                len(term_labels), len(term_lengths)\n            )\n        )\n\n    if start is None:\n        prev_start = index.min()\n    else:\n        prev_start = start\n\n    term_end_targets = [\n        prev_start + timedelta(days=sum(term_lengths[: i + 1]))\n        for i in range(len(term_lengths))\n    ]\n\n    terms = []\n    remaining_index = index[index >= prev_start]\n\n    for label, target_term_length, end_target in zip(\n        term_labels, term_lengths, term_end_targets\n    ):\n        if len(remaining_index) <= 1:\n            break\n\n        next_index = remaining_index.get_indexer([end_target], method=get_loc_method)[0]\n\n        # keep one extra index point for the end NaN - this could be confusing, but\n        # helps identify the full range of the last data point\n        term_index = remaining_index[: next_index + 1]\n\n        # find the next start\n        next_start = remaining_index[next_index]\n\n        # reset the remaining index\n        remaining_index = remaining_index[next_index:]\n\n        # There may be a better way to tell if the term is conclusively complete,\n        # but the logic here is that if there's more than one remaining point then\n        # the term must be complete - since that final point was a worse candidate\n        # than the one before it which was chosen.\n        complete = len(remaining_index) > 1\n\n        terms.append(\n            Term(\n                index=term_index,\n                label=label,\n                target_start_date=prev_start,\n                target_end_date=end_target,\n                target_term_length_days=target_term_length,\n                actual_start_date=term_index[0],\n                actual_end_date=term_index[-1],\n                actual_term_length_days=(term_index[-1] - term_index[0]).days,\n                complete=complete,\n            )\n        )\n\n        # reset the previous start\n        prev_start = next_start\n\n    return terms\n\n\ndef clean_caltrack_billing_data(data, source_interval):\n    # check for empty data\n    if data[\"value\"].dropna().empty:\n        return data[:0]\n\n    if source_interval.startswith(\"billing\"):\n        diff = list((data.index[1:] - data.index[:-1]).days)\n        filter_ = pd.Series(diff + [np.nan], index=data.index)\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        if source_interval == \"billing_monthly\":\n            data = data[\n                (filter_ <= 35) & (filter_ >= 25)  # keep these, inclusive\n            ].reindex(data.index)\n\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        if source_interval == \"billing_bimonthly\":\n            data = data[\n                (filter_ <= 70) & (filter_ >= 25)  # keep these, inclusive\n            ].reindex(data.index)\n\n        # CalTRACK 2.2.3.1\n        \"\"\"\n        Adds estimate to subsequent read if there aren't more than one estimate in a row\n        and then removes the estimated row.\n\n        Input:\n        index   value   estimated\n        1       2       False\n        2       3       False\n        3       5       True\n        4       4       False\n        5       6       True\n        6       3       True\n        7       4       False\n        8       NaN     NaN\n\n        Output:\n        index   value\n        1       2\n        2       3\n        4       9\n        5       NaN\n        7       7\n        8       NaN\n        \"\"\"\n        add_estimated = []\n        remove_estimated_fixed_rows = []\n        orig_data = data.copy()\n        if \"estimated\" in data.columns:\n            data[\"unestimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated == False)].reindex(data.index)\n            )\n            data[\"estimated_value\"] = (\n                data[:-1].value[(data[:-1].estimated)].reindex(data.index)\n            )\n            for i, (index, row) in enumerate(data[:-1].iterrows()):\n                # ensures there is a prev_row and previous row value is null\n                if i > 0 and pd.isnull(prev_row[\"unestimated_value\"]):\n                    # current row value is not null\n                    add_estimated.append(prev_row[\"estimated_value\"])\n                    if not pd.isnull(row[\"unestimated_value\"]):\n                        # get all rows that had only estimated reads that will be\n                        # added to the subsequent row meaning this row\n                        # needs to be removed\n                        remove_estimated_fixed_rows.append(prev_index)\n                else:\n                    add_estimated.append(0)\n                prev_row = row\n                prev_index = index\n            add_estimated.append(np.nan)\n            data[\"value\"] = data[\"unestimated_value\"] + add_estimated\n            data = data[~data.index.isin(remove_estimated_fixed_rows)]\n            data = data[[\"value\"]]  # remove the estimated column\n\n    # check again for empty data\n    if data.dropna().empty:\n        return data[:0]\n\n    return data\n\n\ndef downsample_and_clean_caltrack_daily_data(data):\n    data = as_freq(data.value, \"D\", include_coverage=True)\n\n    # CalTRACK 2.2.2.1 - interpolate with average of non-null values\n    data.loc[data.coverage > 0.5, \"value\"] = (\n        data[data.coverage > 0.5].value / data[data.coverage > 0.5].coverage\n    )\n\n    # CalTRACK 2.2.2.1 - discard days with less than 50% coverage\n    return data[data.coverage > 0.5].reindex(data.index)[[\"value\"]]\n\n\ndef clean_caltrack_billing_daily_data(data, source_interval):\n    # billing data is cleaned but not resampled\n    if source_interval.startswith(\"billing\"):\n        # CalTRACK 2.2.3.4, 2.2.3.5\n        return clean_caltrack_billing_data(data, source_interval)\n\n    # higher intervals like daily, hourly, 30min, 15min are\n    # resampled (daily) or downsampled (hourly, 30min, 15min)\n    elif source_interval == \"daily\":\n        return data\n    else:\n        return downsample_and_clean_caltrack_daily_data(data)\n\n\ndef add_freq(idx, freq=None):\n    \"\"\"Add a frequency attribute to idx, through inference or directly.\n\n     Returns a copy.  If `freq` is None, it is inferred.\n\n     Note: this function is taken from\n     https://stackoverflow.com/questions/46217529/pandas-datetimeindex-frequency-is-none-and-cant-be-set;\n     credit Brad Solomon.\n\n\n    Parameters\n    ----------\n    idx : :any:`pandas.DateTimeIndex`\n        Any DateTimeIndex.\n    freq : :any valid DateTimeIndex Freq in 'str' format\n        The frequency of the datetime index. Defaults to 'None' if frequency is to be inferred.\n\n    Returns\n    -------\n    idx : :any:`pandas.DateTimeIndex`\n        A copy of idx with frequency added.\n    \"\"\"\n\n    idx = idx.copy()\n    if freq is None:\n        if idx.freq is None:\n            freq = pd.infer_freq(idx)\n        else:\n            return idx\n    idx.freq = pd.tseries.frequencies.to_offset(freq)\n    if idx.freq is None:\n        raise AttributeError(\n            \"no discernible frequency found to `idx`.  Specify\"\n            \" a frequency string with `freq`.\"\n        )\n    return idx\n\n\ndef trim(*args, freq=\"h\", tz=\"UTC\"):\n    \"\"\"A helper function which trims a given number of time series dataframes so that they all correspond to the same\n    time periods. Typically used to ensure that both gas, electricity, and temperature datasets cover the same time\n    period. Trim undertakes the following steps:\n\n       - copies dataframes\n       - sets indexes to datetimes if not already\n       - localises index to UTC (default - if other timezone applies this should be specified)\n       - sorts in ascending order against DateTimeIndex\n       - drops nulls at both start and end of df\n       - trims the dataframes by equalising all min(df.index) and max(df.index)\n\n       Trim requires both input dataframes to have some degree of overlap beforehand.\n\n     Parameters\n    ----------\n    *args : : one or more 'pandas.DataFrame's\n        A set of regular time series datasets. If index is not DateTimeIndex, function will convert accordingly. There\n        must be overlap between all datasets otherwise trim will return IndexError. Can function with one dataframe,\n        though not much point given functionality.\n    freq : : any valid DateTimeIndex frequency.\n        This is used to identify any duplicates and missing values in a dataframe and ensure that each dataframe in the\n        returned tuple is of the same length. Freq defaults to '1H' (one hour) but can be, for example '0.5H' (1/2 hour)\n    tz : : any valid timezone 'str'\n        The timezone associated with the given dataframes. If timezone-naive, function will localise to 'UTC' as\n        default.\n\n     Returns\n    -------\n     out_dfs : :any:`tuple` of 'pandas.DataFrame's.\n         A list of dataframes trimmed to equal total intervals, arranged in eemeter format (i.e. with ascending\n         indices).\n    \"\"\"\n    new_tuple = ()\n    if len(list(args)) == 1:\n        args = args[0]\n    for i in args:\n        df = i\n        if not isinstance(df.index, pd.DatetimeIndex):\n            df.index = pd.to_datetime(df.index, infer_datetime_format=True)\n        if df.index.tz is None:\n            df.index = df.index.tz_localize(tz=tz)  # defaults to UTC\n        df = df.sort_index()\n        df = df[~df.index.duplicated(keep=\"first\")]\n        df = df.resample(freq).asfreq()\n        new_tuple = new_tuple + (df,)\n    max_start = max([min(df.index) for df in new_tuple])\n    min_end = min([max(df.index) for df in new_tuple])\n    out_dfs = ()\n    if max_start < min_end:\n        for df in new_tuple:\n            out_dfs = out_dfs + (df.loc[max_start:min_end],)\n    else:\n        raise IndexError(\"Trim requires for all dfs to have some overlap.\")\n\n    return out_dfs\n\n\ndef _check_input_formatting(input, tz=\"UTC\"):\n    if not isinstance(input.index, pd.DatetimeIndex):\n        if isinstance(input.index, pd.RangeIndex):\n            for i in [\n                \"start\",\n                \"Start\",\n                \"Datetime\",\n                \"timestamp\",\n                \"Timestamp\",\n                \"datetime\",\n            ]:  # this is a non-exhaustive list (welcome additions) of possible timestamp headers when not in index.\n                if i in input.columns.values:\n                    input = input.set_index(i)\n        if not isinstance(input.index, pd.DatetimeIndex):\n            input.index = pd.to_datetime(input.index)\n            if input.index[0].tzinfo is None:\n                input.index = input.index.tz_localize(tz=tz)\n        else:\n            raise ValueError(\n                \"Data is not in correct format - index should be of class 'pd.core.indexes.datetimes.DatetimeIndex',\"\n                + \" or datetime column should be labelled one of: 'Start', 'start', 'Datetime', 'timestamp', \"\n                \"'Timestamp', or 'Datetime'.\"\n            )\n    if input.index[0].tzinfo is None:\n        input.index = input.index.tz_localize(tz=tz)\n    return input\n\n\ndef _format_data_for_caltrack_hourly(df, tz=\"UTC\"):\n    if df is not None:\n        df = df.copy()\n        df = _check_input_formatting(df, tz)\n        df = df.sort_index()\n        return df\n    else:\n        return None\n\n\ndef format_energy_data_for_caltrack(*args, method=\"hourly\", tz=\"UTC\"):\n    \"\"\"A helper function which ensures energy consumption data is formatted for eemeter processing.\n\n    Parameters\n    ----------\n    *args : :one or more `pandas.DataFrame`s\n        Energy consumption time series data. Consumption must be measured in the same units.\n    method : : any valid 'str'\n        The relevant eemeter model requiring formatting. Must be either 'hourly', 'daily', or 'billing'. Defaults to\n        'hourly'.\n    tz : : any valid timezone 'str'\n        The timezone associated with the given dataframes. If timezone-naive, function will localise to 'UTC' as\n        default.\n\n    Returns\n    -------\n    args_tuple : any 'list' containing one or more 'pandas.DataFrame's\n        A list of dataframes comprising energy consumption data in eemeter format.\n    \"\"\"\n\n    if method == \"hourly\":\n        freq = \"h\"\n    elif method == \"daily\":\n        freq = \"D\"\n    elif method == \"billing\":\n        freq = MonthEnd()  # \"M\"/\"ME\" depending on pandas version\n    else:\n        raise ValueError(\"'method' must be either 'hourly', 'daily' or 'billing'.\")\n\n    args_tuple = ()\n    for df in args:\n        df = _format_data_for_caltrack_hourly(df, tz)\n        if not isinstance(df, pd.DataFrame):\n            df = pd.DataFrame(df)\n        df = df.resample(freq).sum()\n        df.index = df.index.rename(\"start\")\n        args_tuple = args_tuple + (df,)\n        if df.columns[0] != \"value\":\n            current_col_name = df.columns[0]\n            df.rename(columns={current_col_name: \"value\"}, inplace=True)\n\n    if len(args_tuple) == 1:\n        return args_tuple[0]\n    else:\n        args_list = list(args_tuple)\n        args_list[-1], args_list[-2] = trim(args_list[-1], args_list[-2], freq=freq)\n        args_tuple = tuple(args_list)\n        return args_tuple\n\n\ndef format_temperature_data_for_caltrack(temperature_data, tz=\"UTC\"):\n    \"\"\"A helper function which ensures external temperature data is formatted for eemeter processing.\n\n    Parameters\n    ----------\n    temperature_data : :any:``\n        Hourly external temperature data. If DataFrame, not pd.Series (as required by CalTRACK) function will convert.\n    tz : : any valid timezone 'str'\n        The timezone associated with the given dataframes. If timezone-naive, function will localise to 'UTC' as\n        default.\n    Returns\n    -------\n    temperature_data : :any:``\n        Hourly external temperature data in eemeter format.\n    \"\"\"\n\n    temperature_data = _format_data_for_caltrack_hourly(temperature_data, tz)\n    mask = temperature_data.index.minute == 00\n    temperature_data = temperature_data[mask]\n    if temperature_data.index.freq == None:\n        temperature_data.index = add_freq(temperature_data.index)\n    if isinstance(temperature_data, pd.DataFrame):\n        temperature_data = temperature_data.squeeze()\n    return temperature_data\n"
  },
  {
    "path": "opendsm/eemeter/common/warnings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport logging\nfrom typing import Union\nimport pydantic\n\n__all__ = (\"EEMeterWarning\",)\n\n\nclass EEMeterWarning(pydantic.BaseModel):\n    \"\"\"An object representing a warning and data associated with it.\n\n    Attributes\n    ----------\n    qualified_name : :any:`str`\n        Qualified name, e.g., `'eemeter.method_abc.missing_data'`.\n    description : :any:`str`\n        Prose describing the nature of the warning.\n    data : :any:`dict`\n        Data that reproducibly shows why the warning was issued. Data should\n        be JSON serializable.\n    \"\"\"\n\n    qualified_name: str\n    description: str\n    data: Union[dict, list]\n\n    def __repr__(self):\n        return \"EEMeterWarning(qualified_name={})\".format(self.qualified_name)\n\n    def __str__(self):\n        return repr(self)\n\n    def json(self) -> dict:\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n        return {\n            \"qualified_name\": self.qualified_name,\n            \"description\": self.description,\n            \"data\": self.data,\n        }\n\n    def warn(self):\n        data = \"\"\n        if self.data:\n            data = f\"\\n{self.data}\"\n        logging.getLogger(\"eemeter\").warning(f\"{self.description}{data}\")\n"
  },
  {
    "path": "opendsm/eemeter/models/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .hourly_caltrack import (\n    HourlyModel as HourlyCaltrackModel,\n    HourlyBaselineData as HourlyCaltrackBaselineData,\n    HourlyReportingData as HourlyCaltrackReportingData,\n)\nfrom .hourly import *\nfrom .daily import *\nfrom .billing import *\n"
  },
  {
    "path": "opendsm/eemeter/models/billing/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .data import BillingBaselineData, BillingReportingData\nfrom .model import BillingModel\nfrom .weighted_model import BillingWeightedModel\n\n__all__ = (\n    \"BillingBaselineData\",\n    \"BillingReportingData\",\n    \"BillingModel\",\n    \"BillingWeightedModel\",\n)\n"
  },
  {
    "path": "opendsm/eemeter/models/billing/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport datetime\n\nimport numpy as np\nimport pandas as pd\nfrom pandas.tseries.offsets import MonthBegin, MonthEnd\n\nfrom opendsm.eemeter.common.data_processor_utilities import (\n    as_freq,\n    clean_billing_daily_data,\n    compute_minimum_granularity,\n)\nfrom opendsm.eemeter.common.features import compute_temperature_features\nfrom opendsm.eemeter.common.data_settings import BillingDataSettings\nfrom opendsm.eemeter.common.sufficiency_criteria import BillingSufficiencyCriteria\nfrom opendsm.eemeter.models.daily.data import _DailyData\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\n\n\"\"\"TODO there is still a ton of unecessarily duplicated code between billing+daily.\n    we should be able to perform a few transforms within the billing baseclass, and then call super() for the rest\n\n    unsure whether we should inherit from the public classes because we'll have to take care to use type(data)\n    instead of isinstance(data,  _) when doing the checks in the model/wrapper to avoid unintentionally allowing a mix of data/model type\n\"\"\"\n\n\nclass _BillingData(_DailyData):\n    \"\"\"Baseline data processor for billing data.\n\n    2.2.3.4. Off-cycle reads (spanning less than 25 days) should be dropped from analysis.\n    These readings typically occur due to meter reading problems or changes in occupancy.\n\n    2.2.3.5. For pseudo-monthly billing cycles, periods spanning more than 35 days should be dropped from analysis.\n    For bi-monthly billing cycles, periods spanning more than 70 days should be dropped from the analysis.\n    \"\"\"\n\n    _settings_class = BillingDataSettings\n\n    def _compute_meter_value_df(self, df: pd.DataFrame):\n        \"\"\"\n        Computes the meter value DataFrame by cleaning and processing the observed meter data.\n        1. The minimum granularity is computed from the non null rows. If the billing cycle is mixed between monthly and bimonthly, then the minimum granularity is bimonthly\n        2. The meter data is cleaned and downsampled/upsampled into the correct frequency using clean_billing_daily_data()\n        3. Add missing days as NaN by merging with a full year daily index.\n\n        Parameters\n        ----------\n\n            df (pd.DataFrame): The DataFrame containing the observed meter data.\n\n        Returns\n        -------\n            pd.DataFrame: The cleaned and processed meter value DataFrame.\n        \"\"\"\n        meter_series_full = df[\"observed\"]\n        meter_series = meter_series_full.dropna()\n        if meter_series.empty:\n            return meter_series_full.resample(\"D\").first().to_frame()\n\n        start_date = meter_series_full.index.min()\n        end_date = meter_series_full.index.max().replace(\n            hour=meter_series.index[-1].hour\n        )  # assume final period ends on same hour\n\n        # ensure we adjust backwards to normalize hour, never adding time\n        if end_date > meter_series_full.index.max():\n            end_date = end_date - pd.Timedelta(days=1)\n\n        min_granularity = compute_minimum_granularity(\n            meter_series.index, default_granularity=\"billing_bimonthly\"\n        )\n\n        # Ensure higher frequency data is aggregated to the monthly model\n        if not min_granularity.startswith(\"billing\"):\n            # MS is so that the date for Month Start\n            meter_series = meter_series.resample(\"MS\").sum(min_count=1)\n            # normalize to midnight since we're picking an arbitrary day to represent period start anyway\n            end_date = end_date.normalize()\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.sufficiency_criteria.inferior_model_usage\",\n                    description=(\n                        \"Daily data is provided but the model used is monthly. Are you sure this is the intended model?\"\n                    ),\n                    data={},\n                )\n            )\n            min_granularity = \"billing_monthly\"\n\n        # Adjust index to follow final nan convention--without this, final period will be short one day\n        meter_series[end_date + pd.Timedelta(days=1)] = np.nan\n\n        # This checks for offcycle reads. That is a disqualification if the billing cycle is less than 25 days\n        meter_value_df = clean_billing_daily_data(\n            meter_series.to_frame(\"value\"), min_granularity, self.disqualification\n        )\n\n        # Spread billing data to daily\n        meter_value_df = as_freq(meter_value_df[\"value\"], \"D\").to_frame(\"value\")\n        meter_value_df = meter_value_df[:-1]\n        meter_value_df = meter_value_df.rename(columns={\"value\": \"observed\"})\n\n        # This will ensure that the missing days are kept in the dataframe\n        # Create an index with all the days from the start and end date of 'meter_value_df'\n        if len(meter_value_df) > 0:\n            all_days_index = pd.date_range(\n                start=start_date,\n                end=end_date,\n                freq=\"D\",\n                tz=df.index.tz,\n                ambiguous=True,\n                nonexistent=\"shift_forward\",\n            )\n            all_days_df = pd.DataFrame(index=all_days_index)\n            meter_value_df = meter_value_df.merge(\n                all_days_df, left_index=True, right_index=True, how=\"outer\"\n            )\n\n        return meter_value_df\n\n    def _compute_temperature_features(\n        self, df: pd.DataFrame, meter_index: pd.DatetimeIndex\n    ):\n        \"\"\"\n        Compute temperature features for the given DataFrame and meter index.\n        1. The frequency of the temperature data is inferred and set to hourly if not already. If frequency is not inferred or its lower than hourly, a warning is added.\n        2. The temperature data is downsampled/upsampled into the daily frequency using as_freq()\n        3. High frequency temperature data is checked for missing values and a warning is added if more than 50% of the data is missing, and those rows are set to NaN.\n        4. If frequency was already hourly, compute_temperature_features() is used to recompute the temperature to match with the meter index.\n\n        Parameters\n        ----------\n\n            df (pd.DataFrame): The DataFrame containing temperature data.\n            meter_index (pd.DatetimeIndex): The meter index.\n\n        Returns\n        -------\n\n            pd.Series: The computed temperature values.\n            pd.DataFrame: The computed temperature features.\n        \"\"\"\n        temp_series = df[\"temperature\"]\n        temp_series.index.freq = temp_series.index.inferred_freq\n        if temp_series.index.freq != \"h\":\n            if (\n                temp_series.index.freq is None\n                or isinstance(temp_series.index.freq, MonthEnd)\n                or isinstance(temp_series.index.freq, MonthBegin)\n                or temp_series.index.freq > pd.Timedelta(hours=1)\n            ):\n                # Add warning for frequencies longer than 1 hour\n                self.warnings.append(\n                    EEMeterWarning(\n                        qualified_name=\"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\",\n                        description=(\n                            \"Cannot confirm that pre-aggregated temperature data had sufficient hours kept\"\n                        ),\n                        data={},\n                    )\n                )\n            # TODO consider disallowing this until a later patch\n            if temp_series.index.freq != \"D\":\n                # Downsample / Upsample the temperature data to daily\n                temperature_features = as_freq(\n                    temp_series, \"D\", series_type=\"instantaneous\", include_coverage=True\n                )\n                # If high frequency data check for 50% data coverage in rollup\n                if len(temperature_features[temperature_features.coverage <= 0.5]) > 0:\n                    self.warnings.append(\n                        EEMeterWarning(\n                            qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\",\n                            description=(\n                                \"More than 50% of the high frequency Temperature data is missing.\"\n                            ),\n                            data={\n                                \"high_frequency_data_missing_count\": len(\n                                    temperature_features[\n                                        temperature_features.coverage <= 0.5\n                                    ].index.to_list()\n                                )\n                            },\n                        )\n                    )\n\n                # Set missing high frequency data to NaN\n                temperature_features.loc[\n                    temperature_features.coverage > 0.5, \"value\"\n                ] = (\n                    temperature_features[temperature_features.coverage > 0.5].value\n                    / temperature_features[temperature_features.coverage > 0.5].coverage\n                )\n\n                temperature_features = (\n                    temperature_features[temperature_features.coverage > 0.5]\n                    .reindex(temperature_features.index)[[\"value\"]]\n                    .rename(columns={\"value\": \"temperature_mean\"})\n                )\n\n                if \"coverage\" in temperature_features.columns:\n                    temperature_features = temperature_features.drop(\n                        columns=[\"coverage\"]\n                    )\n            else:\n                temperature_features = temp_series.to_frame(name=\"temperature_mean\")\n\n            temperature_features[\"temperature_null\"] = temp_series.isnull().astype(int)\n            temperature_features[\"temperature_not_null\"] = temp_series.notnull().astype(\n                int\n            )\n            temperature_features[\"n_days_kept\"] = 0  # unused\n            temperature_features[\"n_days_dropped\"] = 0  # unused\n        else:\n            if not meter_index.empty:\n                buffer_idx = meter_index.max() + pd.Timedelta(days=1)\n                meter_index = meter_index.union([buffer_idx])\n\n            temperature_features = compute_temperature_features(\n                meter_index,\n                temp_series,\n                data_quality=True,\n            )\n            temperature_features = temperature_features[:-1]\n            # Only check for high frequency temperature data if it exists\n            # TODO this check causes weird behavior with very sparse temp data.\n            # will still get DQ'd, but final df receives non-nan temperatures\n            median_samples = (\n                temperature_features.temperature_not_null\n                + temperature_features.temperature_null\n            ).median()\n            if median_samples > 1:\n                invalid_temperature_rows = (\n                    temperature_features.temperature_not_null\n                    / (\n                        temperature_features.temperature_not_null\n                        + temperature_features.temperature_null\n                    )\n                ) <= 0.5\n                # check against median in case start/end of data does not cover a full period\n                invalid_temperature_rows |= (\n                    temperature_features.temperature_not_null <= median_samples * 0.5\n                )\n\n                if invalid_temperature_rows.any():\n                    self.warnings.append(\n                        EEMeterWarning(\n                            qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\",\n                            description=(\n                                \"More than 50% of the high frequency temperature data is missing.\"\n                            ),\n                            data=[\n                                timestamp.isoformat()\n                                for timestamp in invalid_temperature_rows.index\n                            ],\n                        )\n                    )\n                    temperature_features.loc[\n                        invalid_temperature_rows, \"temperature_mean\"\n                    ] = np.nan\n\n        temp = temperature_features[\"temperature_mean\"].rename(\"temperature\")\n        features = temperature_features.drop(columns=[\"temperature_mean\"])\n        return temp, features\n\n    # TODO: DELETE THIS after making real billing data class\n    @property\n    def billing_df(self) -> pd.DataFrame | None:\n        \"\"\"Get the corrected input data stored in the class. The actual dataframe is immutable, this returns a copy.\"\"\"\n\n        df = self._df.copy()\n\n        # find indices where observed changes from prior\n        observed_change = df[\"observed\"].diff()\n        observed_change = observed_change[observed_change != 0].index\n        obs_change_idx = df.index.get_indexer(observed_change)\n        obs_change_idx = np.append(obs_change_idx, len(df))\n        obs_change_idx = np.delete(obs_change_idx, np.where(np.diff(obs_change_idx) < 15)[0])\n\n        if obs_change_idx[0] != 0:\n            obs_change_idx = np.insert(obs_change_idx, 0, 0)\n\n        # create vector where value increases at each observed change\n        group = []\n        for i in range(1, len(obs_change_idx)):\n            idx_range = obs_change_idx[i] - obs_change_idx[i-1]\n\n            group.extend([i] * idx_range)\n\n        df[\"group\"] = group\n\n        # get median delta\n\n        # get first datetime, average temperature, sum of observed for each group and make new df\n        df_temp = df.reset_index()\n        df_temp = df_temp.rename(columns={\"index\": \"datetime\"})\n\n        df_grouped = df_temp.groupby(\"group\").agg({\n            \"datetime\": \"first\",\n            \"season\": \"first\",\n            \"weekday_weekend\": \"first\",\n            \"temperature\": \"mean\",\n            \"observed\": \"mean\",\n        }).set_index(\"datetime\")\n\n        # create days column for number of days between current and previous index\n        df_grouped[\"days\"] = df_grouped.index.to_series().diff().dt.days\n\n        df_grouped = df_grouped.dropna()\n\n        # create weights from days column\n        df_grouped[\"weights\"] = df_grouped[\"days\"] / df_grouped[\"days\"].sum()\n\n        df_grouped = df_grouped.drop(columns=[\"days\"])\n\n        if self._df is None:\n            return None\n        else:\n            return df_grouped.copy()\n\n\nclass BillingBaselineData(_BillingData):\n    \"\"\"\n    Data class to represent Billing Baseline Data.\n\n    Only baseline data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Billing data should have an extra month's data appended at the to denote end of period. (Do not append NaN, any other value would work.)\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of ssues with the data, but none that will severely reduce the quality of the model built.\n    \"\"\"\n\n    def _check_data_sufficiency(self, sufficiency_df):\n        \"\"\"\n        Private method which checks the sufficiency of the data for billing baseline calculations using the predefined OpenEEMeter sufficiency criteria.\n\n        Args:\n            sufficiency_df (pandas.DataFrame): DataFrame containing the data for sufficiency check. Should have features such as -\n            temperature_null: number of temperature null periods in each aggregation step\n            temperature_not_null: number of temperature non null periods in each aggregation step\n\n        Returns:\n            disqualification (List): List of disqualifications\n            warnings (list): List of warnings\n\n        \"\"\"\n        bsc = BillingSufficiencyCriteria(\n            data=sufficiency_df, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=False,\n            settings=self.settings.sufficiency,\n        )\n        bsc.check_sufficiency_baseline()\n        disqualification = bsc.disqualification\n        warnings = bsc.warnings\n\n        # _, disqualification, warnings = sufficiency_criteria_baseline(\n        #     sufficiency_df,\n        #     is_reporting_data=False,\n        #     is_electricity_data=self.is_electricity_data,\n        # )\n        return disqualification, warnings\n\n\nclass BillingReportingData(_BillingData):\n    \"\"\"Data class to represent Billing Reporting Data.\n\n    Only reporting data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Meter data input is optional for the reporting class.\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of ssues with the data, but none that will severely reduce the quality of the model built.\n    \"\"\"\n\n    def __init__(\n        self,\n        df: pd.DataFrame, \n        is_electricity_data: bool, \n        settings: dict | None = None\n    ):\n        df = df.copy()\n        if \"observed\" not in df.columns:\n            df[\"observed\"] = np.nan\n\n        super().__init__(df, is_electricity_data, settings=settings)\n\n    @classmethod\n    def from_series(\n        cls,\n        meter_data: pd.Series | pd.DataFrame | None,\n        temperature_data: pd.Series | pd.DataFrame,\n        is_electricity_data: bool,\n        tzinfo: datetime.tzinfo | None = None,\n        settings: dict | None = None,\n    ):\n        \"\"\"Create a BillingReportingData instance from meter data and temperature data.\n\n        Args:\n            meter_data: The meter data to be used for the BillingReportingData instance.\n            temperature_data: The temperature data to be used for the BillingReportingData instance.\n            is_electricity_data: Flag indicating whether the meter data represents electricity data.\n            tzinfo: Timezone information to be used for the meter data.\n\n        Returns:\n            An instance of the Data class.\n        \"\"\"\n        if tzinfo and meter_data is not None:\n            raise ValueError(\n                \"When passing meter data to BillingReportingData, convert its DatetimeIndex to local timezone first; `tzinfo` param should only be used in the absence of reporting meter data.\"\n            )\n        if is_electricity_data is None and meter_data is not None:\n            raise ValueError(\n                \"Must specify is_electricity_data when passing meter data.\"\n            )\n        if meter_data is None:\n            meter_data = pd.DataFrame(\n                {\"observed\": np.nan}, index=temperature_data.index\n            )\n            if tzinfo:\n                meter_data = meter_data.tz_convert(tzinfo)\n        if meter_data.empty:\n            raise ValueError(\n                \"Pass meter_data=None to explicitly create a temperature-only reporting data instance.\"\n            )\n        return super().from_series(meter_data, temperature_data, is_electricity_data, settings=settings)\n\n    def _check_data_sufficiency(self, sufficiency_df):\n        \"\"\"\n        Private method which checks the sufficiency of the data for billing reporting calculations using the predefined OpenEEMeter sufficiency criteria.\n\n        Parameters\n        ----------\n        1. sufficiency_df (pandas.DataFrame): DataFrame containing the data for sufficiency check. Should have features such as -\n            - temperature_null: number of temperature null periods in each aggregation step\n            - temperature_not_null: number of temperature non null periods in each aggregation step\n\n        Returns\n        -------\n            disqualification (List): List of disqualifications\n            warnings (list): List of warnings\n\n        \"\"\"\n        bsc = BillingSufficiencyCriteria(\n            data=sufficiency_df, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=True,\n            settings=self.settings.sufficiency,\n        )\n        bsc.check_sufficiency_reporting()\n        disqualification = bsc.disqualification\n        warnings = bsc.warnings\n\n        # _, disqualification, warnings = sufficiency_criteria_baseline(\n        #     sufficiency_df,\n        #     is_reporting_data=True,\n        #     is_electricity_data=self.is_electricity_data,\n        # )\n        return disqualification, warnings\n"
  },
  {
    "path": "opendsm/eemeter/models/billing/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\nfrom opendsm.eemeter.models.billing.data import (\n    BillingBaselineData,\n    BillingReportingData,\n)\nfrom opendsm.eemeter.models.daily.model import DailyModel\n\n\nclass BillingModel(DailyModel):\n    \"\"\"A class to fit a model to the input meter data.\n\n    BillingModel is a wrapper for the DailyModel class using billing presets.\n\n    Attributes:\n        settings (dict): A dictionary of settings.\n        seasonal_options (list): A list of seasonal options (su: Summer, sh: Shoulder, wi: Winter).\n            Elements in the list are seasons separated by '_' that represent a model split.\n            For example, a list of ['su_sh', 'wi'] represents two splits: summer/shoulder and winter.\n        day_options (list): A list of day options.\n        combo_dictionary (dict): A dictionary of combinations.\n        df_meter (pandas.DataFrame): A dataframe of meter data.\n        error (dict): A dictionary of error metrics.\n        combinations (list): A list of combinations.\n        components (list): A list of components.\n        fit_components (list): A list of fit components.\n        wRMSE_base (float): The mean bias error for no splits.\n        best_combination (list): The best combination of splits.\n        model (sklearn.pipeline.Pipeline): The final fitted model.\n        id (str): The index of the meter data.\n    \"\"\"\n    _baseline_data_type = BillingBaselineData\n    _reporting_data_type = BillingReportingData\n    _data_df_name = \"df\"\n\n    def __init__(self, settings=None, verbose: bool = False,):\n        super().__init__(model=\"legacy\", settings=settings, verbose=verbose)\n\n    def fit(\n        self, \n        baseline_data: BillingBaselineData, \n        ignore_disqualification: bool = False\n    ) -> BillingModel:\n        return super().fit(baseline_data, ignore_disqualification=ignore_disqualification)\n\n    def predict(\n        self,\n        reporting_data: BillingBaselineData | BillingReportingData,\n        aggregation: str | None = None,\n        ignore_disqualification: bool = False,\n    ) -> pd.DataFrame:\n        \"\"\"Predicts the energy consumption using the fitted model.\n\n        Args:\n            reporting_data: The data used for prediction.\n            aggregation: The aggregation level for the prediction. One of [None, 'none', 'monthly', 'bimonthly'].\n            ignore_disqualification: Whether to ignore model disqualification. Defaults to False.\n\n        Returns:\n            Dataframe with input data along with predicted energy consumption.\n\n        Raises:\n            RuntimeError: If the model is not fitted.\n            DisqualifiedModelError: If the model is disqualified and ignore_disqualification is False.\n            TypeError: If the reporting data is not of type BillingBaselineData or BillingReportingData.\n            ValueError: If the aggregation is not one of [None, 'none', 'monthly', 'bimonthly'].\n        \"\"\"\n        if not self.is_fitted:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n\n        if self.disqualification and not ignore_disqualification:\n            raise DisqualifiedModelError(\n                \"Attempting to predict using disqualified model without setting ignore_disqualification=True\"\n            )\n\n        if not isinstance(reporting_data, (BillingBaselineData, BillingReportingData)):\n            raise TypeError(\n                \"reporting_data must be a BillingBaselineData or BillingReportingData object\"\n            )\n\n        df = getattr(reporting_data, self._data_df_name)\n        df_res = self._predict(df)\n\n        if aggregation is None:\n            agg = None\n        elif aggregation.lower() == \"none\":\n            agg = None\n        elif aggregation == \"monthly\":\n            agg = \"MS\"\n        elif aggregation == \"bimonthly\":\n            agg = \"2MS\"\n        else:\n            raise ValueError(\n                \"aggregation must be one of [None, 'monthly', 'bimonthly']\"\n            )\n\n        if agg is not None:\n            sum_quad = lambda x: np.sqrt(np.sum(np.square(x)))\n\n            season = df_res[\"season\"].resample(agg).first()\n            temperature = df_res[\"temperature\"].resample(agg).mean()\n            observed = df_res[\"observed\"].resample(agg).sum()\n            predicted = df_res[\"predicted\"].resample(agg).sum()\n            predicted_unc = df_res[\"predicted_unc\"].resample(agg).apply(sum_quad)\n            heating_load = df_res[\"heating_load\"].resample(agg).sum()\n            cooling_load = df_res[\"cooling_load\"].resample(agg).sum()\n            model_split = df_res[\"model_split\"].resample(agg).first()\n            model_type = df_res[\"model_type\"].resample(agg).first()\n\n            df_res = pd.concat(\n                [\n                    season,\n                    temperature,\n                    observed,\n                    predicted,\n                    predicted_unc,\n                    heating_load,\n                    cooling_load,\n                    model_split,\n                    model_type,\n                ],\n                axis=1,\n            )\n\n        return df_res\n\n    def plot(\n        self,\n        data,\n        aggregation: str | None = None,\n    ):\n        \"\"\"Plot a model fit with baseline or reporting data. Requires matplotlib to use.\n\n        Args:\n            df_eval: The baseline or reporting data object to plot.\n            aggregation: The aggregation level for the prediction. One of [None, 'none', 'monthly', 'bimonthly'].\n        \"\"\"\n        try:\n            from opendsm.eemeter.models.billing.plot import plot\n        except ImportError:  # pragma: no cover\n            raise ImportError(\"matplotlib is required for plotting.\")\n\n        # TODO: pass more kwargs to plotting function\n\n        plot(self, self.predict(data, aggregation=aggregation))\n\n    def to_dict(self) -> dict:\n        \"\"\"Returns a dictionary of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        model_dict = super().to_dict()\n        model_dict[\"settings\"][\"developer_mode\"] = True\n\n        return model_dict\n"
  },
  {
    "path": "opendsm/eemeter/models/billing/plot.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport colorsys\n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfrom opendsm.common.stats.outliers import IQR_outlier\n\nfontsize = 14\nmpl.rc(\"font\", family=\"sans-serif\")\nc = [\"tab:blue\", \"tab:green\", \"tab:purple\"]\n\n\ndef adjust_lightness(color, amount=1.0):\n    try:\n        c = mpl.colors.cnames[color]\n    except:\n        c = color\n\n    c = colorsys.rgb_to_hls(*mpl.colors.to_rgb(c))\n\n    return colorsys.hls_to_rgb(c[0], max(0, min(1, amount * c[1])), c[2])\n\n\ndef plot(\n    fit,\n    meter_eval,\n    include_resid=False,\n    plot_gaussian_ellipses=False,\n    plot_outliers=True,\n):\n    # sort meter_eval by temperature\n    meter_eval = meter_eval.sort_values(by=\"temperature\")\n\n    fig = plt.figure(figsize=(14, 4), dpi=300)\n    if include_resid:\n        gs = fig.add_gridspec(2, hspace=0, height_ratios=[2.5, 1])\n        ax = gs.subplots()\n    else:\n        ax = [fig.subplots()]\n\n    # Plot scatter and Gaussian ellipses\n    for n, season in enumerate([\"summer\", \"shoulder\", \"winter\"]):\n        color = c[n]\n        marker = \"o\"\n        s = 7**2\n        label = f\"{season}\"\n\n        meter_season = meter_eval[\n            (meter_eval[\"season\"] == season) & (meter_eval[\"observed\"].notna())\n        ]\n\n        T = meter_season[\"temperature\"].values\n        obs = meter_season[\"observed\"].values\n        model = meter_season[\"predicted\"].values\n        resid = obs - model\n\n        ax[0].scatter(T, obs, color=color, marker=marker, s=s, label=label)\n        if include_resid:\n            ax[1].scatter(T, resid, color=color, marker=marker, s=s)\n\n    # Plot models\n    for split in meter_eval[\"model_split\"].unique():\n        meter_segment = meter_eval[meter_eval[\"model_split\"] == split]\n\n        name = f\"{split}__{meter_segment['model_type'].iloc[0]}\"\n        ax[0].plot(\n            meter_segment[\"temperature\"],\n            meter_segment[\"predicted\"],\n            color=\"tab:orange\",\n            label=f\"{name}\",\n        )\n\n    # ax[0].plot(T, model[\"c_hdd_baseline\"].model, color=\"tab:red\", label=f\"c_hdd_baseline\")\n\n    if include_resid:\n        ax[1].axhline(y=0, linestyle=(0, (5, 1)), linewidth=1.5, color=(0.4, 0.4, 0.4))\n        ax[0].get_shared_x_axes().join(ax[0], ax[1])\n        ax[1].set_xlabel(\"Temperature\", labelpad=10, fontsize=fontsize)\n        ax[1].set_ylabel(\"Resid\", labelpad=10, fontsize=fontsize)\n\n    else:\n        ax[0].set_xlabel(\"Temperature\", labelpad=10, fontsize=fontsize)\n\n    # ax.plot(hours, meter[:,2], linewidth=1.5, linestyle=(0, (6, 1)), color='firebrick')\n    # ax.plot(hours, meter[:,2], linewidth=2.0, linestyle='-.')\n    # ax.fill_between(hours, cg_lb, cg_ub, alpha=0.3, facecolor='peru')\n\n    # ax.set_xlim([T[0], T[-1]])\n    # ax.set_xticks(np.arange(0, 505, 168))\n    ax[0].tick_params(axis=\"both\", which=\"major\", labelsize=0.85 * fontsize)\n\n    if not plot_outliers:\n        # Ignores crazy points when plotting based on iqr\n        ylim = IQR_outlier(\n            meter_eval[\"observed\"].values, sigma_threshold=1.0, quantile=0.025\n        )\n        ylim_idx = [\n            np.argmin(np.abs(x - meter_eval[\"observed\"].values), axis=0) for x in ylim\n        ]\n        ylim = meter_eval[\"observed\"].values[ylim_idx]\n    else:\n        ylim = np.quantile(meter_eval[\"observed\"], [0, 1])\n\n    ylim_border = 0.1 * (ylim[1] - ylim[0])\n    ax[0].set_ylim([ylim[0] - ylim_border, ylim[1] + ylim_border])\n    # ax.xaxis.set_minor_locator(mpl.ticker.AutoMinorLocator(7))\n    # ax.tick_params(axis='both', which='major', labelsize=0.85*fontsize)\n    # ax.yaxis.set_tick_params(which='minor', left=False)\n    ax[0].set_ylabel(\"Usage\", labelpad=10, fontsize=fontsize)\n\n    legend = ax[0].legend(framealpha=0.0, fontsize=0.5 * fontsize)\n    # legend._legend_box.align = 'left'\n\n    plt.show()\n\n    # if figsize is None:\n    #     figsize = (10, 4)\n\n    # if ax is None:\n    #     fig, ax = plt.subplots(figsize=figsize)\n\n    # color = \"C1\"\n    # alpha = 1\n\n    # temp_min, temp_max = (30, 90) if temp_range is None else temp_range\n\n    # temps = np.arange(temp_min, temp_max)\n\n    # prediction_index = pd.date_range(\n    #     \"2017-01-01T00:00:00Z\", periods=len(temps), freq=\"D\"\n    # )\n\n    # temps_daily = pd.Series(temps, index=prediction_index).resample(\"D\").mean()\n    # prediction = self._predict(temps_daily).model\n\n    # plot_kwargs = {\"color\": color, \"alpha\": alpha or 0.3}\n    # ax.plot(temps, prediction, **plot_kwargs)\n\n    # if title is not None:\n    #     ax.set_title(title)\n\n    # return ax\n"
  },
  {
    "path": "opendsm/eemeter/models/billing/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom opendsm.common.base_settings import CustomField\n\nfrom opendsm.eemeter.models.daily.utilities.settings import DailyLegacySettings\n\n\n\nclass BillingSettings(DailyLegacySettings):\n    segment_minimum_count: int = CustomField(\n        default=3,\n        ge=3,\n        developer=True,\n        description=\"Minimum number of data points for HDD/CDD\",\n    )"
  },
  {
    "path": "opendsm/eemeter/models/billing/weighted_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\nfrom opendsm.eemeter.models.billing.data import (\n    BillingBaselineData,\n    BillingReportingData,\n)\nfrom opendsm.eemeter.models.billing.settings import BillingSettings\nfrom opendsm.eemeter.models.daily.model import DailyModel\n\n\nclass BillingWeightedModel(DailyModel):\n    \"\"\"A class to fit a model to the input meter data.\n\n    BillingModel is a wrapper for the DailyModel class using billing presets.\n\n    Attributes:\n        settings (dict): A dictionary of settings.\n        seasonal_options (list): A list of seasonal options (su: Summer, sh: Shoulder, wi: Winter).\n            Elements in the list are seasons separated by '_' that represent a model split.\n            For example, a list of ['su_sh', 'wi'] represents two splits: summer/shoulder and winter.\n        day_options (list): A list of day options.\n        combo_dictionary (dict): A dictionary of combinations.\n        df_meter (pandas.DataFrame): A dataframe of meter data.\n        error (dict): A dictionary of error metrics.\n        combinations (list): A list of combinations.\n        components (list): A list of components.\n        fit_components (list): A list of fit components.\n        wRMSE_base (float): The mean bias error for no splits.\n        best_combination (list): The best combination of splits.\n        model (sklearn.pipeline.Pipeline): The final fitted model.\n        id (str): The index of the meter data.\n    \"\"\"\n\n    _baseline_data_type = BillingBaselineData\n    _reporting_data_type = BillingReportingData\n    _data_df_name = \"billing_df\"\n\n    # TODO: lot of duplicated code between this and daily model, refactor later\n    def __init__(\n        self,\n        settings: dict | None = None,\n        verbose: bool = False,\n    ):\n        super().__init__(model=\"legacy\", settings=settings, verbose=verbose)\n\n        print(\"The weighted billing model is under development and is not ready for public use.\")\n\n    def _initialize_settings(\n        self,\n        model: str = \"current\",\n        settings: dict | None = None\n    ) -> None:\n\n        # Note: Model designates the base settings, it can be 'current' or 'legacy'\n        #       Settings is to be a dictionary of settings to be changed\n\n        if settings is None:\n            settings = {}\n\n        self.settings = BillingSettings(**settings)\n\n    def fit(\n        self, \n        baseline_data: BillingBaselineData, \n        ignore_disqualification: bool = False\n    ) -> BillingWeightedModel:\n        return super().fit(baseline_data, ignore_disqualification=ignore_disqualification)\n\n    def predict(\n        self,\n        reporting_data: BillingBaselineData | BillingReportingData,\n        aggregation: str | None = None,\n        ignore_disqualification: bool = False,\n    ) -> pd.DataFrame:\n        \"\"\"Predicts the energy consumption using the fitted model.\n\n        Args:\n            reporting_data: The data used for prediction.\n            aggregation: The aggregation level for the prediction. One of [None, 'none', 'monthly', 'bimonthly'].\n            ignore_disqualification: Whether to ignore model disqualification. Defaults to False.\n\n        Returns:\n            Dataframe with input data along with predicted energy consumption.\n\n        Raises:\n            RuntimeError: If the model is not fitted.\n            DisqualifiedModelError: If the model is disqualified and ignore_disqualification is False.\n            TypeError: If the reporting data is not of type BillingBaselineData or BillingReportingData.\n            ValueError: If the aggregation is not one of [None, 'none', 'monthly', 'bimonthly'].\n        \"\"\"\n        if not self.is_fitted:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n\n        if self.disqualification and not ignore_disqualification:\n            raise DisqualifiedModelError(\n                \"Attempting to predict using disqualified model without setting ignore_disqualification=True\"\n            )\n\n        if not isinstance(reporting_data, (BillingBaselineData, BillingReportingData)):\n            raise TypeError(\n                \"reporting_data must be a BillingBaselineData or BillingReportingData object\"\n            )\n\n        df = getattr(reporting_data, self._data_df_name)\n        df_res = self._predict(df)\n\n        if aggregation is None:\n            agg = None\n        elif aggregation.lower() == \"none\":\n            agg = None\n        elif aggregation == \"monthly\":\n            agg = \"MS\"\n        elif aggregation == \"bimonthly\":\n            agg = \"2MS\"\n        else:\n            raise ValueError(\n                \"aggregation must be one of [None, 'monthly', 'bimonthly']\"\n            )\n\n        if agg is not None:\n            sum_quad = lambda x: np.sqrt(np.sum(np.square(x)))\n\n            season = df_res[\"season\"].resample(agg).first()\n            temperature = df_res[\"temperature\"].resample(agg).mean()\n            observed = df_res[\"observed\"].resample(agg).sum()\n            predicted = df_res[\"predicted\"].resample(agg).sum()\n            predicted_unc = df_res[\"predicted_unc\"].resample(agg).apply(sum_quad)\n            heating_load = df_res[\"heating_load\"].resample(agg).sum()\n            cooling_load = df_res[\"cooling_load\"].resample(agg).sum()\n            model_split = df_res[\"model_split\"].resample(agg).first()\n            model_type = df_res[\"model_type\"].resample(agg).first()\n\n            df_res = pd.concat(\n                [\n                    season,\n                    temperature,\n                    observed,\n                    predicted,\n                    predicted_unc,\n                    heating_load,\n                    cooling_load,\n                    model_split,\n                    model_type,\n                ],\n                axis=1,\n            )\n\n        return df_res\n\n    def plot(\n        self,\n        data,\n        aggregation: str | None = None,\n    ):\n        \"\"\"Plot a model fit with baseline or reporting data. Requires matplotlib to use.\n\n        Args:\n            df_eval: The baseline or reporting data object to plot.\n            aggregation: The aggregation level for the prediction. One of [None, 'none', 'monthly', 'bimonthly'].\n        \"\"\"\n        try:\n            from opendsm.eemeter.models.billing.plot import plot\n        except ImportError:  # pragma: no cover\n            raise ImportError(\"matplotlib is required for plotting.\")\n\n        # TODO: pass more kwargs to plotting function\n\n        plot(self, self.predict(data, aggregation=aggregation))\n\n    def to_dict(self) -> dict:\n        \"\"\"Returns a dictionary of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        model_dict = super().to_dict()\n        model_dict[\"settings\"][\"developer_mode\"] = True\n\n        return model_dict\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .data import DailyBaselineData, DailyReportingData\nfrom .model import DailyModel\n\n__all__ = (\n    \"DailyBaselineData\",\n    \"DailyReportingData\",\n    \"DailyModel\",\n)\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/base_models/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/base_models/c_hdd_tidd.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom math import isclose\nfrom typing import Optional\n\nimport numba\nimport numpy as np\n\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.eemeter.models.daily.base_models.full_model import full_model\nfrom opendsm.eemeter.models.daily.base_models.hdd_tidd_cdd import full_model_weight\nfrom opendsm.eemeter.models.daily.objective_function import obj_fcn_decorator\nfrom opendsm.eemeter.models.daily.utilities.opt_settings import OptimizationSettings\nfrom opendsm.eemeter.models.daily.optimize import InitialGuessOptimizer, Optimizer\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients, ModelType\nfrom opendsm.eemeter.models.daily.utilities.base_model import (\n    fix_identical_bnds,\n    get_intercept,\n    get_slope,\n    get_T_bnds,\n    linear_fit,\n)\n\n\ndef fit_c_hdd_tidd(\n    T,\n    obs,\n    weights,\n    settings,\n    opt_options,\n    smooth,\n    x0: Optional[ModelCoefficients] = None,\n    bnds=None,\n    initial_fit=False,\n):\n    \"\"\"\n    This function fits the HDD TIDD smooth model to the given data.\n    Parameters:\n    T (array-like): The independent variable data - temperature.\n    obs (array-like): The dependent variable data - observed.\n    settings (object): An object containing various settings for the model fitting.\n    opt_options (dict): A dictionary containing options for the optimization process.\n    x0 (ModelCoefficients, optional): Initial model coefficients. If None, they will be estimated.\n    bnds (list of tuples, optional): Bounds for the optimization process. If None, they will be estimated.\n    initial_fit (bool, optional): If True, the function performs an initial fit. Default is False.\n    Returns:\n    res (OptimizeResult): The result of the optimization process.\n    \"\"\"\n\n    if initial_fit:\n        alpha = settings.alpha_selection\n    else:\n        alpha = settings.alpha_final\n\n    if x0 is None:\n        x0 = _c_hdd_tidd_x0(T, obs, alpha, settings, smooth)\n    else:\n        x0 = _c_hdd_tidd_x0_final(T, obs, x0, alpha, settings)\n\n    if x0.model_type in [ModelType.HDD_TIDD_SMOOTH, ModelType.HDD_TIDD]:\n        tdd_beta = x0.hdd_beta\n    elif x0.model_type in [ModelType.TIDD_CDD_SMOOTH, ModelType.TIDD_CDD]:\n        tdd_beta = x0.cdd_beta\n    else:\n        raise ValueError\n\n    # limit slope based on initial regression & configurable order of magnitude\n    max_slope = np.abs(tdd_beta) + 10 ** (\n        np.log10(np.abs(tdd_beta)) + np.log10(settings.maximum_slope_oom_scalar)\n    )\n\n    # initial fit bounded by Tmin:Tmax, final fit has minimum T segment buffer\n    T_initial, T_segment = get_T_bnds(T, settings)\n    c_hdd_bnds = T_initial if initial_fit else T_segment\n\n    # set bounds and alter coefficient guess for single slope models w/o an intercept segment\n    if not smooth and not initial_fit:\n        T_min, T_max = T_initial\n        T_min_seg, T_max_seg = T_segment\n        rtol = 1e-5\n        if x0.model_type is ModelType.HDD_TIDD and (\n            x0.hdd_bp >= T_max_seg or isclose(x0.hdd_bp, T_max_seg, rel_tol=rtol)\n        ):\n            # model is heating only, and breakpoint is approximately within max temp buffer\n            x0.intercept -= x0.hdd_bp * T_max\n            x0.hdd_bp = T_max\n            c_hdd_bnds = [T_max, T_max]\n        if x0.model_type is ModelType.TIDD_CDD and (\n            x0.cdd_bp <= T_min_seg or isclose(x0.cdd_bp, T_min_seg, rel_tol=rtol)\n        ):\n            # model is cooling only, and breakpoint is approximately within min temp buffer\n            x0.intercept -= x0.cdd_bp * T_min\n            x0.cdd_bp = T_min\n            c_hdd_bnds = [T_min, T_min]\n\n    # not known whether heating or cooling model on initial fit\n    if initial_fit:\n        c_hdd_beta_bnds = [-max_slope, max_slope]\n    # stick with heating/cooling if using existing x0\n    elif tdd_beta < 0:\n        c_hdd_beta_bnds = [-max_slope, 0]\n    else:\n        c_hdd_beta_bnds = [0, max_slope]\n\n    intercept_bnds = np.quantile(obs, [0.01, 0.99])\n    if smooth:\n        c_hdd_k_bnds = [0, 1e3]\n        bnds_0 = [c_hdd_bnds, c_hdd_beta_bnds, c_hdd_k_bnds, intercept_bnds]\n    else:\n        bnds_0 = [c_hdd_bnds, c_hdd_beta_bnds, intercept_bnds]\n\n    bnds = _c_hdd_tidd_update_bnds(bnds, bnds_0, smooth)\n    if (\n        c_hdd_bnds[0] == c_hdd_bnds[1]\n    ):  # if breakpoint bounds are identical, don't expand\n        bnds[0, :] = c_hdd_bnds\n\n    if smooth:\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"]\n        model_fcn = _c_hdd_tidd_smooth\n        weight_fcn = _c_hdd_tidd_smooth_weight\n        TSS_fcn = None\n    else:\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"intercept\"]\n        model_fcn = _c_hdd_tidd\n        weight_fcn = _c_hdd_tidd_weight\n        TSS_fcn = _c_hdd_tidd_total_sum_of_squares\n    obj_fcn = obj_fcn_decorator(\n        model_fcn, weight_fcn, TSS_fcn, T, obs, weights, settings, alpha, coef_id, initial_fit\n    )\n    res = Optimizer(\n        obj_fcn, x0.to_np_array(), bnds, coef_id, settings, opt_options\n    ).run()\n\n    return res\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef set_full_model_coeffs_smooth(c_hdd_bp, c_hdd_beta, c_hdd_k, intercept):\n    \"\"\"\n    This function sets the smoothed full model coefficients based on the given parameters.\n    Parameters:\n    c_hdd_bp (float): The base point coefficient for heating and cooling degree days.\n    c_hdd_beta (float): The beta coefficient for heating and cooling degree days.\n    c_hdd_k (float): The k coefficient for heating and cooling degree days.\n    intercept (float): The intercept of the model.\n    Returns:\n    np.array: An array containing the coefficients for the full model.\n    \"\"\"\n    hdd_bp = cdd_bp = c_hdd_bp\n\n    if c_hdd_beta < 0:\n        hdd_beta = -c_hdd_beta\n        hdd_k = c_hdd_k\n        cdd_beta = cdd_k = 0.0\n\n    else:\n        cdd_beta = c_hdd_beta\n        cdd_k = c_hdd_k\n        hdd_beta = hdd_k = 0.0\n\n    return np.array([hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept])\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef set_full_model_coeffs(c_hdd_bp, c_hdd_beta, intercept):\n    \"\"\"\n    This function sets the full model coefficients based on the given parameters.\n    Parameters:\n    c_hdd_bp (float): The base point coefficient for heating and cooling degree days.\n    c_hdd_beta (float): The beta coefficient for heating and cooling degree days.\n    intercept (float): The intercept of the model.\n    Returns:\n    np.array: An array containing the coefficients for the full model.\n    \"\"\"\n\n    return set_full_model_coeffs_smooth(c_hdd_bp, c_hdd_beta, 0.0, intercept)\n\n\ndef _c_hdd_tidd_update_bnds(new_bnds, bnds, smooth):\n    \"\"\"\n    This function updates the boundaries of the new_bnds array based on the given bnds array.\n    It sorts the new_bnds array along the axis=1, fixes any identical boundaries, and ensures that the lower boundary is non-negative.\n    Parameters:\n    new_bnds (numpy.ndarray): The array of new boundaries to be updated.\n    bnds (numpy.ndarray): The array of existing boundaries used for updating.\n    Returns:\n    new_bnds (numpy.ndarray): The updated array of new boundaries.\n    \"\"\"\n\n    if new_bnds is None:\n        new_bnds = bnds\n\n    # breakpoint bounds\n    new_bnds[0] = bnds[0]\n\n    # intercept bnds at index 3 for smooth, 2 for unsmooth\n    if smooth:\n        new_bnds[3] = bnds[3]\n    else:\n        new_bnds[2] = bnds[2]\n\n    new_bnds = np.sort(new_bnds, axis=1)\n    new_bnds = fix_identical_bnds(new_bnds)\n\n    # check for negative k bound if using smoothed model\n    if smooth and new_bnds[2, 0] < 0:\n        new_bnds[2, 0] = 0\n\n    return new_bnds\n\n\ndef _tdd_coefficients(\n    intercept, c_hdd_bp, c_hdd_beta, c_hdd_k=None\n) -> ModelCoefficients:\n    \"\"\"\n    infer cdd vs hdd given positive or negative slope.\n    if slope is 0, model will be reduced later\n    \"\"\"\n    if c_hdd_beta < 0:\n        hdd_beta = c_hdd_beta\n        hdd_bp = c_hdd_bp\n        hdd_k = c_hdd_k\n        cdd_beta = None\n        cdd_bp = None\n        cdd_k = None\n        if c_hdd_k is not None:\n            model_type = ModelType.HDD_TIDD_SMOOTH\n        else:\n            model_type = ModelType.HDD_TIDD\n    else:\n        cdd_beta = c_hdd_beta\n        cdd_bp = c_hdd_bp\n        cdd_k = c_hdd_k\n        hdd_beta = None\n        hdd_bp = None\n        hdd_k = None\n        if c_hdd_k is not None:\n            model_type = ModelType.TIDD_CDD_SMOOTH\n        else:\n            model_type = ModelType.TIDD_CDD\n\n    return ModelCoefficients(\n        model_type=model_type,\n        intercept=intercept,\n        hdd_bp=hdd_bp,\n        hdd_beta=hdd_beta,\n        hdd_k=hdd_k,\n        cdd_bp=cdd_bp,\n        cdd_beta=cdd_beta,\n        cdd_k=cdd_k,\n    )\n\n\ndef _c_hdd_tidd_x0(T, obs, alpha, settings, smooth):\n    min_T_idx = settings.segment_minimum_count\n\n    # c_hdd_bp = initial_guess_bp_1(T, obs, s=2, int_method=\"trapezoid\")\n    c_hdd_bp = _c_hdd_tidd_bp0(T, obs, alpha, settings)\n    c_hdd_bp = np.clip([c_hdd_bp], T[min_T_idx - 1], T[-min_T_idx])[0]\n\n    idx_hdd = np.argwhere(T <= c_hdd_bp).flatten()\n    idx_cdd = np.argwhere(T >= c_hdd_bp).flatten()\n\n    hdd_beta, _ = linear_fit(T[idx_hdd], obs[idx_hdd], alpha)\n    if hdd_beta > 0:\n        hdd_beta = 0\n\n    cdd_beta, _ = linear_fit(T[idx_cdd], obs[idx_cdd], alpha)\n    if cdd_beta < 0:\n        cdd_beta = 0\n\n    # choose heating vs cooling based on larger slope\n    # treat opposite degree days as flat tidd\n    if -hdd_beta >= cdd_beta:\n        c_hdd_beta = hdd_beta\n        intercept = np.median(obs[idx_cdd])\n\n    else:\n        c_hdd_beta = cdd_beta\n        intercept = np.median(obs[idx_hdd])\n\n    c_hdd_k = None\n    if smooth:\n        c_hdd_k = 0.0\n\n    return _tdd_coefficients(\n        intercept=intercept,\n        c_hdd_bp=c_hdd_bp,\n        c_hdd_beta=c_hdd_beta,\n        c_hdd_k=c_hdd_k,\n    )\n\n\ndef _c_hdd_tidd_x0_final(T, obs, x0, alpha, settings):\n    c_hdd_k = None\n    if x0.is_smooth:\n        c_hdd_bp, c_hdd_beta, c_hdd_k, intercept = x0.to_np_array()\n    else:\n        c_hdd_bp, c_hdd_beta, intercept = x0.to_np_array()\n\n    min_T_idx = settings.segment_minimum_count\n    idx_hdd = np.argwhere(T <= c_hdd_bp).flatten()\n    idx_cdd = np.argwhere(T >= c_hdd_bp).flatten()\n\n    # can use model type to do this\n    # if x0.model_type in [ModelType.HDD_TIDD_SMOOTH, ModelType.HDD_TIDD]:  etc\n    if (c_hdd_beta < 0) and (len(idx_hdd) >= min_T_idx):  # hdd\n        c_hdd_beta = get_slope(T[idx_hdd], obs[idx_hdd], c_hdd_bp, intercept, alpha)\n\n    elif (c_hdd_beta >= 0) and (len(idx_cdd) >= min_T_idx):  # cdd\n        c_hdd_beta = get_slope(T[idx_cdd], obs[idx_cdd], c_hdd_bp, intercept, alpha)\n\n    return _tdd_coefficients(\n        c_hdd_bp=c_hdd_bp, c_hdd_beta=c_hdd_beta, c_hdd_k=c_hdd_k, intercept=intercept\n    )\n\n\ndef _c_hdd_tidd_bp0(T, obs, alpha, settings, min_weight=0.0):\n    min_T_idx = settings.segment_minimum_count\n\n    idx_sorted = np.argsort(T).flatten()\n    T = T[idx_sorted]\n    obs = obs[idx_sorted]\n\n    T_fit_bnds = np.array([T[0], T[-1]])\n\n    def bp_obj_fcn_dec(T, obs):\n        def bp_obj_fcn(x, grad=[]):\n            [c_hdd_bp] = x\n\n            idx_hdd = np.argwhere(T <= c_hdd_bp).flatten()\n            idx_cdd = np.argwhere(T >= c_hdd_bp).flatten()\n\n            hdd_beta, _ = linear_fit(T[idx_hdd], obs[idx_hdd], alpha)\n            if hdd_beta > 0:\n                hdd_beta = 0\n\n            cdd_beta, _ = linear_fit(T[idx_cdd], obs[idx_cdd], alpha)\n            if cdd_beta < 0:\n                cdd_beta = 0\n\n            if -hdd_beta >= cdd_beta:\n                c_hdd_beta = hdd_beta\n                intercept = get_intercept(obs[idx_cdd], alpha)\n\n            else:\n                c_hdd_beta = cdd_beta\n                intercept = get_intercept(obs[idx_hdd], alpha)\n\n            model = _c_hdd_tidd(\n                c_hdd_bp, c_hdd_beta, intercept, T_fit_bnds=T_fit_bnds, T=T\n            )\n\n            resid = model - obs\n            weight, _, _ = adaptive_weights(\n                resid, alpha=alpha, sigma=2.698, quantile=0.25, min_weight=min_weight\n            )\n\n            loss = np.sum(weight * (resid) ** 2)\n\n            return loss\n\n        return bp_obj_fcn\n\n    obj_fcn = bp_obj_fcn_dec(T, obs)\n\n    T_min = T[min_T_idx - 1]\n    T_max = T[-min_T_idx]\n    T_range = T_max - T_min\n\n    x0 = np.array([T_range * 0.5]) + T_min\n    bnds = np.array([[T_min, T_max]])\n\n    opt_settings = OptimizationSettings(\n        algorithm=settings.initial_guess_algorithm_choice,\n        stop_criteria_type=\"iteration maximum\",\n        stop_criteria_value=100,\n        initial_step=settings.initial_step_percentage,\n        x_tol_rel=1e-3,\n        f_tol_rel=0.5,\n    )\n\n    res = InitialGuessOptimizer(\n        obj_fcn, x0, bnds, opt_settings\n    ).run()\n\n    return res.x[0]\n\n\ndef _c_hdd_tidd(\n    c_hdd_bp, c_hdd_beta, intercept, T_fit_bnds=np.array([]), T=np.array([])\n):\n    model_vars = set_full_model_coeffs(c_hdd_bp, c_hdd_beta, intercept)\n    return full_model(*model_vars, T_fit_bnds, T)\n\n\ndef _c_hdd_tidd_smooth(\n    c_hdd_bp, c_hdd_beta, c_hdd_k, intercept, T_fit_bnds=np.array([]), T=np.array([])\n):\n    x = set_full_model_coeffs_smooth(c_hdd_bp, c_hdd_beta, c_hdd_k, intercept)\n    return full_model(*x, T_fit_bnds, T)\n\n\ndef _c_hdd_tidd_weight(\n    c_hdd_bp,\n    c_hdd_beta,\n    intercept,\n    T,\n    residual,\n    sigma=3.0,\n    quantile=0.25,\n    alpha=2.0,\n    min_weight=0.0,\n):\n    model_vars = set_full_model_coeffs(c_hdd_bp, c_hdd_beta, intercept)\n    return full_model_weight(\n        *model_vars, T, residual, sigma, quantile, alpha, min_weight\n    )\n\n\ndef _c_hdd_tidd_smooth_weight(\n    c_hdd_bp,\n    c_hdd_beta,\n    c_hdd_k,\n    intercept,\n    T,\n    residual,\n    sigma=3.0,\n    quantile=0.25,\n    alpha=2.0,\n    min_weight=0.0,\n):\n    \"\"\"\n    This function calculates the weight for the full model using the given parameters.\n    Parameters:\n    c_hdd_bp (float): The base point for the HDD.\n    c_hdd_beta (float): The beta value for the HDD.\n    c_hdd_k (float): The k value for the HDD.\n    intercept (float): The intercept for the model.\n    T (float): The temperature.\n    residual (float): The residual value.\n    sigma (float, optional): The sigma value. Default is 3.0.\n    quantile (float, optional): The quantile value. Default is 0.25.\n    alpha (float, optional): The alpha value. Default is 2.0.\n    min_weight (float, optional): The minimum weight. Default is 0.0.\n    Returns:\n    float: The calculated weight for the full model.\n    \"\"\"\n\n    model_vars = set_full_model_coeffs_smooth(c_hdd_bp, c_hdd_beta, c_hdd_k, intercept)\n    return full_model_weight(\n        *model_vars, T, residual, sigma, quantile, alpha, min_weight\n    )\n\n\ndef _c_hdd_tidd_total_sum_of_squares(c_hdd_bp, c_hdd_beta, intercept, T, obs):\n    idx_bp = np.argmin(np.abs(T - c_hdd_bp))\n\n    TSS = []\n    for observed in [obs[:idx_bp], obs[idx_bp:]]:\n        if len(observed) == 0:\n            continue\n\n        TSS.append(np.sum((observed - np.mean(observed)) ** 2))\n\n    TSS = np.sum(TSS)\n\n    return TSS\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/base_models/full_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numba\nimport numpy as np\n\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.common.utils import LN_MAX_POS_SYSTEM_VALUE, LN_MIN_POS_SYSTEM_VALUE\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef full_model(\n    hdd_bp,\n    hdd_beta,\n    hdd_k,\n    cdd_bp,\n    cdd_beta,\n    cdd_k,\n    intercept,\n    T_fit_bnds=np.array([]),\n    T=np.array([]),\n):\n    \"\"\"\n    This function predicts the total energy consumption based on the given parameters.\n\n    Parameters:\n    hdd_bp (float): The base point for the heating model.\n    hdd_beta (float): The beta value for the heating model.\n    hdd_k (float): The k value for the heating model.\n    cdd_bp (float): The base point for the cooling model.\n    cdd_beta (float): The beta value for the cooling model.\n    cdd_k (float): The k value for the cooling model.\n    intercept (float): The intercept value for the model.\n    T_fit_bnds (numpy array): The temperature bounds for the model fitting. Default is an empty numpy array.\n    T (numpy array): The temperature values. Default is an empty numpy array.\n\n    Returns:\n    numpy array: The total energy consumption for each temperature value in T.\n    \"\"\"\n\n    # if all variables are zero, return tidd model\n    if (hdd_beta == 0) and (cdd_beta == 0):\n        return np.ones_like(T) * intercept\n\n    [T_min, T_max] = T_fit_bnds\n\n    if cdd_bp < hdd_bp:\n        hdd_bp, cdd_bp = cdd_bp, hdd_bp\n        hdd_beta, cdd_beta = cdd_beta, hdd_beta\n        hdd_k, cdd_k = cdd_k, hdd_k\n\n    E_tot = np.empty_like(T)\n    for n, Ti in enumerate(T):\n        if (Ti < hdd_bp) or (\n            (hdd_bp == cdd_bp) and (cdd_bp >= T_max)\n        ):  # Temperature is within the heating model\n            T_bp = hdd_bp\n            beta = -hdd_beta\n            k = hdd_k\n\n        elif (Ti > cdd_bp) or (\n            (hdd_bp == cdd_bp) and (hdd_bp <= T_min)\n        ):  # Temperature is within the cooling model\n            T_bp = cdd_bp\n            beta = cdd_beta\n            k = -cdd_k\n\n        else:  # Temperature independent\n            beta = 0.0\n\n        # Evaluate\n        if beta == 0:  # tidd\n            E_tot[n] = intercept\n\n        elif k == 0:  # c_hdd\n            E_tot[n] = beta * (Ti - T_bp) + intercept\n\n        else:  # smoothed c_hdd\n            c_hdd = beta * (Ti - T_bp) + intercept\n\n            exp_interior = 1 / k * (Ti - T_bp)\n            exp_interior = np.clip(\n                exp_interior, LN_MIN_POS_SYSTEM_VALUE, LN_MAX_POS_SYSTEM_VALUE\n            )\n            E_tot[n] = abs(beta * k) * (np.exp(exp_interior) - 1) + c_hdd\n\n    return E_tot\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef get_full_model_x(model_key, x, T_min, T_max, T_min_seg, T_max_seg):\n    \"\"\"\n    This function adjusts the parameters of a full model based on certain conditions.\n\n    Parameters:\n    x (list): A list containing the parameters of the model.\n    T_min_seg (float): The minimum temperature segment.\n    T_max_seg (float): The maximum temperature segment.\n\n    Returns:\n    list: A list of adjusted parameters.\n\n    \"\"\"\n\n    if model_key == \"hdd_tidd_cdd_smooth\":\n        [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept] = x\n\n    elif model_key == \"hdd_tidd_cdd\":\n        [hdd_bp, hdd_beta, cdd_bp, cdd_beta, intercept] = x\n        hdd_k = cdd_k = 0.0\n\n    elif model_key == \"c_hdd_tidd_smooth\":\n        [c_hdd_bp, c_hdd_beta, c_hdd_k, intercept] = x\n        hdd_bp = cdd_bp = c_hdd_bp\n\n        if c_hdd_beta < 0:\n            hdd_beta = -c_hdd_beta\n            hdd_k = c_hdd_k\n            cdd_beta = cdd_k = 0.0\n\n        else:\n            cdd_beta = c_hdd_beta\n            cdd_k = c_hdd_k\n            hdd_beta = hdd_k = 0.0\n\n    elif model_key == \"c_hdd_tidd\":\n        [c_hdd_bp, c_hdd_beta, intercept] = x\n\n        if c_hdd_bp < T_min_seg:\n            cdd_bp = hdd_bp = T_min_seg\n        elif c_hdd_bp > T_max_seg:\n            cdd_bp = hdd_bp = T_max_seg\n        else:\n            hdd_bp = cdd_bp = c_hdd_bp\n\n        if c_hdd_beta < 0:\n            hdd_beta = -c_hdd_beta\n            cdd_beta = cdd_k = hdd_k = 0.0\n\n        else:\n            cdd_beta = c_hdd_beta\n            hdd_beta = hdd_k = cdd_k = 0.0\n\n    elif model_key == \"tidd\":\n        [intercept] = x\n        hdd_bp = hdd_beta = hdd_k = cdd_bp = cdd_beta = cdd_k = 0.0\n\n    x = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n    return fix_full_model_x(x, T_min, T_max)\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef fix_full_model_x(x, T_min_seg, T_max_seg):\n    \"\"\"\n    This function adjusts the parameters of a full model based on certain conditions.\n\n    Parameters:\n    x (list): A list containing the parameters of the model [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept].\n    T_min_seg (float): The minimum temperature segment.\n    T_max_seg (float): The maximum temperature segment.\n\n    Returns:\n    list: A list of adjusted parameters [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept].\n\n    \"\"\"\n\n    hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept = x\n\n    # swap breakpoint order if they are reversed [hdd, cdd]\n    if cdd_bp < hdd_bp:\n        hdd_bp, cdd_bp = cdd_bp, hdd_bp\n        hdd_beta, cdd_beta = cdd_beta, hdd_beta\n        hdd_k, cdd_k = cdd_k, hdd_k\n\n    # if there is a slope, but the breakpoint is at the end, it's a c_hdd_tidd model\n    if hdd_bp != cdd_bp:\n        if cdd_bp >= T_max_seg:\n            cdd_beta = 0.0\n        elif hdd_bp <= T_min_seg:\n            hdd_beta = 0.0\n\n    # if slopes are zero then smoothing is zero\n    if hdd_beta == 0:\n        hdd_k = 0.0\n\n    if cdd_beta == 0:\n        cdd_k = 0.0\n\n    return [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n\ndef full_model_weight(\n    hdd_bp,\n    hdd_beta,\n    hdd_k,\n    cdd_bp,\n    cdd_beta,\n    cdd_k,\n    intercept,\n    T,\n    residual,\n    sigma=3.0,\n    quantile=0.25,\n    alpha=2.0,\n    min_weight=0.0,\n):\n    \"\"\"\n    This function calculates the weights, C and alpha for a full model using adaptive weights\n\n    Parameters:\n    hdd_bp (float): The base point for heating degree days.\n    hdd_beta (float): The beta value for heating degree days.\n    hdd_k (float): The k value for heating degree days.\n    cdd_bp (float): The base point for cooling degree days.\n    cdd_beta (float): The beta value for cooling degree days.\n    cdd_k (float): The k value for cooling degree days.\n    intercept (float): The intercept of the model.\n    T (array-like): The temperature array.\n    residual (array-like): The residual array.\n    weights (array-like): The input weights array.\n    sigma (float, optional): The standard deviation. Default is 3.0.\n    quantile (float, optional): The quantile to be used. Default is 0.25.\n    alpha (float, optional): The alpha value. Default is 2.0.\n    min_weight (float, optional): The minimum weight. Default is 0.0.\n\n    Returns:\n    tuple: Returns a tuple containing the weights, C and alpha for the full model.\n    \"\"\"\n\n    if hdd_bp > cdd_bp:\n        hdd_bp, cdd_bp = cdd_bp, hdd_bp\n\n    if (hdd_beta == 0) and (cdd_beta == 0):  # intercept only\n        resid_all = [residual]\n\n    elif (cdd_bp >= T[-1]) or (hdd_bp <= T[0]):  # hdd or cdd only\n        resid_all = [residual]\n\n    elif hdd_beta == 0:\n        idx_cdd_bp = np.argmin(np.abs(T - cdd_bp))\n\n        resid_all = [residual[:idx_cdd_bp], residual[idx_cdd_bp:]]\n\n    elif cdd_beta == 0:\n        idx_hdd_bp = np.argmin(np.abs(T - hdd_bp))\n\n        resid_all = [residual[:idx_hdd_bp], residual[idx_hdd_bp:]]\n\n    else:\n        idx_hdd_bp = np.argmin(np.abs(T - hdd_bp))\n        idx_cdd_bp = np.argmin(np.abs(T - cdd_bp))\n\n        if hdd_bp == cdd_bp:\n            resid_all = [residual[:idx_hdd_bp], residual[idx_cdd_bp:]]\n\n        else:\n            resid_all = [\n                residual[:idx_hdd_bp],\n                residual[idx_hdd_bp:idx_cdd_bp],\n                residual[idx_cdd_bp:],\n            ]\n\n    weight = []\n    C = []\n    a = []\n    for resid in resid_all:\n        if len(resid) == 0:\n            continue\n\n        elif len(resid) < 3:\n            weight.append(np.ones_like(resid))\n            C.append(np.ones_like(resid))\n            a.append(np.ones_like(resid) * 2.0)\n\n            continue\n\n        _weight, _C, _a = adaptive_weights(\n            resid, alpha=alpha, sigma=sigma, quantile=quantile, min_weight=min_weight\n        )\n\n        weight.append(_weight)\n        C.append(np.ones_like(resid) * _C)\n        a.append(np.ones_like(resid) * _a)\n\n    weight_out = np.hstack(weight)\n    C_out = np.hstack(weight)\n    a_out = np.hstack(a)\n\n    return weight_out, C_out, a_out\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/base_models/hdd_tidd_cdd.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom typing import Optional\n\nimport numba\nimport numpy as np\n\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.eemeter.models.daily.base_models.full_model import (\n    full_model,\n    full_model_weight,\n)\nfrom opendsm.eemeter.models.daily.objective_function import obj_fcn_decorator\nfrom opendsm.eemeter.models.daily.utilities.opt_settings import OptimizationSettings\nfrom opendsm.eemeter.models.daily.optimize import InitialGuessOptimizer, Optimizer\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients, ModelType\nfrom opendsm.eemeter.models.daily.utilities.base_model import (\n    fix_identical_bnds,\n    get_intercept,\n    get_slope,\n    get_smooth_coeffs,\n)\n\n\ndef fit_hdd_tidd_cdd(\n    T,\n    obs,\n    weights,\n    settings,\n    opt_options,\n    smooth,\n    x0: Optional[ModelCoefficients] = None,\n    bnds=None,\n    initial_fit=False,\n):\n    # assert x0 is None or x0.model_type is ModelType.HDD_TIDD_CDD_SMOOTH\n\n    if initial_fit:\n        alpha = settings.alpha_selection\n    else:\n        alpha = settings.alpha_final\n\n    if x0 is None:\n        x0 = _hdd_tidd_cdd_smooth_x0(T, obs, alpha, settings, smooth)\n\n    max_slope = np.max([x0.hdd_beta, x0.cdd_beta])\n    if max_slope != 0:\n        max_slope += 10 ** (\n            np.log10(np.abs(max_slope)) + np.log10(settings.maximum_slope_oom_scalar)\n        )\n\n    if initial_fit:\n        T_min = np.min(T)\n        T_max = np.max(T)\n    else:\n        N_min = settings.segment_minimum_count\n\n        T_min = np.partition(T, N_min)[N_min]\n        T_max = np.partition(T, -N_min)[-N_min]\n\n    c_hdd_bnds = [T_min, T_max]\n    c_hdd_beta_bnds = [0, np.abs(max_slope)]\n    intercept_bnds = np.quantile(obs, [0.01, 0.99])\n    if smooth:\n        c_hdd_k_bnds = [0, 1]\n        bnds_0 = [\n            c_hdd_bnds,\n            c_hdd_beta_bnds,\n            c_hdd_k_bnds,\n            c_hdd_bnds,\n            c_hdd_beta_bnds,\n            c_hdd_k_bnds,\n            intercept_bnds,\n        ]\n    else:\n        bnds_0 = [\n            c_hdd_bnds,\n            c_hdd_beta_bnds,\n            c_hdd_bnds,\n            c_hdd_beta_bnds,\n            intercept_bnds,\n        ]\n\n    bnds = _hdd_tidd_cdd_smooth_update_bnds(bnds, bnds_0, smooth)\n\n    if smooth:\n        coef_id = [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"hdd_k\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"cdd_k\",\n            \"intercept\",\n        ]\n        model_fcn = evaluate_hdd_tidd_cdd_smooth\n        weight_fcn = _hdd_tidd_cdd_smooth_weight\n        TSS_fcn = None\n    else:\n        coef_id = [\"hdd_bp\", \"hdd_beta\", \"cdd_bp\", \"cdd_beta\", \"intercept\"]\n        model_fcn = _hdd_tidd_cdd\n        weight_fcn = _hdd_tidd_cdd_weight\n        TSS_fcn = _hdd_tidd_cdd_total_sum_of_squares\n\n    obj_fcn = obj_fcn_decorator(\n        model_fcn, weight_fcn, TSS_fcn, T, obs, weights, settings, alpha, coef_id, initial_fit\n    )\n\n    res = Optimizer(\n        obj_fcn, x0.to_np_array(), bnds, coef_id, settings, opt_options\n    ).run()\n\n    return res\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef _hdd_tidd_cdd(\n    hdd_bp,\n    hdd_beta,\n    cdd_bp,\n    cdd_beta,\n    intercept,\n    T_fit_bnds=np.array([]),\n    T=np.array([]),\n):\n    hdd_k = cdd_k = 0\n\n    return full_model(\n        hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept, T_fit_bnds, T\n    )\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef _hdd_tidd_cdd_smooth(*args):\n    return full_model(*args)\n\n\ndef evaluate_hdd_tidd_cdd_smooth(\n    hdd_bp,\n    hdd_beta,\n    hdd_k,\n    cdd_bp,\n    cdd_beta,\n    cdd_k,\n    intercept,\n    T_fit_bnds,\n    T,\n    pct_k=True,\n):\n    if pct_k:\n        [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_smooth_coeffs(hdd_bp, hdd_k, cdd_bp, cdd_k)\n\n    return _hdd_tidd_cdd_smooth(\n        hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept, T_fit_bnds, T\n    )\n\n\ndef _hdd_tidd_cdd_smooth_x0(T, obs, alpha, settings, smooth, min_weight=0.0):\n    min_T_idx = settings.segment_minimum_count\n    lasso_a = settings.regularization_alpha\n\n    idx_sorted = np.argsort(T).flatten()\n    T = T[idx_sorted]\n    obs = obs[idx_sorted]\n\n    N = len(obs)\n\n    T_fit_bnds = np.array([T[0], T[-1]])\n\n    def bp_obj_fcn_dec(T, obs, min_T_idx):\n        def lasso_penalty(X, wRMSE):\n            X_lasso = np.array(X).copy()\n\n            T_range = T_fit_bnds[1] - T_fit_bnds[0]\n\n            X_lasso = np.array(\n                [np.min(np.abs(X[idx] - T_fit_bnds)) for idx in range(len(X))]\n            )\n            X_lasso += (X[1] - X[0]) / 2\n            X_lasso *= wRMSE / T_range\n\n            return lasso_a * np.linalg.norm(X_lasso, 1)\n\n        def bp_obj_fcn(x, grad=[], optimize_flag=True):\n            if len(x) == 1:\n                hdd_bp = cdd_bp = x[0]\n            else:\n                if x[0] < x[1]:\n                    [hdd_bp, cdd_bp] = x\n                else:\n                    [cdd_bp, hdd_bp] = x\n\n            hdd_beta, cdd_beta, intercept = estimate_betas_and_intercept(\n                T, obs, hdd_bp, cdd_bp, min_T_idx, alpha\n            )\n            hdd_k = cdd_k = 0\n\n            model = _hdd_tidd_cdd_smooth(\n                hdd_bp,\n                hdd_beta,\n                hdd_k,\n                cdd_bp,\n                cdd_beta,\n                cdd_k,\n                intercept,\n                T_fit_bnds,\n                T,\n            )\n            resid = model - obs\n\n            if alpha == 2:\n                resid_mean = np.mean(resid)\n                resid -= resid_mean\n                intercept += resid_mean\n            else:\n                resid_median = np.median(resid)\n                resid -= resid_median\n                intercept += resid_median\n\n            weight, _, _ = adaptive_weights(\n                resid, alpha=alpha, sigma=2.698, quantile=0.25, min_weight=min_weight\n            )\n\n            loss = np.sum(weight * (resid) ** 2)\n            loss += lasso_penalty(x, np.sqrt(loss / N))\n\n            if optimize_flag:\n                return loss\n\n            return np.array(\n                [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n            )\n\n        return bp_obj_fcn\n\n    obj_fcn = bp_obj_fcn_dec(T, obs, min_T_idx)\n\n    T_bnds = [T[min_T_idx - 1], T[-min_T_idx]]\n    if T_bnds[0] == T_bnds[1]:\n        T_bnds = [\n            np.min(T),\n            np.max(T),\n        ]  # should be able to do [0] and [-1] but getting error where min > max\n\n    if T_bnds[1] < T_bnds[0]:\n        T_bnds = [T_bnds[1], T_bnds[0]]\n\n    T_min = T_bnds[0]\n    T_max = T_bnds[1]\n    T_range = T_max - T_min\n\n    x0 = np.array([T_range * 0.10, T_range * 0.90]) + T_min\n    bnds = np.array([T_bnds, T_bnds])\n\n    opt_settings = OptimizationSettings(\n        algorithm=settings.initial_guess_algorithm_choice,\n        stop_criteria_type=\"iteration maximum\",\n        stop_criteria_value=200,\n        initial_step=settings.initial_step_percentage,\n        x_tol_rel=1e-3,\n        f_tol_rel=0.5,\n    )\n\n    res = InitialGuessOptimizer(\n        obj_fcn, x0, bnds, opt_settings\n    ).run()\n\n    x0 = obj_fcn(res.x, optimize_flag=False)\n\n    if smooth:\n        model_type = ModelType.HDD_TIDD_CDD_SMOOTH\n        hdd_k = x0[2]\n        cdd_k = x0[5]\n    else:\n        model_type = ModelType.HDD_TIDD_CDD\n        hdd_k = cdd_k = None\n\n    return ModelCoefficients(\n        model_type=model_type,\n        hdd_bp=x0[0],\n        hdd_beta=x0[1],\n        hdd_k=hdd_k,\n        cdd_bp=x0[3],\n        cdd_beta=x0[4],\n        cdd_k=cdd_k,\n        intercept=x0[6],\n    )\n\n\ndef estimate_betas_and_intercept(T, obs, hdd_bp, cdd_bp, min_T_idx, alpha):\n    idx_hdd = np.argwhere(T < hdd_bp).flatten()\n    idx_tidd = np.argwhere((hdd_bp <= T) & (T <= cdd_bp)).flatten()\n    idx_cdd = np.argwhere(cdd_bp < T).flatten()\n\n    if len(idx_tidd) > 0:\n        intercept = get_intercept(obs[idx_tidd], alpha)\n    elif (\n        (len(idx_cdd) >= min_T_idx)\n        and (len(idx_hdd) >= min_T_idx)\n        and (idx_cdd[min_T_idx - 1] - idx_hdd[-min_T_idx]) > 0\n    ):\n        intercept = get_intercept(\n            obs[idx_hdd[-min_T_idx] : idx_cdd[min_T_idx - 1]], alpha\n        )\n    else:\n        intercept = np.quantile(obs, 0.20)\n\n    hdd_beta = get_slope(T[idx_hdd], obs[idx_hdd], hdd_bp, intercept, alpha)\n    if hdd_beta > 0:\n        hdd_beta = 0\n    else:\n        hdd_beta *= -1\n\n    cdd_beta = get_slope(T[idx_cdd], obs[idx_cdd], cdd_bp, intercept, alpha)\n    if cdd_beta < 0:\n        cdd_beta = 0\n\n    return hdd_beta, cdd_beta, intercept\n\n\ndef _hdd_tidd_cdd_smooth_update_bnds(new_bnds, bnds, smooth):\n    if new_bnds is None:\n        new_bnds = bnds\n\n    # breakpoint bounds\n    new_bnds[0] = bnds[0]\n    if smooth:\n        new_bnds[3] = bnds[3]\n    else:\n        new_bnds[2] = bnds[2]\n\n    # intercept bounds at index 6 for smooth, 4 for unsmooth\n    if smooth:\n        new_bnds[6] = bnds[6]\n    else:\n        new_bnds[4] = bnds[4]\n\n    new_bnds = np.sort(new_bnds, axis=1)\n    new_bnds = fix_identical_bnds(new_bnds)\n\n    # beta and k must be non-negative\n    if smooth:\n        beta_k_idx = [1, 2, 4, 5]\n    else:\n        beta_k_idx = [1, 3]\n    for i in beta_k_idx:\n        if new_bnds[i][0] < 0:\n            new_bnds[i][0] = 0\n\n    return new_bnds\n\n\ndef _hdd_tidd_cdd_weight(\n    hdd_bp,\n    hdd_beta,\n    cdd_bp,\n    cdd_beta,\n    intercept,\n    T,\n    residual,\n    sigma=3.0,\n    quantile=0.25,\n    alpha=2.0,\n    min_weight=0.0,\n):\n    hdd_k = cdd_k = 0\n    model_vars = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n    return full_model_weight(\n        *model_vars, T, residual, sigma, quantile, alpha, min_weight\n    )\n\n\ndef _hdd_tidd_cdd_smooth_weight(\n    hdd_bp,\n    hdd_beta,\n    hdd_k,\n    cdd_bp,\n    cdd_beta,\n    cdd_k,\n    intercept,\n    T,\n    residual,\n    sigma=3.0,\n    quantile=0.25,\n    alpha=2.0,\n    min_weight=0.0,\n):\n    model_vars = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n    return full_model_weight(\n        *model_vars, T, residual, sigma, quantile, alpha, min_weight\n    )\n\n\ndef _hdd_tidd_cdd_total_sum_of_squares(\n    hdd_bp, hdd_beta, cdd_bp, cdd_beta, intercept, T, obs\n):\n    if hdd_bp > cdd_bp:\n        hdd_bp, cdd_bp = cdd_bp, hdd_bp\n\n    idx_hdd_bp = np.argmin(np.abs(T - hdd_bp))\n    idx_cdd_bp = np.argmin(np.abs(T - cdd_bp))\n\n    TSS = []\n    for observed in [obs[:idx_hdd_bp], obs[idx_hdd_bp:idx_cdd_bp], obs[idx_cdd_bp:]]:\n        if len(observed) == 0:\n            continue\n\n        TSS.append(np.sum((observed - np.mean(observed)) ** 2))\n\n    TSS = np.sum(TSS)\n\n    return TSS\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/base_models/tidd.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom typing import Optional\n\nimport numba\nimport numpy as np\n\nfrom opendsm.eemeter.models.daily.base_models.full_model import (\n    full_model,\n    full_model_weight,\n)\nfrom opendsm.eemeter.models.daily.objective_function import obj_fcn_decorator\nfrom opendsm.eemeter.models.daily.optimize import Optimizer\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients, ModelType\nfrom opendsm.eemeter.models.daily.utilities.base_model import fix_identical_bnds\n\n\ndef fit_tidd(\n    T,\n    obs,\n    weights,\n    settings,\n    opt_options,\n    x0: Optional[ModelCoefficients] = None,\n    bnds=None,\n    initial_fit=False,\n):\n    if x0 is None:\n        x0 = _tidd_x0(T, obs)\n\n    if initial_fit:\n        alpha = settings.alpha_selection\n    else:\n        alpha = settings.alpha_final\n\n    intercept_bnds = np.quantile(obs, [0.01, 0.99])\n    bnds_0 = np.array([intercept_bnds])\n\n    if bnds is None:\n        bnds = bnds_0\n\n    bnds = _tidd_update_bnds(bnds, bnds_0)\n\n    coef_id = [\"intercept\"]\n    model_fcn = _tidd\n    weight_fcn = _tidd_weight\n    TSS_fcn = _tidd_total_sum_of_squares\n    obj_fcn = obj_fcn_decorator(\n        model_fcn, weight_fcn, TSS_fcn, T, obs, weights, settings, alpha, coef_id, initial_fit\n    )\n\n    res = Optimizer(\n        obj_fcn, x0.to_np_array(), bnds, coef_id, settings, opt_options\n    ).run()\n\n    return res\n\n\n# Model Functions\ndef _tidd_x0(T, obs):\n    intercept = np.median(obs)\n    return ModelCoefficients(model_type=ModelType.TIDD, intercept=intercept)\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef set_full_model_coeffs(intercept):\n    hdd_bp = hdd_beta = hdd_k = cdd_bp = cdd_beta = cdd_k = 0\n\n    return np.array([hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept])\n\n\ndef _tidd(intercept, T_fit_bnds=np.array([]), T=np.array([])):\n    model_vars = set_full_model_coeffs(intercept)\n\n    return full_model(*model_vars, T_fit_bnds, T)\n\n\ndef _tidd_total_sum_of_squares(intercept, T, obs):\n    TSS = np.sum((obs - np.mean(obs)) ** 2)\n\n    return TSS\n\n\ndef _tidd_update_bnds(new_bnds, bnds):\n    new_bnds = bnds\n\n    new_bnds = np.sort(new_bnds, axis=1)\n    new_bnds = fix_identical_bnds(new_bnds)\n\n    return new_bnds\n\n\ndef _tidd_weight(\n    intercept, T, residual, sigma=3.0, quantile=0.25, alpha=2.0, min_weight=0.0\n):\n    model_vars = set_full_model_coeffs(intercept)\n\n    return full_model_weight(\n        *model_vars, T, residual, sigma, quantile, alpha, min_weight\n    )\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport datetime\n\nimport numpy as np\nimport pandas as pd\n\nimport opendsm.common.const as _const\nfrom opendsm.eemeter.common.data_processor_utilities import (\n    as_freq,\n    clean_billing_daily_data,\n    compute_minimum_granularity,\n    remove_duplicates,\n)\nfrom opendsm.eemeter.common.features import compute_temperature_features\nfrom opendsm.eemeter.common.data_settings import DailyDataSettings\nfrom opendsm.eemeter.common.sufficiency_criteria import DailySufficiencyCriteria\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\n\nclass _DailyData:\n    \"\"\"Private base class for daily baseline and reporting data.\n\n    Will raise exception during data sufficiency check if instantiated\n\n    Args:\n        df (pd.DataFrame): The DataFrame containing the observed meter data.\n        is_electricity_data (bool): A flag indicating whether the data represents electricity data. This is required as electricity data with 0 values are converted to NaNs.\n    \"\"\"\n\n    # Abstract the settings class for easier inheritance and alteration\n    _settings_class = DailyDataSettings\n\n    def __init__(\n        self, \n        df: pd.DataFrame, \n        is_electricity_data: bool, \n        settings: dict | None = None\n    ):\n        self._df = None\n        self.is_electricity_data = is_electricity_data\n        self.tz = None\n\n        self.warnings = []\n        self.disqualification = []\n\n        # Initialize settings using the abstracted class\n        if settings is None:\n            self.settings = self._settings_class()\n        elif isinstance(settings, dict):\n            self.settings = self._settings_class(**settings)\n\n        self.settings.is_electricity_data = is_electricity_data\n            \n        # TODO re-examine dq/warning pattern. keep consistent between\n        # either implicitly setting as side effects, or returning and assigning outside\n        self._df, temp_coverage = self._set_data(df)\n\n        sufficiency_df = self._df.merge(\n            temp_coverage, left_index=True, right_index=True, how=\"left\"\n        )\n        disqualification, warnings = self._check_data_sufficiency(sufficiency_df)\n\n        self.disqualification += disqualification\n        self.warnings += warnings\n        self.log_warnings()\n\n    @property\n    def df(self) -> pd.DataFrame | None:\n        \"\"\"Get the corrected input data stored in the class. The actual dataframe is immutable, this returns a copy.\"\"\"\n\n        if self._df is None:\n            return None\n        else:\n            return self._df.copy()\n\n    @classmethod\n    def from_series(\n        cls,\n        meter_data: pd.Series | pd.DataFrame,\n        temperature_data: pd.Series | pd.DataFrame,\n        is_electricity_data: bool,\n        settings: dict | None = None,\n    ):\n        \"\"\"Create an instance of the Data class from meter data and temperature data.\n\n        Public method that can can handle two separate series (meter and temperature) and join them to create a single dataframe. The temperature column should have values in Fahrenheit.\n\n        Args:\n            meter_data: The meter data.\n            temperature_data: The temperature data.\n            is_electricity_data: A flag indicating whether the data represents electricity data. This is required as electricity data with 0 values are converted to NaNs.\n\n        Returns:\n            An instance of the Data class with the dataframe populated with the corrected data, along with warnings and disqualifications based on the input.\n        \"\"\"\n        if isinstance(meter_data, pd.Series):\n            meter_data = meter_data.to_frame()\n        if isinstance(temperature_data, pd.Series):\n            temperature_data = temperature_data.to_frame()\n        meter_data = meter_data.rename(columns={meter_data.columns[0]: \"observed\"})\n        temperature_data = temperature_data.rename(\n            columns={temperature_data.columns[0]: \"temperature\"}\n        )\n        temperature_data.index = temperature_data.index.tz_convert(\n            meter_data.index.tzinfo\n        )\n\n        if temperature_data.empty:\n            raise ValueError(\"Temperature data cannot be empty.\")\n        if meter_data.empty:\n            # reporting from_series always passes a full index of nan\n            raise ValueError(\"Meter data cannot by empty.\")\n\n        is_billing_data = False\n        if not meter_data.empty:\n            is_billing_data = compute_minimum_granularity(\n                meter_data.index, \"billing\"\n            ).startswith(\"billing\")\n\n        # first, trim the data to exclude NaNs on the outer edges of the data\n        last_meter_index = meter_data.last_valid_index()\n        if is_billing_data:\n            # preserve final NaN for billing data only\n            last = meter_data.last_valid_index()\n            if last and last != meter_data.index[-1]:\n                # TODO include warning here for non-NaN final billing row since it will be discarded\n                last_meter_index = meter_data.index[meter_data.index.get_loc(last) + 1]\n        meter_data = meter_data.loc[meter_data.first_valid_index() : last_meter_index]\n        temperature_data = temperature_data.loc[\n            temperature_data.first_valid_index() : temperature_data.last_valid_index()\n        ]\n\n        # TODO consider a refactor of the period offset calculation/slicing.\n        # it seems like a fairly dense block of code for something conceptually simple.\n        # at the very least, try to clarify variable names a bit\n\n        period_diff_first = pd.Timedelta(0)\n        period_diff_last = pd.Timedelta(0)\n        # calculate difference in period length for first and last rows in meter/temp\n        # first/last will generally be the same offset for daily/hourly, but billing can be quite variable\n        # could consider using to_offset(index.inferred_freq) if available,\n        # but the intent here is just to provide a lenient first trim.\n        # checking for consistent frequency is done later during __init__\n        if len(meter_data.index) > 1 and len(temperature_data.index) > 1:\n            period_meter_first = meter_data.index[1] - meter_data.index[0]\n            period_temp_first = temperature_data.index[1] - temperature_data.index[0]\n            period_diff_first = period_meter_first - period_temp_first\n\n            period_meter_last = meter_data.index[-1] - meter_data.index[-2]\n            period_temp_last = temperature_data.index[-1] - temperature_data.index[-2]\n            period_diff_last = period_meter_last - period_temp_last\n\n        # if diff is positive, meter period is longer (lower frequency)\n        zero_offset = pd.Timedelta(0)\n        meter_period_first_longer = period_diff_first > zero_offset\n        meter_period_last_longer = period_diff_last > zero_offset\n\n        # large period needs a buffer for the min index, and no buffer for the max index\n        # short period needs a buffer for the max index, and no buffer for the min index\n        meter_offset_first = (\n            period_diff_first if meter_period_first_longer else zero_offset\n        )\n        meter_offset_last = (\n            -period_diff_last if not meter_period_last_longer else zero_offset\n        )\n        temp_offset_first = (\n            -period_diff_first if not meter_period_first_longer else zero_offset\n        )\n        temp_offset_last = period_diff_last if meter_period_last_longer else zero_offset\n\n        # if the shorter period ends on an exact index of the longer, we accept it.\n        # the data should be DQ'd later due to insufficiency for the period\n\n        # constrain meter index to temperature index\n        temp_index_min = temperature_data.index.min() - meter_offset_first\n        temp_index_max = temperature_data.index.max() + meter_offset_last\n        meter_data = meter_data[temp_index_min:temp_index_max]\n        if meter_data.empty:\n            raise ValueError(\"Meter and temperature data are fully misaligned.\")\n\n        # if billing detected, subtract one day from final index since dataframe input assumes final row is part of period\n        if is_billing_data:\n            new_index = meter_data.index[:-1].union(\n                [(meter_data.index[-1] - pd.Timedelta(days=1))]\n            )\n            if len(new_index) == len(meter_data.index):\n                meter_data.index = new_index\n            else:\n                # handles the case of a 1 day off-cycle read at end of series\n                meter_data = meter_data[:-1]\n\n        # constrain temperature index to meter index\n        meter_index_min = meter_data.index.min() - temp_offset_first\n        meter_index_max = meter_data.index.max() + temp_offset_last\n        if is_billing_data and len(meter_data) > 1:\n            # last billing period is offset by one index\n            meter_index_max = meter_data.index[-2] + temp_offset_last\n        temperature_data = temperature_data[meter_index_min:meter_index_max]\n\n        if is_billing_data:\n            # TODO consider adding misaligned data warning here if final row was not already NaN\n            meter_data.iloc[-1] = np.nan\n\n        df = pd.concat([meter_data, temperature_data], axis=1)\n\n        return cls(df, is_electricity_data, settings=settings)\n\n    def log_warnings(self) -> None:\n        \"\"\"Logs the warnings and disqualifications associated with the data.\n\n        View the disqualifications and warnings associated with the current data input provided.\n\n        Returns:\n            None\n        \"\"\"\n        for warning in self.warnings + self.disqualification:\n            warning.warn()\n\n    def _compute_meter_value_df(self, df: pd.DataFrame):\n        \"\"\"\n        Computes the meter value DataFrame by cleaning and processing the observed meter data.\n        1. The minimum granularity is computed from the non null rows.\n        2. The meter data is cleaned and downsampled/upsampled into the correct frequency using clean_billing_daily_data()\n        3. Add missing days as NaN by merging with a full year daily index.\n\n        Parameters\n        ----------\n\n            df (pd.DataFrame): The DataFrame containing the observed meter data.\n\n        Returns\n        -------\n            pd.DataFrame: The cleaned and processed meter value DataFrame.\n        \"\"\"\n        meter_series = df[\"observed\"].dropna()\n        if meter_series.empty:\n            return df[\"observed\"].resample(\"D\").first().to_frame()\n\n        # Dropping the NaNs is beneficial when the meter data is spread over hourly temperature data, causing lots of NaNs\n        # But causes problems in detection of frequency when there are genuine missing values. The missing days are accounted for in the sufficiency_criteria_baseline method\n        # whereas they should actually be kept.\n        start_date = df.index.min()\n        end_date = df.index.max()\n        min_granularity = compute_minimum_granularity(meter_series.index, \"daily\")\n        if min_granularity.startswith(\"billing\"):\n            # TODO : make this a warning instead of an exception\n            raise ValueError(\"Billing data is not allowed in the daily model\")\n        meter_value_df = clean_billing_daily_data(\n            meter_series, min_granularity, self.warnings\n        )\n\n        meter_value_df = meter_value_df.rename(columns={\"value\": \"observed\"})\n\n        # To account for the above issue, we create an index with all the days and then merge the meter_value_df with it\n        # This will ensure that the missing days are kept in the dataframe\n        # Create an index with all the days from the start and end date of 'meter_value_df'\n        all_days_index = pd.date_range(\n            start=start_date,\n            end=end_date,\n            freq=\"D\",\n            tz=df.index.tz,\n            ambiguous=True,\n            nonexistent=\"shift_forward\",\n        )\n        all_days_df = pd.DataFrame(index=all_days_index)\n        # the following drops common days to handle DST issues with pytz.\n        # doesn't seem to be a problem with ZoneInfo, so we can\n        # probably handle this better once 3.8 is EOL and we disallow pytz tzinfo.\n        # TODO regardless, it feels like there should be a better way to match\n        # the indices on date than by comparing strftime in this manner\n        all_days_df = all_days_df[\n            ~all_days_df.index.strftime(\"%Y%m%d\").isin(\n                meter_series.index.strftime(\"%Y%m%d\")\n            )\n        ]\n        meter_value_df = meter_value_df.merge(\n            all_days_df, left_index=True, right_index=True, how=\"outer\"\n        )\n\n        return meter_value_df\n\n    def _compute_temperature_features(\n        self, df: pd.DataFrame, meter_index: pd.DatetimeIndex\n    ):\n        \"\"\"\n        Compute temperature features for the given DataFrame and meter index.\n        1. The frequency of the temperature data is inferred and set to hourly if not already. If frequency is not inferred or its lower than hourly, a warning is added.\n        2. The temperature data is downsampled/upsampled into the daily frequency using as_freq()\n        3. High frequency temperature data is checked for missing values and a warning is added if more than 50% of the data is missing, and those rows are set to NaN.\n        4. If frequency was already hourly, compute_temperature_features() is used to recompute the temperature to match with the meter index.\n\n        Parameters\n        ----------\n\n            df (pd.DataFrame): The DataFrame containing temperature data.\n            meter_index (pd.DatetimeIndex): The meter index.\n\n        Returns\n        -------\n\n            pd.Series: The computed temperature values.\n            pd.DataFrame: The computed temperature features.\n        \"\"\"\n        temp_series = df[\"temperature\"]\n        temp_series.index.freq = temp_series.index.inferred_freq\n        if temp_series.index.freq != \"h\":\n            if temp_series.index.freq is None or temp_series.index.freq > pd.Timedelta(\n                hours=1\n            ):\n                # Add warning for frequencies longer than 1 hour\n                self.warnings.append(\n                    EEMeterWarning(\n                        qualified_name=\"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\",\n                        description=(\n                            \"Cannot confirm that pre-aggregated temperature data had sufficient hours kept\"\n                        ),\n                        data={},\n                    )\n                )\n            if temp_series.index.freq != \"D\":\n                # Downsample / Upsample the temperature data to daily\n                temperature_features = as_freq(\n                    temp_series, \"D\", series_type=\"instantaneous\", include_coverage=True\n                )\n                # If high frequency data check for 50% data coverage in rollup\n                if len(temperature_features[temperature_features.coverage <= 0.5]) > 0:\n                    self.warnings.append(\n                        EEMeterWarning(\n                            qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\",\n                            description=(\n                                \"More than 50% of the high frequency Temperature data is missing.\"\n                            ),\n                            data={\n                                \"high_frequency_data_missing_count\": len(\n                                    temperature_features[\n                                        temperature_features.coverage <= 0.5\n                                    ].index.to_list()\n                                )\n                            },\n                        )\n                    )\n\n                # Set missing high frequency data to NaN\n                temperature_features.loc[\n                    temperature_features.coverage > 0.5, \"value\"\n                ] = (\n                    temperature_features[temperature_features.coverage > 0.5].value\n                    / temperature_features[temperature_features.coverage > 0.5].coverage\n                )\n\n                temperature_features = (\n                    temperature_features[temperature_features.coverage > 0.5]\n                    .reindex(temperature_features.index)[[\"value\"]]\n                    .rename(columns={\"value\": \"temperature_mean\"})\n                )\n\n                if \"coverage\" in temperature_features.columns:\n                    temperature_features = temperature_features.drop(\n                        columns=[\"coverage\"]\n                    )\n            else:\n                temperature_features = temp_series.to_frame(name=\"temperature_mean\")\n\n            temperature_features[\"temperature_null\"] = temp_series.isnull().astype(int)\n            temperature_features[\"temperature_not_null\"] = temp_series.notnull().astype(\n                int\n            )\n            temperature_features[\"n_days_kept\"] = 0  # unused\n            temperature_features[\"n_days_dropped\"] = 0  # unused\n        else:\n            # TODO hacky method of avoiding the last index nan convention\n            if not meter_index.empty:\n                buffer_idx = meter_index.max() + pd.Timedelta(days=1)\n                meter_index = meter_index.union([buffer_idx])\n            temperature_features = compute_temperature_features(\n                meter_index,\n                temp_series,\n                data_quality=True,\n            )\n            temperature_features = temperature_features[:-1]\n\n            # Only check for high frequency temperature data if it exists\n            if (\n                temperature_features.temperature_not_null\n                + temperature_features.temperature_null\n            ).median() > 1:\n                invalid_temperature_rows = (\n                    temperature_features.temperature_not_null\n                    / (\n                        temperature_features.temperature_not_null\n                        + temperature_features.temperature_null\n                    )\n                ) <= 0.5\n\n                # Set high frequency temperature data with more than 50% data missing as NaN\n                if invalid_temperature_rows.any():\n                    self.warnings.append(\n                        EEMeterWarning(\n                            qualified_name=\"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\",\n                            description=(\n                                \"More than 50% of the high frequency temperature data is missing.\"\n                            ),\n                            data=[\n                                timestamp.isoformat()\n                                for timestamp in invalid_temperature_rows[invalid_temperature_rows].index\n                            ],\n                        )\n                    )\n                    temperature_features.loc[\n                        invalid_temperature_rows, \"temperature_mean\"\n                    ] = np.nan\n\n        temp = temperature_features[\"temperature_mean\"].rename(\"temperature\")\n        features = temperature_features.drop(columns=[\"temperature_mean\"])\n        return temp, features\n\n    def _merge_meter_temp(self, meter, temp):\n        \"\"\"\n        Merge the meter and temperature dataframes and reorder the columns to have the order -\n            [season, weekday_weekend, temperature, observed (if present)]\n\n        Parameters\n        ----------\n            meter (pd.DataFrame): The meter dataframe.\n            temp (pd.DataFrame): The temperature dataframe.\n\n        Returns\n        -------\n            pd.DataFrame: The merged and transformed dataframe.\n        \"\"\"\n        df = meter.merge(\n            temp, left_index=True, right_index=True, how=\"left\"\n        ).tz_convert(meter.index.tz)\n        if df[\"observed\"].dropna().empty:\n            df = df.drop(columns=[\"observed\"])\n\n        # Add Season and Weekday_weekend\n        df[\"season\"] = df.index.month_name().map(_const.default_season_def)\n        df[\"weekday_weekend\"] = df.index.day_name().map(\n            _const.default_weekday_weekend_def\n        )\n\n        # Reorder the columns Create a list of columns\n        columns = [\"season\", \"weekday_weekend\", \"temperature\"]\n        if \"observed\" in df.columns:\n            columns.append(\"observed\")\n        df = df[columns]\n\n        return df\n\n    def _check_data_sufficiency(self, sufficiency_df):\n        raise NotImplementedError(\n            \"Can't instantiate class _DailyData, use DailyBaselineData or DailyReportingData.\"\n        )\n\n    def _set_data(self, data: pd.DataFrame):\n        \"\"\"Process data input for the Daily Model Baseline Class\n        Datetime has to be either index or a separate column in the dataframe.\n        Electricity data with 0 meter values are converted to NaNs.\n\n        Parameters\n        ----------\n        data : pd.DataFrame\n            Required columns - datetime, observed, temperature\n\n            observed\n\n        Returns\n        -------\n        processed_data : pd.DataFrame\n            Dataframe appended with the correct season and day of week.\n        \"\"\"\n\n        # Copy the input dataframe so that the original is not modified\n        df = data.copy()\n\n        expected_columns = [\n            \"observed\",\n            \"temperature\",\n        ]\n        # TODO maybe check datatypes\n        if not set(expected_columns).issubset(set(df.columns)):\n            # show the columns that are missing\n\n            raise ValueError(\n                \"Data is missing required columns: {}\".format(\n                    set(expected_columns) - set(df.columns)\n                )\n            )\n\n        # Check that the datetime index is timezone aware timestamp\n        if not isinstance(df.index, pd.DatetimeIndex) and \"datetime\" not in df.columns:\n            raise ValueError(\"Index is not datetime and datetime not provided\")\n\n        elif \"datetime\" in df.columns:\n            if df[\"datetime\"].dt.tz is None:\n                raise ValueError(\"Datatime is missing timezone information\")\n            df[\"datetime\"] = pd.to_datetime(df[\"datetime\"])\n            df.set_index(\"datetime\", inplace=True)\n\n        elif df.index.tz is None:\n            raise ValueError(\"Datatime is missing timezone information\")\n        elif str(df.index.tz) == \"UTC\":\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.data_quality.utc_index\",\n                    description=(\n                        \"Datetime index is in UTC. Use tz_localize() with the local timezone to ensure correct aggregations\"\n                    ),\n                    data={},\n                )\n            )\n        self.tz = df.index.tz\n        self.settings.time_zone = self.tz\n\n        # prevent later issues when merging on generated datetimes, which default to ns precision\n        # there is almost certainly a smoother way to accomplish this conversion, but this works\n        if df.index.dtype.unit != \"ns\":\n            utc_index = df.index.tz_convert(\"UTC\")\n            ns_index = utc_index.astype(\"datetime64[ns, UTC]\")\n            df.index = ns_index.tz_convert(self.tz)\n\n        # Convert electricity data having 0 meter values to NaNs\n        if self.is_electricity_data:\n            df.loc[df[\"observed\"] == 0, \"observed\"] = np.nan\n\n        # Caltrack 2.3.2 - Drop duplicates\n        df = remove_duplicates(df)\n\n        meter = self._compute_meter_value_df(df)\n        temp, temp_coverage = self._compute_temperature_features(df, meter.index)\n        final_df = self._merge_meter_temp(meter, temp)\n        return final_df, temp_coverage\n\n\nclass DailyBaselineData(_DailyData):\n    \"\"\"Data class to represent Daily Baseline Data.\n\n    Only baseline data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of issues with the data, but none that will severely reduce the quality of the model built.\n\n    \"\"\"\n\n    def _check_data_sufficiency(self, sufficiency_df):\n        \"\"\"\n        Private method which checks the sufficiency of the data for daily baseline calculations using the predefined OpenEEMeter sufficiency criteria.\n\n        Args:\n            sufficiency_df (pandas.DataFrame): DataFrame containing the data for sufficiency check. Should have features such as -\n            temperature_null: number of temperature null periods in each aggregation step\n            temperature_not_null: number of temperature non null periods in each aggregation step\n\n        Returns:\n            disqualification (List): List of disqualifications\n            warnings (list): List of warnings\n\n        \"\"\"\n        # 90% coverage per period only required for billing models\n        dsc = DailySufficiencyCriteria(\n            data=sufficiency_df, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=False,\n            settings=self.settings.sufficiency,\n        )\n        dsc.check_sufficiency_baseline()\n        disqualification = dsc.disqualification\n        warnings = dsc.warnings\n\n        return disqualification, warnings\n\n\nclass DailyReportingData(_DailyData):\n    \"\"\"Data class to represent Daily Reporting Data.\n\n    Only reporting data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Meter data input is optional for the reporting class.\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of issues with the data, but none that will severely reduce the quality of the model built.\n    \"\"\"\n\n    def __init__(\n        self,\n        df: pd.DataFrame, \n        is_electricity_data: bool, \n        settings: dict | None = None\n    ):\n        df = df.copy()\n        if \"observed\" not in df.columns:\n            df[\"observed\"] = np.nan\n\n        super().__init__(df, is_electricity_data, settings=settings)\n\n    @classmethod\n    def from_series(\n        cls,\n        meter_data: pd.Series | pd.DataFrame | None,\n        temperature_data: pd.Series | pd.DataFrame,\n        is_electricity_data: bool | None = None,\n        tzinfo: datetime.tzinfo | None = None,\n        settings: dict | None = None,\n    ) -> DailyReportingData:\n        \"\"\"Create an instance of the Data class from meter data and temperature data.\n\n        Args:\n            meter_data: The meter data to be used for the DailyReportingData instance.\n            temperature_data: The temperature data to be used for the DailyReportingData instance.\n            is_electricity_data: Flag indicating whether the meter data represents electricity data.\n            tzinfo: Timezone information to be used for the meter data.\n\n        Returns:\n            An instance of the Data class.\n        \"\"\"\n        if tzinfo and meter_data is not None:\n            raise ValueError(\n                \"When passing meter data to DailyReportingData, convert its DatetimeIndex to local timezone first; `tzinfo` param should only be used in the absence of reporting meter data.\"\n            )\n        if is_electricity_data is None and meter_data is not None:\n            raise ValueError(\n                \"Must specify is_electricity_data when passing meter data.\"\n            )\n        if meter_data is None:\n            meter_data = pd.DataFrame(\n                {\"observed\": np.nan}, index=temperature_data.index\n            )\n            if tzinfo:\n                meter_data = meter_data.tz_convert(tzinfo)\n\n            # If is_electricity_data is not specified, set it to True for proper functioning in the parent class. If it hits this point it's all NaNs anyway.\n            if is_electricity_data is None:\n                is_electricity_data = True\n        if meter_data.empty:\n            raise ValueError(\n                \"Pass meter_data=None rather than an empty series in order to explicitly create a temperature-only reporting data instance.\"\n            )\n        return super().from_series(meter_data, temperature_data, is_electricity_data, settings=settings)\n\n    def _check_data_sufficiency(self, sufficiency_df):\n        \"\"\"\n        Private method which checks the sufficiency of the data for daily reporting calculations using the predefined OpenEEMeter sufficiency criteria.\n\n        Parameters\n        ----------\n        1. sufficiency_df (pandas.DataFrame): DataFrame containing the data for sufficiency check. Should have features such as -\n            - temperature_null: number of temperature null periods in each aggregation step\n            - temperature_not_null: number of temperature non null periods in each aggregation step\n\n        Returns\n        -------\n            disqualification (List): List of disqualifications\n            warnings (list): List of warnings\n\n        \"\"\"\n        # 90% coverage per period only required for billing models\n        dsc = DailySufficiencyCriteria(\n            data=sufficiency_df, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=True,\n            settings=self.settings.sufficiency,\n        )\n        dsc.check_sufficiency_reporting()\n        disqualification = dsc.disqualification\n        warnings = dsc.warnings\n\n        return disqualification, warnings\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/fit_base_models.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\n\nfrom opendsm.common.utils import OoM\nfrom opendsm.eemeter.models.daily.base_models.c_hdd_tidd import fit_c_hdd_tidd\nfrom opendsm.eemeter.models.daily.base_models.hdd_tidd_cdd import fit_hdd_tidd_cdd\nfrom opendsm.eemeter.models.daily.base_models.tidd import fit_tidd\nfrom opendsm.eemeter.models.daily.optimize_results import OptimizedResult\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients\nfrom opendsm.eemeter.models.daily.utilities.settings import FullModelSelection\nfrom opendsm.eemeter.models.daily.utilities.opt_settings import OptimizationSettings\n\n\ndef _get_opt_settings(settings):\n    \"\"\"\n    Returns a dictionary containing optimization options for the global and local optimization algorithms.\n\n    Parameters:\n        settings: A DailySettings object containing the settings for the optimization algorithm.\n\n    Returns:\n        A dictionary containing the optimization options for the optimization algorithm.\n    \"\"\"\n\n    opt_dict = {\n        \"ALGORITHM\": settings.algorithm_choice,\n        \"INITIAL_STEP\": settings.initial_step_percentage,\n    }\n    opt_settings = OptimizationSettings(**opt_dict)\n\n    return opt_settings\n\n\ndef fit_initial_models_from_full_model(df_meter, settings, print_res=False):\n    \"\"\"\n    Fits initial models from the full model based on the given settings.\n\n    Parameters:\n        df_meter (pandas.DataFrame): The meter data to fit the models to. Columns : date, observed, temperature\n        settings (Settings): The settings object containing the model selection and fitting options.\n        print_res (bool, optional): Whether to print the results of the model fitting. Defaults to False.\n\n    Returns:\n        ModelResult: The result of the model fitting.\n    \"\"\"\n\n    T = df_meter[\"temperature\"].values\n    obs = df_meter[\"observed\"].values\n\n    if \"weights\" in df_meter.columns:\n        weights = df_meter[\"weights\"].values\n    else:\n        weights = None\n\n    opt_settings = _get_opt_settings(settings)\n    fit_input = [T, obs, weights, settings, opt_settings]\n\n    # initial fitting of the most complicated model allowed\n    if settings.full_model == FullModelSelection.HDD_TIDD_CDD:\n        model_res = fit_hdd_tidd_cdd(\n            *fit_input, smooth=settings.allow_smooth_model, initial_fit=True\n        )\n    elif settings.full_model == FullModelSelection.C_HDD_TIDD:\n        model_res = fit_c_hdd_tidd(\n            *fit_input, smooth=settings.allow_smooth_model, initial_fit=True\n        )\n    elif settings.full_model == FullModelSelection.TIDD:\n        model_res = fit_tidd(*fit_input, initial_fit=True)\n\n    if print_res:\n        criterion = model_res.selection_criterion\n        # print(f\"{model_key:<30s} {model_res.loss:<8.3f} {model_res.alpha:<8.2f} {model_res.C:<8.3f} {model_res.time_elapsed:>8.2f} ms\")\n        print(\n            f\"{model_res.model_name:<30s} {criterion:<8.3g} {model_res.alpha:<8.2f} {model_res.time_elapsed:>8.2f} ms\"\n        )\n\n    return model_res\n\n\ndef fit_model(model_key, fit_input, x0: ModelCoefficients, bnds):\n    \"\"\"\n    Fits a model based on the given model key and input data.\n\n    Args:\n        model_key (str): The key for the model to be fitted.\n        fit_input (tuple): The input data for the model.\n        x0 (ModelCoefficients): The initial coefficients for the model.\n        bnds (tuple): The bounds for the model coefficients.\n\n    Returns:\n        The result of the model fitting.\n    \"\"\"\n\n    if model_key == \"hdd_tidd_cdd_smooth\":\n        res = fit_hdd_tidd_cdd(\n            *fit_input, smooth=True, x0=x0, bnds=bnds, initial_fit=False\n        )\n\n    elif model_key == \"hdd_tidd_cdd\":\n        res = fit_hdd_tidd_cdd(\n            *fit_input, smooth=False, x0=x0, bnds=bnds, initial_fit=False\n        )\n\n    elif model_key == \"c_hdd_tidd_smooth\":\n        res = fit_c_hdd_tidd(\n            *fit_input, smooth=True, x0=x0, bnds=bnds, initial_fit=False\n        )\n\n    elif model_key == \"c_hdd_tidd\":\n        res = fit_c_hdd_tidd(\n            *fit_input, smooth=False, x0=x0, bnds=bnds, initial_fit=False\n        )\n\n    elif model_key == \"tidd\":\n        res = fit_tidd(*fit_input, x0, bnds, initial_fit=False)\n\n    return res\n\n\ndef fit_final_model(df_meter, HoF: OptimizedResult, settings, print_res=False):\n    \"\"\"\n    Fits the final model using the optimized result and returns the optimized result with updated coefficients.\n    HoF (Hall of Fame) denotes the optimized results.\n\n    Args:\n        df_meter (pandas.DataFrame): DataFrame containing temperature and observed values.\n        HoF (OptimizedResult): OptimizedResult object containing the optimized model and coefficients.\n        settings (Settings): DailySettings object containing the settings for the model fitting.\n        print_res (bool, optional): Whether to print the results. Defaults to False.\n\n    Returns:\n        OptimizedResult: OptimizedResult object with updated coefficients.\n    \"\"\"\n\n    def get_bnds(x0, bnds_scalar):\n        x_oom = 10 ** (OoM(x0, method=\"exact\") + np.log10(bnds_scalar))\n        bnds = (x0 + (np.array([-1, 1]) * x_oom[:, None]).T).T\n\n        return bnds\n\n    T = df_meter[\"temperature\"].values\n    obs = df_meter[\"observed\"].values\n\n    if \"weights\" in df_meter.columns:\n        weights = df_meter[\"weights\"].values\n    else:\n        weights = None\n\n    opt_settings = _get_opt_settings(settings)\n    fit_input = [T, obs, weights, settings, opt_settings]\n\n    x0 = HoF.x\n    bnds = get_bnds(x0, settings.final_bounds_scalar)\n\n    HoF = fit_model(HoF.model_key, fit_input, HoF.named_coeffs, bnds)\n\n    if print_res:\n        print(\n            f\"{HoF.model_name:<30s} {HoF.loss_alpha:<8.2f} {HoF.time_elapsed:>8.2f} ms\"\n        )\n\n    return HoF\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport itertools\nimport json\nfrom typing import Union\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\nfrom opendsm.eemeter.models.daily.base_models.full_model import (\n    full_model,\n    get_full_model_x,\n)\nfrom opendsm.eemeter.models.daily.data import DailyBaselineData, DailyReportingData\nfrom opendsm.eemeter.models.daily.fit_base_models import (\n    fit_final_model,\n    fit_initial_models_from_full_model,\n)\nfrom opendsm.eemeter.models.daily.parameters import (\n    DailyModelParameters,\n    DailySubmodelParameters,\n)\nfrom opendsm.eemeter.models.daily.utilities.base_model import get_smooth_coeffs\nfrom opendsm.eemeter.models.daily.utilities.settings import (\n    DailySettings,\n    DailyLegacySettings,\n    update_daily_settings,\n)\nfrom opendsm.eemeter.models.daily.utilities.ellipsoid_test import ellipsoid_split_filter\nfrom opendsm.eemeter.models.daily.utilities.selection_criteria import selection_criteria\nfrom opendsm.common.metrics import BaselineMetrics, BaselineMetricsFromDict\n\nclass DailyModel:\n    \"\"\"\n    A class to fit a model to the input meter data.\n\n    Attributes:\n        settings (dict): A dictionary of settings.\n        seasonal_options (list): A list of seasonal options (su: Summer, sh: Shoulder, wi: Winter).\n            Elements in the list are seasons separated by '_' that represent a model split.\n            For example, a list of ['su_sh', 'wi'] represents two splits: summer/shoulder and winter.\n        day_options (list): A list of day options.\n        combo_dictionary (dict): A dictionary of combinations.\n        df_meter (pandas.DataFrame): A dataframe of meter data.\n        error (dict): A dictionary of error metrics.\n        combinations (list): A list of combinations.\n        components (list): A list of components.\n        fit_components (list): A list of fit components.\n        wRMSE_base (float): The mean bias error for no splits.\n        best_combination (list): The best combination of splits.\n        model (sklearn.pipeline.Pipeline): The final fitted model.\n        id (str): The index of the meter data.\n    \"\"\"\n\n    _baseline_data_type = DailyBaselineData\n    _reporting_data_type = DailyReportingData\n    _data_df_name = \"df\"\n\n    def __init__(\n        self,\n        model: str = \"current\",\n        settings: dict | None = None,\n        verbose: bool = False,\n    ):\n        \"\"\"\n        Args:\n            model: The model to use (either 'current' or 'legacy').\n            settings: DailySettings to be changed.\n            verbose: Whether to print verbose output.\n        \"\"\"\n\n        # Initialize settings\n        self._initialize_settings(model, settings)\n\n        # Initialize seasons and weekday/weekend\n        self.seasonal_options = [\n            [\"su_sh_wi\"],\n            [\"su\", \"sh_wi\"],\n            [\"su_sh\", \"wi\"],\n            [\"su_wi\", \"sh\"],\n            [\"su\", \"sh\", \"wi\"],\n        ]\n        self.day_options = [[\"wd\", \"we\"]]\n\n        # make dictionary is_weekday from settings\n        day_dict = self.settings.weekday_weekend._num_dict\n        n_week = list(range(len(day_dict)))\n        self.combo_dictionary = {\n            \"su\": \"summer\",\n            \"sh\": \"shoulder\",\n            \"wi\": \"winter\",\n            \"fw\": [n + 1 for n in n_week],\n            \"wd\": [n + 1 for n in n_week if day_dict[n+1] == \"weekday\"],\n            \"we\": [n + 1 for n in n_week if day_dict[n+1] == \"weekend\"],\n        }\n        self.verbose = verbose\n\n    def _initialize_settings(\n        self,\n        model: str = \"current\",\n        settings: dict | None = None\n    ) -> None:\n\n        # Note: Model designates the base settings, it can be 'current' or 'legacy'\n        #       Settings is to be a dictionary of settings to be changed\n\n        if settings is None:\n            settings = {}\n\n        if model.replace(\" \", \"\").replace(\"_\", \".\").lower() in [\"current\", \"default\"]:\n            self.settings = DailySettings(**settings)\n        elif model.replace(\" \", \"\").replace(\"_\", \".\").lower() in [\"legacy\"]:\n            self.settings = DailyLegacySettings(**settings)\n        else:\n            raise Exception(\n                \"Invalid 'settings' choice: must be 'current', 'default', or 'legacy'\"\n            )\n\n    def fit(\n        self, \n        baseline_data: DailyBaselineData, \n        ignore_disqualification: bool = False\n    ) -> DailyModel:\n        \"\"\"Fit the model using baseline data.\n\n        Args:\n            baseline_data: DailyBaselineData object.\n            ignore_disqualification: Whether to ignore disqualification errors / warnings.\n\n        Returns:\n            The fitted model.\n\n        Raises:\n            TypeError: If baseline_data is not a DailyBaselineData object.\n            DataSufficiencyError: If the model can't be fit on disqualified baseline data.\n        \"\"\"\n        if not isinstance(baseline_data, self._baseline_data_type):\n            raise TypeError(f\"baseline_data must be a {self._baseline_data_type.__name__} object\")\n        baseline_data.log_warnings()\n        if baseline_data.disqualification and not ignore_disqualification:\n            raise DataSufficiencyError(\"Can't fit model on disqualified baseline data\")\n        self.baseline_timezone = baseline_data.tz\n        self.warnings = baseline_data.warnings\n        self.disqualification = baseline_data.disqualification\n        df = getattr(baseline_data, self._data_df_name)\n        self._fit(df)\n        self._check_model_fit()\n\n        return self\n\n    def _fit(self, meter_data):\n        # Initialize dataframe\n        self.df_meter, _ = self._initialize_data(meter_data)\n\n        # Begin fitting\n        self.combinations = self._combinations()\n        self.components = self._components()\n        self.fit_components = self._fit_components()\n\n        # calculate mean bias error for no splits\n        self.wRMSE_base = self._get_error_metrics(\"fw-su_sh_wi\").wrmse\n\n        # find best combination\n        self.best_combination = self._best_combination(print_out=False)\n        self.model = self._final_fit(self.best_combination)\n\n        self.id = meter_data.index.unique()[0]\n\n        self.baseline_metrics = self._get_error_metrics(self.best_combination)\n\n        self.params = self._create_params_from_fit_model()\n        self.is_fitted = True\n        return self\n\n    def predict(\n        self,\n        reporting_data: DailyBaselineData | DailyReportingData,\n        ignore_disqualification=False,\n    ) -> pd.DataFrame:\n        \"\"\"Predicts the energy consumption using the fitted model.\n\n        Args:\n            reporting_data (Union[DailyBaselineData, DailyReportingData]): The data used for prediction.\n            ignore_disqualification (bool, optional): Whether to ignore model disqualification. Defaults to False.\n\n        Returns:\n            Dataframe with input data along with predicted energy consumption.\n\n        Raises:\n            RuntimeError: If the model is not fitted.\n            DisqualifiedModelError: If the model is disqualified and ignore_disqualification is False.\n            ValueError: If the reporting data has a different timezone than the model.\n            TypeError: If the reporting data is not of type DailyBaselineData or DailyReportingData.\n        \"\"\"\n        if not self.is_fitted:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n\n        if self.disqualification and not ignore_disqualification:\n            raise DisqualifiedModelError(\n                \"Attempting to predict using disqualified model without setting ignore_disqualification=True\"\n            )\n\n        if str(self.baseline_timezone) != str(reporting_data.tz):\n            \"\"\"would be preferable to directly compare, but\n            * using str() helps accomodate mixed tzinfo implementations,\n            * the likelihood of sub-hour offset inconsistencies being relevant to the daily model is low\n            \"\"\"\n            raise ValueError(\n                \"Reporting data must use the same timezone that the model was initially fit on.\"\n            )\n\n        if not isinstance(reporting_data, (self._baseline_data_type, self._reporting_data_type)):\n            raise TypeError(\n                f\"reporting_data must be a {self._baseline_data_type.__name__} or {self._reporting_data_type.__name__} object\"\n            )\n\n        df = getattr(reporting_data, self._data_df_name)\n        df_res = self._predict(df)\n\n        return df_res\n\n    def _predict(self, df_eval, mask_observed_with_missing_temperature=True):\n        \"\"\"\n        Makes model prediction on given temperature data.\n\n        Parameters:\n            df_eval (pandas.DataFrame): The evaluation dataframe.\n\n        Returns:\n            pandas.DataFrame: The evaluation dataframe with model predictions added.\n        \"\"\"\n        # TODO decide whether to allow temperature series vs requiring \"design matrix\"\n        if isinstance(df_eval, pd.Series):\n            df_eval = df_eval.to_frame(\"temperature\")\n\n        # initialize data to input dataframe\n        df_eval, dropped_rows = self._initialize_data(df_eval)\n\n        df_all_models = []\n        for component_key in self.params.submodels.keys():\n            eval_segment = self._meter_segment(component_key, df_eval)\n            T = eval_segment[\"temperature\"].values\n\n            # model, unc, hdd_load, cdd_load = self.model[component_key].eval(T)\n            model, unc, hdd_load, cdd_load = self._predict_submodel(\n                self.params.submodels[component_key], T\n            )\n\n            df_model = pd.DataFrame(\n                data={\n                    \"predicted\": model,\n                    \"predicted_unc\": unc,\n                    \"heating_load\": hdd_load,\n                    \"cooling_load\": cdd_load,\n                },\n                index=eval_segment.index,\n            )\n            df_model[\"model_split\"] = component_key\n            df_model[\"model_type\"] = self.params.submodels[\n                component_key\n            ].model_type.value\n\n            df_all_models.append(df_model)\n\n        df_model_prediction = pd.concat(df_all_models, axis=0)\n        df_eval = df_eval.join(df_model_prediction)\n\n        # 3.5.1.1. If a day is missing a temperature value, the corresponding consumption value for that day should be masked.\n        if mask_observed_with_missing_temperature:\n            dropped_rows[dropped_rows[\"temperature\"].isna()][\"observed\"] = np.nan\n\n        df_eval = pd.concat([df_eval, dropped_rows])\n\n        return df_eval.sort_index()\n\n    def _check_model_fit(self):\n        cvrmse = self.baseline_metrics.cvrmse_adj\n        pnrmse = self.baseline_metrics.pnrmse_adj\n\n        cvrmse_threshold = self.settings.cvrmse_threshold\n        pnrmse_threshold = self.settings.pnrmse_threshold\n\n        def _model_fit_is_acceptable(cvrmse, pnrmse):\n            # sufficient is (0 <= cvrmse <= threshold) or (0 <= pnrmse <= threshold)\n            if cvrmse is not None:\n                if (0 <= cvrmse) and (cvrmse <= cvrmse_threshold):\n                    return True\n                \n            if pnrmse is not None:\n                # less than 0 is not possible, but just in case\n                if (0 <= pnrmse) and (pnrmse <= pnrmse_threshold):\n                    return True\n\n            return False\n\n        if not _model_fit_is_acceptable(cvrmse, pnrmse):\n            model_fit_warning = EEMeterWarning(\n                qualified_name=\"eemeter.model_fit_metrics\",\n                description=\"Model disqualified due to poor fit.\",\n                data={\n                    \"cvrmse_threshold\": cvrmse_threshold,\n                    \"cvrmse\": cvrmse,\n                    \"pnrmse_threshold\": pnrmse_threshold,\n                    \"pnrmse\": pnrmse,\n                },\n            )\n            model_fit_warning.warn()\n            self.disqualification.append(model_fit_warning)\n            \n    def to_dict(self) -> dict:\n        \"\"\"Returns a dictionary of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        return self.params.model_dump()\n\n    def to_json(self) -> str:\n        \"\"\"Returns a JSON string of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        return json.dumps(self.to_dict())\n\n    @classmethod\n    def from_dict(cls, data) -> DailyModel:\n        \"\"\"Create a instance of the class from a dictionary (such as one produced from the to_dict method).\n\n        Args:\n            data (dict): The dictionary containing the model data.\n\n        Returns:\n            An instance of the class.\n\n        \"\"\"\n        settings = data.get(\"settings\")\n        daily_model = cls(settings=settings)\n        info = data.get(\"info\")\n        daily_model.params = DailyModelParameters(\n            submodels=data.get(\"submodels\"),\n            info=info,\n            settings=settings,\n        )\n\n        def deserialize_warnings(warnings):\n            if not warnings:\n                return []\n            warn_list = []\n            for warning in warnings:\n                warn_list.append(\n                    EEMeterWarning(\n                        qualified_name=warning.get(\"qualified_name\"),\n                        description=warning.get(\"description\"),\n                        data=warning.get(\"data\"),\n                    )\n                )\n            return warn_list\n\n        daily_model.disqualification = deserialize_warnings(\n            info.get(\"disqualification\")\n        )\n        daily_model.warnings = deserialize_warnings(info.get(\"warnings\"))\n        daily_model.baseline_timezone = info.get(\"baseline_timezone\")\n        if info.get(\"metrics\") is not None:\n            daily_model.baseline_metrics = BaselineMetricsFromDict(info.get(\"metrics\"))\n        elif info.get(\"error\") is not None:\n            # Make all keys in metrics_dict lowercase\n            # will contain ['wRMSE', 'RMSE', 'MAE', 'CVRMSE', 'PNRMSE']\n            metrics_dict_lower = {k.lower(): v for k, v in info.get(\"error\").items()}\n            # do not have adjusted metrics in prior versions, so we use unadjusted metrics\n            metrics_dict_lower[\"cvrmse_adj\"] = metrics_dict_lower[\"cvrmse\"]\n            metrics_dict_lower[\"pnrmse_adj\"] = metrics_dict_lower[\"pnrmse\"]\n            daily_model.baseline_metrics = BaselineMetricsFromDict(metrics_dict_lower)\n\n        daily_model.is_fitted = True\n\n        return daily_model\n\n    @classmethod\n    def from_json(cls, str_data: str) -> DailyModel:\n        \"\"\"Create an instance of the class from a JSON string.\n\n        Args:\n            str_data: The JSON string representing the object.\n\n        Returns:\n            An instance of the class.\n\n        \"\"\"\n        return cls.from_dict(json.loads(str_data))\n\n    @classmethod\n    def from_2_0_dict(cls, data) -> DailyModel:\n        \"\"\"Create an instance of the class from a legacy (2.0) model dictionary.\n\n        Args:\n            data (dict): A dictionary containing the necessary data (legacy 2.0) to create a DailyModel instance.\n\n        Returns:\n            An instance of the class.\n\n        \"\"\"\n        daily_model = cls(model=\"legacy\")\n        daily_model.params = DailyModelParameters.from_2_0_params(data)\n        daily_model.warnings = []\n        daily_model.disqualification = []\n        daily_model.baseline_timezone = \"UTC\"\n        daily_model.is_fitted = True\n        return daily_model\n\n    @classmethod\n    def from_2_0_json(cls, str_data: str) -> DailyModel:\n        \"\"\"Create an instance of the class from a legacy (2.0) JSON string.\n\n        Args:\n            str_data: The JSON string.\n\n        Returns:\n            An instance of the class.\n\n        \"\"\"\n        return cls.from_2_0_dict(json.loads(str_data))\n\n    def plot(\n        self,\n        data: DailyBaselineData | DailyReportingData,\n    ) -> None:\n        \"\"\"Plot a model fit with baseline or reporting data. Requires matplotlib to use.\n\n        Args:\n            df_eval: The baseline or reporting data object to plot.\n        \"\"\"\n        try:\n            from opendsm.eemeter.models.daily.plot import plot\n        except ImportError:  # pragma: no cover\n            raise ImportError(\"matplotlib is required for plotting.\")\n\n        # TODO: pass more kwargs to plotting function\n\n        plot(self, self._predict(data.df))\n\n    def _create_params_from_fit_model(self):\n        submodels = {}\n        for key, submodel in self.model.items():\n            temperature_constraints = {\n                \"T_min\": submodel.T_min,\n                \"T_max\": submodel.T_max,\n                \"T_min_seg\": submodel.T_min_seg,\n                \"T_max_seg\": submodel.T_max_seg,\n            }\n            submodels[key] = DailySubmodelParameters(\n                coefficients=submodel.named_coeffs,\n                temperature_constraints=temperature_constraints,\n                f_unc=submodel.f_unc,\n            )\n        params = DailyModelParameters(\n            submodels=submodels,\n            settings=self.settings.model_dump(),\n            info={\n                \"metrics\": self.baseline_metrics.model_dump(),\n                \"baseline_timezone\": str(self.baseline_timezone),\n                \"disqualification\": [dq.json() for dq in self.disqualification],\n                \"warnings\": [warning.json() for warning in self.warnings],\n            },\n        )\n\n        return params\n\n    def _initialize_data(self, meter_data):\n        \"\"\"\n        Initializes the meter data by performing the following operations:\n        - Renames the 'model' column to 'model_old' if it exists\n        - Converts the index to a DatetimeIndex if it is not already\n        - Adds a 'season' column based on the month of the index using the settings.season dictionary\n        - Adds a 'day_of_week' column based on the day of the week of the index\n        - Removes any rows with NaN values in the 'temperature' or 'observed' columns\n        - Sorts the data by the index\n        - Reorders the columns to have 'season' and 'day_of_week' first, followed by the remaining columns\n\n        Parameters:\n        - meter_data: A pandas DataFrame containing the meter data\n\n        Returns:\n        - A pandas DataFrame containing the initialized meter data\n        - A pandas DataFrame containing rows which were dropped due to NaN in either column\n        \"\"\"\n\n        if \"predicted\" in meter_data.columns:\n            meter_data = meter_data.rename(columns={\"predicted\": \"predicted_old\"})\n\n        cols = list(meter_data.columns)\n\n        if \"datetime\" in cols:\n            meter_data.set_index(\"datetime\", inplace=True)\n            cols.remove(\"datetime\")\n\n        if not isinstance(meter_data.index, pd.DatetimeIndex):\n            try:\n                meter_data.index = pd.to_datetime(meter_data.index)\n            except:\n                raise TypeError(\"Could not convert 'meter_data.index' to datetime\")\n\n        for col in [\"season\", \"day_of_week\"]:\n            if col in cols:\n                meter_data.drop([col], axis=1, inplace=True)\n                cols.remove(col)\n\n        meter_data[\"season\"] = meter_data.index.month.map(self.settings.season._num_dict)\n        meter_data[\"day_of_week\"] = meter_data.index.dayofweek + 1\n        meter_data = meter_data.sort_index()\n        meter_data = meter_data[[\"season\", \"day_of_week\", *cols]]\n\n        dropped_rows = meter_data.copy()\n        meter_data = meter_data.dropna()\n        if meter_data.empty:\n            # return early to avoid np.isfinite exception\n            return meter_data, dropped_rows\n        meter_data = meter_data[np.isfinite(meter_data[\"temperature\"])]\n        if \"observed\" in cols:\n            meter_data = meter_data[np.isfinite(meter_data[\"observed\"])]\n\n        dropped_rows = dropped_rows.loc[~dropped_rows.index.isin(meter_data.index)]\n        return meter_data, dropped_rows\n\n    def _combinations(self):\n        \"\"\"\n        This method generates all possible combinations of seasonal and day options for the given data.\n        It then trims the combinations based on certain conditions such as minimum number of days per season,\n        and whether to allow separate splits for summer, shoulder and winter seasons.\n        \"\"\"\n\n        settings = self.settings\n\n        def _get_combinations():\n            def add_prefix(list_str, prefix):\n                return [f\"{prefix}-{s}\" for s in list_str]\n\n            def expand_combinations(combos_in):\n                \"\"\"\n                Given a list of combinations, expands each combination by adding a new item to it.\n                The new item is chosen from the intersection of the items in two specific combinations.\n                The new item is then added to a third combination, which is created by combining the remaining items from the two specific combinations.\n                The resulting expanded combinations are returned as a list.\n\n                Parameters:\n                combos_in (list): A list of combinations, where each combination is a list of items.\n\n                Returns:\n                list: A list of expanded combinations, where each expanded combination is a list of items.\n                \"\"\"\n\n                combo_expanded = []\n                for combo in combos_in:\n                    combo_expanded.append(list(combo))\n                    prefixes = [item[0] for item in combo]\n\n                    if \"wd\" in prefixes and \"we\" in prefixes:\n                        i_wd = prefixes.index(\"wd\")\n                        i_we = prefixes.index(\"we\")\n                    else:\n                        continue\n\n                    if \"fw\" in prefixes:\n                        i_fw = prefixes.index(\"fw\")\n                    else:\n                        i_fw = None\n\n                    for item in combo[i_wd][1]:\n                        if item in combo[i_we][1]:\n                            combo_0_trim = [x for x in combo[i_wd][1] if x != item]\n                            combo_1_trim = [x for x in combo[i_we][1] if x != item]\n\n                            if i_fw is None:\n                                fw_item = [\"fw\", [item]]\n                            else:\n                                fw_item = [\"fw\", [*combo[i_fw][1], item]]\n\n                            if len(combo_0_trim) == 0 and len(combo_1_trim) == 0:\n                                combo_new = [fw_item]\n                            elif len(combo_0_trim) > 0 and len(combo_1_trim) == 0:\n                                combo_new = [fw_item, [combo[i_wd][0], combo_0_trim]]\n                            elif len(combo_0_trim) == 0 and len(combo_1_trim) > 0:\n                                combo_new = [fw_item, [combo[i_we][0], combo_1_trim]]\n                            else:\n                                combo_new = [\n                                    fw_item,\n                                    [combo[i_wd][0], combo_0_trim],\n                                    [combo[i_we][0], combo_1_trim],\n                                ]\n\n                            combo_expanded.append(combo_new)\n\n                return combo_expanded\n\n            def stringify(combos):\n                \"\"\"\n                Converts a list of tuples into a list of strings, where each string is a combination of the tuple values\n                separated by '__'. The tuples are expected to have a prefix and a value, and the prefix is used to add context\n                to the value.\n\n                Parameters:\n                    combos (list): A list of tuples, where each tuple contains a prefix and a value.\n\n                Returns:\n                    list: A list of strings, where each string is a combination of the tuple values separated by '__'.\n                \"\"\"\n\n                combos_str = []\n                for combo in combos:\n                    combo = [add_prefix(item[1], item[0]) for item in combo]\n                    combo = [item for sublist in combo for item in sublist]\n                    combo = \"__\".join(combo)\n\n                    combos_str.append(combo)\n\n                combos_str = sorted(list(set(combos_str)), key=lambda x: (len(x), x))\n\n                return combos_str\n\n            for days in self.day_options:\n                season_day_combo = []\n                for day in days:\n                    season_day_combo.append(\n                        list(itertools.product([day], self.seasonal_options))\n                    )\n\n                combos_expanded = list(itertools.product(*season_day_combo))\n                for _ in range(max([len(item) for item in self.seasonal_options])):\n                    combos_expanded = expand_combinations(combos_expanded)\n\n                combos_str = stringify(combos_expanded)\n\n            return combos_str\n\n        def _trim_combinations(combo_list, split_min_days=30):\n            \"\"\"\n            Trims the list of combinations to be tested based on various conditions.\n            - Checks if the ellipsoids created are separated enough to warrant separate seasons and weekday/weekend splits.\n            - Checks if there are enough days in each season and weekday/weekend to warrant separate splits.\n\n            Args:\n                combo_list (list): List of combinations to be tested.\n                split_min_days (int, optional): Minimum number of days required for a split. Defaults to 30.\n\n            Returns:\n                list: Trimmed list of combinations to be tested.\n            \"\"\"\n\n            meter = self.df_meter\n            allow_sep_summer = settings.split_selection.allow_separate_summer\n            allow_sep_shoulder = settings.split_selection.allow_separate_shoulder\n            allow_sep_winter = settings.split_selection.allow_separate_winter\n            allow_sep_weekday_weekend = settings.split_selection.allow_separate_weekday_weekend\n\n            if settings.split_selection.reduce_splits_by_gaussian:\n                allow_split = ellipsoid_split_filter(\n                    self.df_meter, n_std=settings.split_selection.reduce_splits_num_std\n                )\n\n                if allow_sep_summer and not allow_split[\"summer\"]:\n                    allow_sep_summer = False\n\n                if allow_sep_shoulder and not allow_split[\"shoulder\"]:\n                    allow_sep_shoulder = False\n\n                if allow_sep_winter and not allow_split[\"winter\"]:\n                    allow_sep_winter = False\n\n                if allow_sep_weekday_weekend and not allow_split[\"weekday_weekend\"]:\n                    allow_sep_weekday_weekend = False\n\n            we_days = self.combo_dictionary[\"we\"]\n\n            if (meter[\"season\"].values == \"summer\").sum() < split_min_days:\n                allow_sep_summer = False\n\n            if (meter[\"season\"].values == \"shoulder\").sum() < split_min_days:\n                allow_sep_shoulder = False\n\n            if (meter[\"season\"].values == \"winter\").sum() < split_min_days:\n                allow_sep_winter = False\n\n            combo_list_trimmed = []\n            for combo in combo_list:\n                if \"fw-su_sh_wi\" == combo:  # always fit the full model with all data\n                    combo_list_trimmed.append(combo)\n                    continue\n                elif \"wd\" in combo and not allow_sep_weekday_weekend:\n                    continue\n\n                banned_season_split = {\n                    \"su\": not allow_sep_summer,\n                    \"sh\": not allow_sep_shoulder,\n                    \"wi\": not allow_sep_winter,\n                }\n\n                valid_combo = True\n                components = [item[3:] for item in combo.split(\"__\")]\n                for component in components:\n                    seasons = component.split(\"_\")\n\n                    if (len(seasons) == 1) and banned_season_split[component]:\n                        valid_combo = False\n                        break\n\n                    we_count = 0\n                    for season in seasons:\n                        we_count += (\n                            (meter[\"season\"].values == self.combo_dictionary[season])\n                            & meter[\"day_of_week\"].isin(we_days).values\n                        ).sum()\n\n                    if we_count < split_min_days / 3.75:\n                        valid_combo = False\n                        break\n\n                if valid_combo:\n                    combo_list_trimmed.append(combo)\n\n            return combo_list_trimmed\n\n        def _remove_duplicate_permutations(combo_list):\n            \"\"\"\n            Removes duplicate permutations from a list of strings.\n\n            Args:\n                combo_list (list): A list of strings representing permutations.\n\n            Returns:\n                list: A list of unique permutations.\n            \"\"\"\n\n            unique_sorted_combos = []\n            unique_combos = []\n            for combo in combo_list:\n                sorted_combo = \"__\".join(sorted(combo.split(\"__\")))\n\n                if sorted_combo not in unique_sorted_combos:\n                    unique_sorted_combos.append(combo)\n                    unique_combos.append(combo)\n\n            return unique_combos\n\n        combo_list = _get_combinations()\n        combo_list = _remove_duplicate_permutations(combo_list)\n        combo_list = _trim_combinations(combo_list)\n\n        return combo_list\n\n    def _meter_segment(self, component, meter=None):\n        \"\"\"\n        Returns a meter segment based on the given component and meter data.\n\n        Parameters:\n            component (str): A string representing the component to filter the meter data by.\n            meter (pandas.DataFrame, optional): A pandas DataFrame containing the meter data. Defaults to None.\n\n        Returns:\n            pandas.DataFrame: A pandas DataFrame containing the meter data filtered by the given component.\n        \"\"\"\n\n        if meter is None:\n            meter = self.df_meter\n\n        season_list = component[3:].split(\"_\")\n        day_list = component[:2]\n\n        seasons = [self.combo_dictionary[key] for key in season_list]\n        days = self.combo_dictionary[day_list]\n\n        meter_segment = meter[\n            meter[\"season\"].isin(seasons) & meter[\"day_of_week\"].isin(days)\n        ]\n\n        return meter_segment\n\n    def _components(self):\n        \"\"\"\n        Returns a sorted list of unique components from the combinations attribute.\n        \"\"\"\n\n        components = list(\n            set([i for item in self.combinations for i in item.split(\"__\")])\n        )\n        components = sorted(components, key=lambda x: (len(x), x))\n\n        return components\n\n    def _fit_components(self):\n        \"\"\"\n        Fits initial models for each component using the meter segment data and component settings.\n\n        If the alpha_final_type is \"last\", the settings are updated to disable the final bounds scalar and set alpha_final_type to None.\n\n        Returns:\n            dict: A dictionary containing the fitted components.\n        \"\"\"\n\n        if self.settings.alpha_final_type == \"last\":\n            settings_update = {\n                \"DEVELOPER_MODE\": True,\n                \"SILENT_DEVELOPER_MODE\": True, \n                \"ALPHA_FINAL_TYPE\": None,\n                \"FINAL_BOUNDS_SCALAR\": None,\n            }\n\n            self.component_settings = update_daily_settings(\n                self.settings, settings_update\n            )\n        else:\n            self.component_settings = self.settings\n\n        fit_components = {item: None for item in self.components}\n        for component in fit_components.keys():\n            meter_segment = self._meter_segment(component)\n\n            # Fit new models\n            fit_components[component] = fit_initial_models_from_full_model(\n                meter_segment, self.component_settings, print_res=False\n            )\n\n        return fit_components\n\n    def _combination_selection_criteria(self, combination):\n        \"\"\"\n        Calculates the selection criteria for a given combination of components.\n\n        Parameters:\n            combination (str): A string representing the combination of components.\n\n        Returns:\n            float: The selection criteria for the given combination.\n        \"\"\"\n\n        components = combination.split(\"__\")\n\n        N = np.sum([self.fit_components[X].N for X in components])\n        TSS = np.sum([self.fit_components[X].TSS for X in components])\n        # starts as added penalties based on # splits\n        # num_coeffs = 3*self.df_penalties[combination] # + np.sum([self.fit_components[X].num_coeffs for X in components])\n        num_coeffs = len(components)\n\n        if combination == \"fw-su_sh_wi\":\n            wRMSE = self.wRMSE_base\n        else:\n            wRMSE = self._get_error_metrics(combination).wrmse\n\n        loss = wRMSE / self.wRMSE_base\n\n        criteria_type = self.settings.split_selection.criteria.lower()\n        penalty_multiplier = self.settings.split_selection.penalty_multiplier\n        penalty_power = self.settings.split_selection.penalty_power\n\n        criteria = selection_criteria(\n            loss, TSS, N, num_coeffs, criteria_type, penalty_multiplier, penalty_power\n        )\n\n        return criteria\n\n    def _best_combination(self, print_out=False):\n        \"\"\"\n        Finds the best combination of parameters based on the selection criteria.\n\n        Parameters:\n            print_out (bool): Whether to print the combination and selection criteria for each iteration.\n\n        Returns:\n            str: The best combination of parameters as a string.\n        \"\"\"\n\n        HoF = {\"combination_str\": None, \"selection_criteria\": np.inf}\n        for combo in self.combinations:\n            selection_criteria = self._combination_selection_criteria(combo)\n\n            if selection_criteria < HoF[\"selection_criteria\"]:\n                HoF[\"combination_str\"] = combo\n                HoF[\"selection_criteria\"] = selection_criteria\n\n            if print_out:\n                print(f\"{combo:>40s} {selection_criteria:>8.1f}\")\n\n        if print_out:\n            print(f\"{HoF['combination_str']:>40s} {HoF['selection_criteria']:>8.1f}\")\n\n        return HoF[\"combination_str\"]\n\n    def _final_fit(self, combination):\n        \"\"\"\n        Fits the final model for a given combination of components.\n\n        Parameters:\n            combination (str): A string representing the combination of components.\n\n        Returns:\n            dict: A dictionary containing the fitted models for each component in the combination.\n        \"\"\"\n\n        model = {}\n        for component in combination.split(\"__\"):\n            settings = self.settings\n            prior_model = self.fit_components[component]\n\n            if settings.alpha_final_type is None:\n                if self.verbose:\n                    print(f\"{component}__{prior_model.model_name}\")\n\n                model[component] = prior_model\n                continue\n\n            settings_update = {\n                \"DEVELOPER_MODE\": True, \n                \"SILENT_DEVELOPER_MODE\": True, \n                \"REGULARIZATION_ALPHA\": 0.0\n            }\n            settings = update_daily_settings(self.settings, settings_update)\n\n            # separate meter appropriately\n            meter_segment = self._meter_segment(component)\n\n            # Fit new models\n            if self.verbose:\n                print(f\"{component}__{prior_model.model_name}\")\n\n            model[component] = fit_final_model(\n                meter_segment, prior_model, settings, print_res=self.verbose\n            )\n\n            model[component].settings = self.settings  # overwrite to input settings\n\n        return model\n\n    def _get_error_metrics(self, combination):\n        \"\"\"\n        Calculates the error metrics for a given combination of components.\n        RMSE and MAE are calculated as the mean of the residuals, wRMSE is calculated as the weighted mean of the residuals.\n\n        Parameters:\n            combination (str): A string representing the combination of components to calculate error metrics for.\n                If None, the best combination will be used.\n\n        Returns:\n            tuple: A tuple containing the calculated error metrics (wRMSE, RMSE, MAE).\n        \"\"\"\n\n        if combination is None:\n            combination = self.best_combination\n\n        N = 0\n        num_coeffs = 0\n        wSSE = 0\n        obs = []\n        predicted = []\n        for component in combination.split(\"__\"):\n            fit_component = self.fit_components[component]\n\n            wSSE += fit_component.wSSE\n            N += fit_component.N\n            num_coeffs += fit_component.num_coeffs\n            obs.append(fit_component.obs)\n            predicted.append(fit_component.model)\n        \n        obs = np.hstack(obs)   \n        predicted = np.hstack(predicted)\n        \n        df_meter = pd.DataFrame({\n            'observed': obs,\n            'predicted': predicted,\n        })\n\n        metrics = BaselineMetrics(\n            df=df_meter, num_model_params=num_coeffs\n        )\n\n        wRMSE = np.sqrt(wSSE / N)\n        metrics.wrmse = wRMSE\n\n        return metrics\n\n    def _predict_submodel(self, submodel, T):\n        \"\"\"\n        Predicts submodel output for a given set of temperatures.\n\n        Parameters:\n            T (numpy.ndarray): Array of temperatures.\n\n        Returns:\n            Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]:\n                Tuple containing the following arrays:\n                - model: Array of model values.\n                - f_unc: Array of uncertainties.\n                - hdd_load: Array of heating degree day loads.\n                - cdd_load: Array of cooling degree day loads.\n        \"\"\"\n\n        T_min = submodel.temperature_constraints[\"T_min\"]\n        T_max = submodel.temperature_constraints[\"T_max\"]\n        x = get_full_model_x(\n            submodel.coefficients.model_key,\n            submodel.coefficients.to_np_array(),\n            T_min,\n            T_max,\n            submodel.temperature_constraints[\"T_min_seg\"],\n            submodel.temperature_constraints[\"T_max_seg\"],\n        )\n\n        if submodel.coefficients.model_key == \"hdd_tidd_cdd_smooth\":\n            [hdd_bp, hdd_beta, pct_hdd_k, cdd_bp, cdd_beta, pct_cdd_k, intercept] = x\n            [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_smooth_coeffs(\n                hdd_bp, pct_hdd_k, cdd_bp, pct_cdd_k\n            )\n            x = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n        hdd_bp, cdd_bp, intercept = x[0], x[3], x[6]\n        T_fit_bnds = np.array([T_min, T_max])\n\n        model = full_model(*x, T_fit_bnds, T.astype(np.float64))\n        f_unc = np.ones_like(model) * submodel.f_unc\n\n        load_only = model - intercept\n\n        hdd_load = np.zeros_like(model)\n        cdd_load = np.zeros_like(model)\n\n        hdd_idx = np.argwhere(T <= hdd_bp).flatten()\n        cdd_idx = np.argwhere(T >= cdd_bp).flatten()\n\n        hdd_load[hdd_idx] = load_only[hdd_idx]\n        cdd_load[cdd_idx] = load_only[cdd_idx]\n\n        return model, f_unc, hdd_load, cdd_load"
  },
  {
    "path": "opendsm/eemeter/models/daily/objective_function.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\n\nfrom opendsm.common.utils import OoM\nfrom opendsm.common.stats.basic import fast_std as stdev\n\n\ndef get_idx(A, B):\n    \"\"\"\n    Returns a sorted list of indices of items in A that are found in any string in B.\n\n    Parameters:\n    A (list): List of items to search for in B.\n    B (list): List of strings to search for items in A.\n\n    Returns:\n    list: Sorted list of indices of items in A that are found in any string in B.\n    \"\"\"\n\n    idx = []\n    for item in A:\n        try:\n            idx.extend([B.index(txt) for txt in B if item in txt])\n        except:\n            continue\n\n    idx.sort()\n\n    return idx\n\n\ndef no_weights_obj_fcn(X, aux_inputs):\n    \"\"\"\n    Calculates the sum of squared errors (SSE) between the model output and the observed data.\n\n    Parameters:\n        X (array-like): Input values for the model.\n        aux_inputs (tuple): A tuple containing the model function, observed data, and breakpoint indices.\n\n    Returns:\n        float: The SSE between the model output and the observed data.\n    \"\"\"\n\n    model_fcn, obs, idx_bp = aux_inputs\n\n    # flip breakpoints if they are not in the correct order\n    if (len(idx_bp) > 1) and (X[idx_bp[0]] > X[idx_bp[1]]):\n        X[idx_bp[1]], X[idx_bp[0]] = X[idx_bp[0]], X[idx_bp[1]]\n\n    model = model_fcn(X)\n    resid = model - obs\n\n    SSE = np.sum(resid**2)\n\n    return SSE\n\n\ndef model_fcn_dec(model_fcn_full, T_fit_bnds, T):\n    \"\"\"\n    Returns a function that takes only X as input and returns the output of model_fcn_full with X, T_fit_bnds, and T as inputs.\n\n    Parameters:\n    - model_fcn_full: function\n        The full model function that takes X, T_fit_bnds, and T as inputs.\n    - T_fit_bnds: tuple\n        The bounds of the temperature range.\n    - T: float\n        The temperature value.\n\n    Returns:\n    - function\n        A function that takes only X as input and returns the output of model_fcn_full with X, T_fit_bnds, and T as inputs.\n    \"\"\"\n\n    def model_fcn_X_only(X):\n        return model_fcn_full(*X, T_fit_bnds, T)\n\n    return model_fcn_X_only\n\n\ndef obj_fcn_decorator(\n    model_fcn_full,\n    weight_fcn,\n    TSS_fcn,\n    T,\n    obs,\n    base_weights,\n    settings,\n    alpha=2.0,\n    coef_id=[],\n    initial_fit=True,\n):\n    \"\"\"\n    A decorator function that calculates the elastic net penalty for a given set of inputs and the objective function\n    for input to optimization algorithms.\n\n    Parameters:\n    model_fcn_full (function): The full model function.\n    weight_fcn (function): The weight function.\n    TSS_fcn (function): The TSS (Total Sum of Squares) function.\n    T (array-like): The temperature array.\n    obs (array-like): The observation array.\n    settings (object): The DailySettings object.\n    alpha (float): The alpha value for the elastic net penalty. Default is 2.0.\n    coef_id (list): The list of coefficient IDs. Default is an empty list.\n    initial_fit (bool): Whether or not this is the initial fit. Default is True.\n\n    Returns:\n    obj_fcn (function): an objective function having the required inputs for optimization via SciPy and NLopt.\n    \"\"\"\n\n    N = np.shape(obs)[0]\n    N_min = settings.segment_minimum_count  # N minimum for a sloped segment\n    sigma = 2.698  # 1.5 IQR\n    quantile = 0.25\n    min_weight = 0.00\n\n    T_fit_bnds = np.array([np.min(T), np.max(T)])\n    T_range = T_fit_bnds[1] - T_fit_bnds[0]\n\n    model_fcn = model_fcn_dec(model_fcn_full, T_fit_bnds, T)\n\n    lasso_a = settings.regularization_percent_lasso * settings.regularization_alpha\n    ridge_a = (\n        1 - settings.regularization_percent_lasso\n    ) * settings.regularization_alpha\n\n    idx_k = get_idx([\"dd_k\"], coef_id)\n    idx_beta = get_idx([\"dd_beta\"], coef_id)\n    idx_bp = get_idx([\"dd_bp\"], coef_id)\n    # idx_reg = get_idx([\"dd_beta\", \"dd_k\"], coef_id) # drop bps and intercept from regularization\n    idx_reg = get_idx(\n        [\"dd_beta\", \"dd_k\", \"dd_bp\"], coef_id\n    )  # drop intercept from regularization\n\n    def elastic_net_penalty(X, T_sorted, obs_sorted, weight_sorted, wRMSE):\n        \"\"\"\n        Calculates the elastic net penalty for a given set of inputs. The elastic net is a regularized\n        regression method that linearly combines the L1 and L2 penalties of the lasso and ridge methods.\n\n        Parameters:\n        X (array-like): The input array.\n        T_sorted (array-like): The sorted temperature array.\n        obs_sorted (array-like): The sorted observation array.\n        weight_sorted (array-like): The sorted weight array.\n        wRMSE (float): The weighted root mean squared error.\n\n        Returns:\n        penalty (float): The elastic net penalty.\n        \"\"\"\n\n        # Elastic net\n        X_enet = np.array(X).copy()\n\n        ## Scale break points ##\n        if len(idx_bp) > 0:\n            X_enet[idx_bp] = [\n                np.min(np.abs(X_enet[idx] - T_fit_bnds)) for idx in idx_bp\n            ]\n\n            if len(idx_bp) == 2:\n                X_enet[idx_bp] += (X[idx_bp][1] - X[idx_bp][0]) / 2\n\n            X_enet[idx_bp] *= wRMSE / T_range\n\n        # Find idx for regions\n        if len(idx_bp) == 2:\n            [hdd_bp, cdd_bp] = X[idx_bp]\n\n            idx_hdd = np.argwhere(T_sorted < hdd_bp).flatten()\n            idx_tidd = np.argwhere(\n                (hdd_bp <= T_sorted) & (T_sorted <= cdd_bp)\n            ).flatten()\n            idx_cdd = np.argwhere(cdd_bp < T_sorted).flatten()\n\n        elif len(idx_bp) == 1:\n            bp = X[idx_bp]\n            if X_enet[idx_beta] < 0:  # HDD_TIDD\n                idx_hdd = np.argwhere(T_sorted <= bp).flatten()\n                idx_tidd = np.argwhere(bp < T_sorted).flatten()\n                idx_cdd = np.array([])\n\n            else:\n                idx_hdd = np.array([])  # CDD_TIDD\n                idx_tidd = np.argwhere(T_sorted < bp).flatten()\n                idx_cdd = np.argwhere(bp <= T_sorted).flatten()\n\n        else:\n            idx_hdd = np.array([])\n            idx_tidd = np.arange(0, len(T_sorted))\n            idx_cdd = np.array([])\n\n        len_hdd = len(idx_hdd)\n        len_tidd = len(idx_tidd)\n        len_cdd = len(idx_cdd)\n\n        # combine tidd with hdd/cdd if cdd/hdd are large enough to get stdev\n        if (len_hdd < N_min) and (len_cdd >= N_min):\n            idx_hdd = np.hstack([idx_hdd, idx_tidd])\n        elif (len_hdd >= N_min) and (len_cdd < N_min):\n            idx_cdd = np.hstack([idx_tidd, idx_cdd])\n\n        # change to idx_hdd and idx_cdd to int arrays\n        idx_hdd = idx_hdd.astype(int)\n        idx_cdd = idx_cdd.astype(int)\n\n        ## Normalize slopes ##\n        # calculate stdevs\n        if (len(idx_bp) == 2) and (len(idx_hdd) >= N_min) and (len(idx_cdd) >= N_min):\n            N_beta = np.array([len_hdd, len_cdd])\n            T_stdev = np.array(\n                [\n                    stdev(T_sorted[idx_hdd], weights=weight_sorted[idx_hdd]),\n                    stdev(T_sorted[idx_cdd], weights=weight_sorted[idx_cdd]),\n                ]\n            )\n            obs_stdev = np.array(\n                [\n                    stdev(obs_sorted[idx_hdd], weights=weight_sorted[idx_hdd]),\n                    stdev(obs_sorted[idx_cdd], weights=weight_sorted[idx_cdd]),\n                ]\n            )\n\n        elif (len(idx_bp) == 1) and (len(idx_hdd) >= N_min):\n            N_beta = np.array([len_hdd])\n            T_stdev = stdev(T_sorted[idx_hdd], weights=weight_sorted[idx_hdd])\n            obs_stdev = stdev(obs_sorted[idx_hdd], weights=weight_sorted[idx_hdd])\n\n        elif (len(idx_bp) == 1) and (len(idx_cdd) >= N_min):\n            N_beta = np.array([len_cdd])\n            T_stdev = stdev(T_sorted[idx_cdd], weights=weight_sorted[idx_cdd])\n            obs_stdev = stdev(obs_sorted[idx_cdd], weights=weight_sorted[idx_cdd])\n\n        else:\n            N_beta = np.array([len_tidd])\n            T_stdev = stdev(T_sorted, weights=weight_sorted)\n            obs_stdev = stdev(obs_sorted, weights=weight_sorted)\n\n        X_enet[idx_beta] *= T_stdev / obs_stdev\n\n        # add penalty to slope for not having enough datapoints\n        X_enet[idx_beta] = np.where(\n            N_beta < N_min, X_enet[idx_beta] * 1e30, X_enet[idx_beta]\n        )\n\n        ## Scale smoothing parameter ##\n        if len(idx_k) > 0:  # reducing X_enet size allows for more smoothing\n            X_enet[idx_k] = X[idx_k]\n\n            if (len(idx_k) == 2) and (np.sum(X_enet[idx_k]) > 1):\n                X_enet[idx_k] /= np.sum(X_enet[idx_k])\n\n            X_enet[idx_k] *= (\n                X_enet[idx_beta] / 2\n            )  # uncertain what to divide by, this seems to work well\n\n        X_enet = X_enet[idx_reg]\n\n        if ridge_a == 0:\n            penalty = lasso_a * np.linalg.norm(X_enet, 1)\n        else:\n            penalty = lasso_a * np.linalg.norm(X_enet, 1) + ridge_a * np.linalg.norm(\n                X_enet, 2\n            )\n\n        return penalty\n\n    def obj_fcn(X, grad=[], optimize_flag=True):\n        \"\"\"\n        Creates an objective function having the required inputs for optimization via SciPy and NLopt. If the optimize_flag is true,\n        only return the loss. If the optimize_flag is false, return the loss and the model output parameters.\n\n        Parameters:\n        - X: array-like\n            Array of coefficients.\n        - grad: array-like, optional\n            Gradient array. Default is an empty list.\n        - optimize_flag: bool, optional\n            Whether to optimize. Default is True.\n\n        Returns:\n        - obj: float\n            Objective function value.\n        \"\"\"\n        X = np.array(X)\n\n        model = model_fcn(X)\n        idx_sorted = np.argsort(T).flatten()\n        idx_initial = np.argsort(idx_sorted).flatten()\n        resid = model - obs\n\n        T_sorted = T[idx_sorted]\n        # model_sorted = model[idx_sorted]\n        obs_sorted = obs[idx_sorted]\n        resid_sorted = resid[idx_sorted]\n\n        weight_sorted, c, a = weight_fcn(\n            *X, T_sorted, resid_sorted, sigma, quantile, alpha, min_weight\n        )\n        if base_weights is not None:\n            weight_sorted *= base_weights[idx_sorted]\n            weight_sorted /= np.sum(weight_sorted)\n\n        weight = weight_sorted[idx_initial]\n        wSSE = np.sum(weight * resid**2)\n        loss = wSSE / N\n\n        if settings.regularization_alpha != 0:\n            loss += elastic_net_penalty(\n                X, T_sorted, obs_sorted, weight_sorted, np.sqrt(loss)\n            )\n\n        if optimize_flag:\n            return loss\n\n        else:\n            if (\"r_squared\" in settings.split_selection.criteria) and callable(TSS_fcn):\n                TSS = TSS_fcn(*X, T_sorted, obs_sorted)\n            else:\n                TSS = wSSE\n\n            if initial_fit:\n                jac = None\n            else:\n                eps = 10 ** (OoM(X, method=\"floor\") - 2)\n                X_lower = X - eps\n                X_upper = X + eps\n\n                # select correct finite difference scheme based on variable type and value\n                # NOTE: finite differencing was not returning great results. Looking into switching to JAX autodiff\n                fd_type = [\"central\"] * len(X)\n                for i in range(len(X)):\n                    if i in idx_k:\n                        if X_lower[i] < 0:\n                            fd_type[i] = \"forward\"\n                        elif X_upper[i] > 1:\n                            fd_type[i] = \"backward\"\n                    elif i in idx_beta:\n                        if (X[i] > 0) and (X_lower[i] < 0):\n                            fd_type[i] = \"forward\"\n                        elif (X[i] < 0) and (X_upper[i] > 0):\n                            fd_type[i] = \"backward\"\n                    elif i in idx_bp:\n                        if X_lower[i] < T_sorted[0]:\n                            fd_type[i] = \"forward\"\n                        elif X_lower[i] > T_sorted[-1]:\n                            fd_type[i] = \"backward\"\n\n                # https://stackoverflow.com/questions/70572362/compute-efficiently-hessian-matrices-in-jax\n                # hess = jit(jacfwd(jacrev(no_weights_obj_fcn), has_aux=True), has_aux=True)(X, [model_fcn, obs, idx_bp])\n                # print(hess)\n                # obj_grad_fcn = lambda X: no_weights_obj_fcn(X, [model_fcn, obs, idx_bp])\n\n                # jac = numerical_jacobian(obj_grad_fcn, X, dx=eps, fd_type=fd_type)\n                jac = None\n\n            return X, loss, TSS, T, model, weight, resid, jac, np.mean(a), c\n\n    return obj_fcn\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/optimize.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom timeit import default_timer as timer\n\nimport nlopt\nimport numpy as np\nfrom scipy.optimize import (\n    direct as scipy_direct,\n    minimize as scipy_minimize,\n    minimize_scalar as scipy_minimize_scalar,\n)\nfrom opendsm.eemeter.models.daily.optimize_results import OptimizedResult\n\nnlopt_algorithms = {\n    \"nlopt_direct\": nlopt.GN_DIRECT,\n    \"nlopt_direct_noscal\": nlopt.GN_DIRECT_NOSCAL,\n    \"nlopt_direct_l\": nlopt.GN_DIRECT_L,\n    \"nlopt_direct_l_rand\": nlopt.GN_DIRECT_L_RAND,\n    \"nlopt_direct_l_noscal\": nlopt.GN_DIRECT_L_NOSCAL,\n    \"nlopt_direct_l_rand_noscal\": nlopt.GN_DIRECT_L_RAND_NOSCAL,\n    \"nlopt_orig_direct\": nlopt.GN_ORIG_DIRECT,\n    \"nlopt_orig_direct_l\": nlopt.GN_ORIG_DIRECT_L,\n    \"nlopt_crs2_lm\": nlopt.GN_CRS2_LM,\n    \"nlopt_mlsl_lds\": nlopt.G_MLSL_LDS,\n    \"nlopt_mlsl\": nlopt.G_MLSL,\n    \"nlopt_stogo\": nlopt.GD_STOGO,\n    \"nlopt_stogo_rand\": nlopt.GD_STOGO_RAND,\n    \"nlopt_ags\": nlopt.GN_AGS,\n    \"nlopt_isres\": nlopt.GN_ISRES,\n    \"nlopt_esch\": nlopt.GN_ESCH,\n    \"nlopt_cobyla\": nlopt.LN_COBYLA,\n    \"nlopt_bobyqa\": nlopt.LN_BOBYQA,\n    \"nlopt_newuoa\": nlopt.LN_NEWUOA,\n    \"nlopt_newuoa_bound\": nlopt.LN_NEWUOA_BOUND,\n    \"nlopt_praxis\": nlopt.LN_PRAXIS,\n    \"nlopt_neldermead\": nlopt.LN_NELDERMEAD,\n    \"nlopt_sbplx\": nlopt.LN_SBPLX,\n    \"nlopt_mma\": nlopt.LD_MMA,\n    \"nlopt_ccsaq\": nlopt.LD_CCSAQ,\n    \"nlopt_slsqp\": nlopt.LD_SLSQP,\n    \"nlopt_lbfgs\": nlopt.LD_LBFGS,\n    \"nlopt_tnewton\": nlopt.LD_TNEWTON,\n    \"nlopt_tnewton_precond\": nlopt.LD_TNEWTON_PRECOND,\n    \"nlopt_tnewton_restart\": nlopt.LD_TNEWTON_RESTART,\n    \"nlopt_tnewton_precond_restart\": nlopt.LD_TNEWTON_PRECOND_RESTART,\n    \"nlopt_var1\": nlopt.LD_VAR1,\n    \"nlopt_var2\": nlopt.LD_VAR2,\n}\n\nnlopt_algorithms = {k.lower(): v for k, v in nlopt_algorithms.items()}\n\npos_msg = [\n    \"Optimization terminated successfully.\",\n    \"Optimization terminated: Stop Value was reached.\",\n    \"Optimization terminated: Function tolerance was reached.\",\n    \"Optimization terminated: X tolerance was reached.\",\n    \"Optimization terminated: Max number of evaluations was reached.\",\n    \"Optimization terminated: Max time was reached.\",\n]\nneg_msg = [\n    \"Optimization failed\",\n    \"Optimization failed: Invalid arguments given\",\n    \"Optimization failed: Out of memory\",\n    \"Optimization failed: Roundoff errors limited progress\",\n    \"Optimization failed: Forced termination\",\n]\n\n\ndef obj_fcn_dec(obj_fcn, x0, bnds):\n    \"\"\"\n    Returns a function that evaluates the objective function with the given bounds.\n\n    Args:\n    - obj_fcn: the objective function to be evaluated\n    - x0: the initial guess for the optimization\n    - bnds: the bounds for the optimization\n\n    Returns:\n    - obj_fcn_eval: a function that evaluates the objective function with the given bounds\n    - idx_opt: the indices of the variables with non-equal bounds\n    \"\"\"\n\n    idx_opt = [n for n in range(np.shape(bnds)[0]) if (bnds[n, 0] < bnds[n, 1])]\n\n    def obj_fcn_eval(\n        x, *args, **kwargs\n    ):  # only modify x0 where it has bounds which are not the same\n        x0[idx_opt] = x\n\n        return obj_fcn(x0, *args, **kwargs)\n\n    return obj_fcn_eval, idx_opt\n\n\nclass BaseOptimizedResult:\n    x = None\n    success = None\n    status = None\n    message = None\n    fun = None\n    jac = None\n    hess = None\n    hess_inv = None\n    nfev = None\n    njev = None\n    nhev = None\n    nit = None\n    maxcv = None\n    time_elapsed = None\n\n\nclass BaseOptimizer:\n    def __init__(self, obj_fcn, x0, bnds, settings):\n        \"\"\"\n        The constructor for the Optimizer class.\n\n        Parameters:\n            obj_fcn (function): The objective function to be optimized.\n            x0 (np.array): The initial guess for the optimization.\n            bnds (list): The bounds for the optimization.\n            coef_id (str): The identifier for the coefficient.\n            settings (dict): The settings for the optimization.\n            opt_settings (Opt_Settings): The settings for the optimization.\n        \"\"\"\n        self.bnds = np.array(bnds)\n        self.x0 = np.clip(\n            x0, bnds[:, 0], bnds[:, 1]\n        )  # clip x0 to the bnds, just in case\n\n        self.obj_fcn, self.idx_opt = obj_fcn_dec(obj_fcn, x0, bnds)\n\n        self.settings = settings\n\n\nclass SciPyOptimizer(BaseOptimizer):\n    def run(self):\n        \"\"\"\n        Optimize the objective function using the SciPy library. Different optimization options are available,\n        such as scipy_COBYLA, scipy_SLSQP, scipy_L_BFGS_B, scipy_TNC, scipy_BFGS, scipy_Powell, scipy_Nelder-Mead.\n        options argument needs to have the algorithm specified.\n\n        Args:\n            x0 (list): Initial guess for the optimization.\n            bnds (tuple): Bounds for the optimization.\n            settings (Opt_Settings): The settings for the optimization.\n\n        Returns:\n            res_out (OptimizedResult): An object containing the results of the optimization.\n        \"\"\"\n\n        settings = self.settings\n        x0 = self.x0\n        bnds = self.bnds\n\n        timer_start = timer()\n\n        algorithm = settings.algorithm[6:]\n\n        if algorithm.lower() in [\"brent\", \"golden\", \"bounded\"]:\n            scipy_obj_fcn = lambda x: self.obj_fcn([x])\n\n            if algorithm.lower() in [\"brent\", \"golden\"]:\n                res = scipy_minimize_scalar(\n                    scipy_obj_fcn, bracket=bnds[0], method=algorithm.lower()\n                )\n\n            elif algorithm.lower() == \"bounded\":\n                res = scipy_minimize_scalar(\n                    scipy_obj_fcn, bounds=bnds[0], method=\"bounded\"\n                )\n\n            res.x = [res.x]\n\n        else:\n            x0_opt = x0[self.idx_opt]\n            bnds_opt = bnds[self.idx_opt, :]\n            bnds_opt = tuple(map(tuple, bnds_opt))\n\n            scipy_obj_fcn = lambda x: self.obj_fcn(x)\n\n            if algorithm.lower() == \"direct\":\n                res = scipy_direct(\n                    scipy_obj_fcn, \n                    bnds_opt,\n                    maxiter=int(settings.stop_criteria_value),\n                    f_min_rtol=settings.f_tol_rel,\n                )\n            else:\n                res = scipy_minimize(\n                    scipy_obj_fcn, x0_opt, method=algorithm, bounds=bnds_opt\n                )\n\n        res.time_elapsed = timer() - timer_start\n\n        return res\n    \n\nclass NLoptOptimizer(BaseOptimizer):\n    def run(self):\n        \"\"\"\n        Optimize the objective function using the NLopt library.\n\n        Args:\n            x0 (ndarray): Initial guess for the optimization.\n            bnds (ndarray): Bounds on the variables.\n            options (dict): Dictionary of options for the optimization.\n\n        Returns:\n            res_out (OptimizedResult): Object containing the results of the optimization.\n        \"\"\"\n        settings = self.settings\n        x0 = self.x0\n        bnds = self.bnds\n\n        timer_start = timer()\n\n        obj_fcn = self.obj_fcn\n        idx_opt = self.idx_opt\n\n        x0_opt = x0[idx_opt]\n        bnds_opt = bnds[idx_opt, :].T\n        \n        algorithm = nlopt_algorithms[settings.algorithm]\n\n        opt = nlopt.opt(algorithm, np.size(x0_opt))\n        opt.set_min_objective(obj_fcn)\n        if settings.stop_criteria_type == \"iteration maximum\":\n            opt.set_maxeval(int(settings.stop_criteria_value) - 1)\n        elif settings.stop_criteria_type == \"maximum time [min]\":\n            opt.set_maxtime(settings.stop_criteria_value * 60)\n\n        opt.set_xtol_rel(settings.x_tol_rel)\n        opt.set_ftol_rel(settings.f_tol_rel)\n        opt.set_lower_bounds(bnds_opt[0])\n        opt.set_upper_bounds(bnds_opt[1])\n\n        # initial_step\n        max_initial_step = np.max(np.abs(bnds_opt - x0_opt), axis=0)\n\n        initial_step = (bnds_opt[1] - bnds_opt[0]) * settings.initial_step\n\n        # TODO: bring this back in at some point?\n        # coef_id_opt = [id for n, id in enumerate(self.coef_id) if n in idx_opt]\n        # for n, coef_name in enumerate(coef_id_opt):\n        #     if \"dd_bp\" in coef_name:\n        #         initial_step[n] *= 2\n\n        #     if coef_name == \"hdd_bp\":\n        #         initial_step[n] *= -1\n\n        initial_step = np.clip(initial_step, -max_initial_step, max_initial_step)\n\n        x1 = x0_opt + initial_step\n        np.putmask(\n            initial_step, (x1 < bnds_opt[0]) | (x1 > bnds_opt[1]), -initial_step\n        )  # first step in direction of more variable space\n\n        opt.set_initial_step(initial_step)\n\n        # alter default size of population in relevant algorithms\n        if settings.algorithm == \"nlopt_crs2_lm\":\n            default_pop_size = 10 * (len(x0_opt) + 1)\n        elif settings.algorithm in [\"nlopt_mlsl_lds\", \"nlopt_mlsl\"]:\n            default_pop_size = 4\n        elif settings.algorithm == \"nlopt_isres\":\n            default_pop_size = 20 * (len(x0_opt) + 1)\n\n            opt.set_population(\n                int(np.rint(default_pop_size * settings[\"initial_pop_multiplier\"]))\n            )\n\n        # if using multistart algorithm as global, set subopt\n        if (settings.algorithm == \"nlopt_mlsl_lds\"):  \n            raise NotImplementedError(\"nlopt_mlsl_lds not implemented\")\n            local_algorithm = nlopt_algorithms[self.opt_settings.algorithm]\n            sub_opt = nlopt.opt(local_algorithm, np.size(x0_opt))\n            sub_opt.set_initial_step(initial_step)\n            sub_opt.set_xtol_rel(settings.x_tol_rel)\n            sub_opt.set_ftol_rel(settings.f_tol_rel)\n            opt.set_local_optimizer(sub_opt)\n\n        x_opt = opt.optimize(x0_opt)  # optimize!\n\n        if nlopt.SUCCESS > 0:\n            success = True\n            msg = pos_msg[nlopt.SUCCESS - 1]\n        else:\n            success = False\n            msg = neg_msg[nlopt.SUCCESS - 1]\n\n        res = BaseOptimizedResult()\n        res.x = x_opt\n        res.success = success\n        res.message = msg\n        res.fun = opt.last_optimum_value()\n        res.nfev = opt.get_numevals()\n        res.time_elapsed = timer() - timer_start\n\n        return res\n\n\nclass InitialGuessOptimizer:\n    def __init__(self, obj_fcn, x0, bnds, settings):\n        \"\"\"\n        The constructor for the Optimizer class.\n\n        Parameters:\n            obj_fcn (function): The objective function to be optimized.\n            x0 (np.array): The initial guess for the optimization.\n            bnds (list): The bounds for the optimization.\n            opt_settings (Opt_Settings): The settings for the optimization.\n        \"\"\"\n        self.x0 = np.array(x0)\n        self.bnds = np.array(bnds)\n        \n        self.obj_fcn = obj_fcn\n\n        self.settings = settings\n\n    def run(self):\n        \"\"\"\n        This method runs the optimization process.\n\n        Returns:\n            OptimizedResult: An object containing the results of the optimization.\n        \"\"\"\n        bnds = self.bnds\n\n        res_all = []\n        for settings in [self.settings]:\n            if len(res_all) == 0:\n                x0 = self.x0\n            else:\n                x0 = res_all[list(res_all.keys())[-1]].x\n\n            if settings.algorithm[:5] == \"scipy\":\n                res = SciPyOptimizer(self.obj_fcn, x0, bnds, settings).run()\n            elif settings.algorithm[:5] == \"nlopt\":\n                res = NLoptOptimizer(self.obj_fcn, x0, bnds, settings).run()\n\n            res_all.append(res)\n\n            if (\n                settings.algorithm == \"nlopt_MLSL_LDS\"\n            ):  # if using multistart algorithm, break upon finishing loop\n                break\n\n        return res_all[-1]\n\n\nclass Optimizer:\n    \"\"\"\n    This class is used to perform optimization on a given objective function using either the SciPy or NLopt library.\n    The optimization can be performed globally or locally based on the options provided.\n\n    Attributes:\n        bnds (np.array): The bounds for the optimization.\n        x0 (np.array): The initial guess for the optimization.\n        obj_fcn (function): The objective function to be optimized.\n        idx_opt (int): The index of the optimal solution.\n        coef_id (str): The identifier for the coefficient.\n        settings (dict): The settings for the optimization.\n        opt_options (dict): The options for the optimization.\n    \"\"\"\n\n    def __init__(self, obj_fcn, x0, bnds, coef_id, settings, opt_settings):\n        \"\"\"\n        The constructor for the Optimizer class.\n\n        Parameters:\n            obj_fcn (function): The objective function to be optimized.\n            x0 (np.array): The initial guess for the optimization.\n            bnds (list): The bounds for the optimization.\n            coef_id (str): The identifier for the coefficient.\n            settings (dict): The settings for the optimization.\n            opt_settings (Opt_Settings): The settings for the optimization.\n        \"\"\"\n        self.coef_id = coef_id\n        self.x0 = np.array(x0)\n        self.bnds = np.array(bnds)\n        \n        self.obj_fcn = obj_fcn\n\n        self.settings = settings\n        self.opt_settings = opt_settings\n\n    def run(self):\n        \"\"\"\n        This method runs the optimization process.\n\n        Returns:\n            OptimizedResult: An object containing the results of the optimization.\n        \"\"\"\n        bnds = self.bnds\n\n        res_all = []\n        for settings in [self.opt_settings]:\n            if len(res_all) == 0:\n                x0 = self.x0\n            else:\n                x0 = res_all[list(res_all.keys())[-1]].x\n\n            if settings.algorithm[:5] == \"scipy\":\n                optimizer_class = SciPyOptimizer\n            elif settings.algorithm[:5] == \"nlopt\":\n                optimizer_class = NLoptOptimizer\n                \n            optimizer = optimizer_class(self.obj_fcn, x0, bnds, settings)\n            res = optimizer.run()\n\n            x, mean_loss, TSS, T, model, weight, resid, jac, alpha, C = optimizer.obj_fcn(\n                res.x, optimize_flag=False\n            )\n\n            res = OptimizedResult(\n                x,\n                bnds,\n                self.coef_id,\n                alpha,\n                C,\n                T,\n                model,\n                weight,\n                resid,\n                jac,\n                mean_loss,\n                TSS,\n                res.success,\n                res.message,\n                res.nfev,\n                res.time_elapsed,\n                self.settings,\n            )\n\n            res_all.append(res)\n\n            if (\n                settings.algorithm == \"nlopt_MLSL_LDS\"\n            ):  # if using multistart algorithm, break upon finishing loop\n                break\n\n        return res_all[-1]"
  },
  {
    "path": "opendsm/eemeter/models/daily/optimize_results.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom copy import deepcopy as copy\n\nimport numpy as np\n\nfrom opendsm.common.stats.basic import unc_factor\nfrom opendsm.eemeter.models.daily.base_models.full_model import (\n    full_model,\n    get_full_model_x,\n)\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients\nfrom opendsm.eemeter.models.daily.utilities.base_model import (\n    get_smooth_coeffs,\n    get_T_bnds,\n)\nfrom opendsm.common.metrics import BaselineMetrics\nimport pandas as pd\n\ndef get_k(X, T_min_seg, T_max_seg):\n    \"\"\"\n    Calculates the heating and cooling degree day breakpoints and slopes based on the given input parameters.\n\n    Parameters:\n    X (tuple): A tuple containing the following parameters:\n        - float: The maximum temperature for the segment.\n        - float: The heating degree day value for the segment.\n        - float: The minimum temperature for the segment.\n        - float: The cooling degree day value for the segment.\n    T_min_seg (float): The minimum temperature for the segment.\n    T_max_seg (float): The maximum temperature for the segment.\n\n    Returns:\n    list: A list containing the following values:\n        - float: The heating degree day breakpoint.\n        - float: The heating degree day slope.\n        - float: The cooling degree day breakpoint.\n        - float: The cooling degree day slope.\n    \"\"\"\n\n    [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_smooth_coeffs(*X)\n\n    if X[0] >= T_max_seg:\n        hdd_bp = X[0]\n        hdd_k = 0.0\n\n        if (cdd_k == 0) and (hdd_k == 0):\n            cdd_bp = hdd_bp\n\n    if X[2] <= T_min_seg:\n        cdd_bp = X[2]\n        cdd_k = 0.0\n\n        if (cdd_k == 0) and (hdd_k == 0):\n            hdd_bp = cdd_bp\n\n    return [hdd_bp, hdd_k, cdd_bp, cdd_k]\n\n\ndef reduce_model(\n    hdd_bp,\n    hdd_beta,\n    pct_hdd_k,\n    cdd_bp,\n    cdd_beta,\n    pct_cdd_k,\n    intercept,\n    T_min,\n    T_max,\n    T_min_seg,\n    T_max_seg,\n    model_key,\n):\n    \"\"\"\n    This function takes in various parameters related to heating degree days (hdd) and cooling degree days (cdd) and\n    returns a reduced model based on the values of these parameters. The reduced model is returned as a list of\n    coefficients and a list of corresponding values.\n\n    Parameters:\n    hdd_bp (float): The heating degree day base point.\n    hdd_beta (float): The heating degree day beta value.\n    pct_hdd_k (float): The percentage of heating degree days.\n    cdd_bp (float): The cooling degree day base point.\n    cdd_beta (float): The cooling degree day beta value.\n    pct_cdd_k (float): The percentage of cooling degree days.\n    intercept (float): The intercept value.\n    T_min (float): The minimum temperature value.\n    T_max (float): The maximum temperature value.\n    T_min_seg (float): The minimum temperature segment value.\n    T_max_seg (float): The maximum temperature segment value.\n    model_key (str): The key for the model.\n\n    Returns:\n    coef_id (list): A list of coefficients for the reduced model.\n    x (list): A list of corresponding values for the reduced model.\n    \"\"\"\n\n    if (cdd_beta != 0) and (hdd_beta != 0) and ((pct_cdd_k != 0) or (pct_hdd_k != 0)):\n        coef_id = [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"hdd_k\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"cdd_k\",\n            \"intercept\",\n        ]\n        x = [hdd_bp, hdd_beta, pct_hdd_k, cdd_bp, cdd_beta, pct_cdd_k, intercept]\n\n        return coef_id, x\n\n    elif (cdd_beta != 0) and (hdd_beta != 0) and (pct_cdd_k == 0) and (pct_hdd_k == 0):\n        coef_id = [\"hdd_bp\", \"hdd_beta\", \"cdd_bp\", \"cdd_beta\", \"intercept\"]\n        x = [hdd_bp, hdd_beta, cdd_bp, cdd_beta, intercept]\n\n        return coef_id, x\n\n    if (hdd_beta != 0) and (cdd_beta == 0) and (pct_hdd_k != 0):\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"]\n        if model_key == \"hdd_tidd_cdd_smooth\":\n            [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_k(\n                [hdd_bp, pct_hdd_k, cdd_bp, pct_cdd_k], T_min_seg, T_max_seg\n            )\n            if (hdd_k == 0) and (cdd_k == 0):\n                x = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n                return reduce_model(\n                    *x, T_min, T_max, T_min_seg, T_max_seg, \"c_hdd_tidd_smooth\"\n                )\n        else:\n            hdd_k = pct_hdd_k\n\n        hdd_beta = -hdd_beta\n        x = [hdd_bp, hdd_beta, hdd_k, intercept]\n\n    elif (hdd_beta == 0) and (cdd_beta != 0) and (pct_cdd_k != 0):\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"]\n        if model_key == \"hdd_tidd_cdd_smooth\":\n            [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_k(\n                [hdd_bp, pct_hdd_k, cdd_bp, pct_cdd_k], T_min_seg, T_max_seg\n            )\n            if (hdd_k == 0) and (cdd_k == 0):\n                x = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n                return reduce_model(\n                    *x, T_min, T_max, T_min_seg, T_max_seg, \"c_hdd_tidd_smooth\"\n                )\n\n        else:\n            cdd_k = pct_cdd_k\n\n        x = [cdd_bp, cdd_beta, cdd_k, intercept]\n\n    elif (hdd_beta != 0) and (cdd_beta == 0) and (pct_hdd_k == 0):\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"intercept\"]\n        if hdd_bp >= T_max_seg:\n            hdd_bp = T_max_seg\n\n        hdd_beta = -hdd_beta\n        x = [hdd_bp, hdd_beta, intercept]\n\n    elif (hdd_beta == 0) and (cdd_beta != 0) and (pct_cdd_k == 0):\n        coef_id = [\"c_hdd_bp\", \"c_hdd_beta\", \"intercept\"]\n        if cdd_bp <= T_min_seg:\n            cdd_bp = T_min_seg\n\n        x = [cdd_bp, cdd_beta, intercept]\n\n    elif (cdd_beta == 0) and (hdd_beta == 0):\n        coef_id = [\"intercept\"]\n        x = [intercept]\n\n    return coef_id, x\n\n\n# consider rename\nclass OptimizedResult:\n    def __init__(\n        self,\n        x,\n        bnds,\n        coef_id,\n        loss_alpha,\n        C,\n        T,\n        model,\n        weight,\n        resid,\n        jac,\n        mean_loss,\n        TSS,\n        success,\n        message,\n        nfev,\n        time_elapsed,\n        settings,\n    ):\n        \"\"\"\n        Class representing the results of the optimization procedure, which can either be via Scipy or NLopt.\n\n        Parameters:\n            x (numpy.ndarray): Array of optimized coefficients.\n            bnds (List[Tuple[float, float]]): List of bounds for each coefficient.\n            coef_id (List[str]): List of coefficient names.\n            loss_alpha (float): Alpha value for the loss function.\n            C (numpy.ndarray): Array of C values.\n            T (numpy.ndarray): Array of temperatures.\n            model (numpy.ndarray): Array of model values.\n            weight (numpy.ndarray): Array of weights.\n            resid (numpy.ndarray): Array of residuals.\n            jac (numpy.ndarray): Array of jacobian values.\n            mean_loss (float): Mean loss value.\n            TSS (float): Total sum of squares.\n            success (bool): Whether the optimization was successful.\n            message (str): Optimization message.\n            nfev (int): Number of function evaluations.\n            time_elapsed (float): Time elapsed during optimization.\n            settings (OptimizationSettings): Optimization settings.\n        \"\"\"\n\n        self.coef_id = coef_id\n        self.x = x\n        self.num_coeffs = len(x)\n        self.bnds = bnds\n\n        self.loss_alpha = loss_alpha\n        self.C = C\n\n        self.N = np.shape(T)[0]\n        self.T = T\n        [self.T_min, self.T_max], [self.T_min_seg, self.T_max_seg] = get_T_bnds(\n            T, settings\n        )\n\n        self.obs = model - resid\n        self.model = model\n        self.weight = weight\n        self.resid = resid\n        self.wSSE = np.sum(weight * resid**2)\n\n        self.baseline_metrics = BaselineMetrics(\n            df=pd.DataFrame({\n                \"observed\": self.obs,\n                \"predicted\": self.model,\n            }),\n            num_model_params=self.num_coeffs\n        )\n        self.baseline_metrics.wsse = self.wSSE\n        self.baseline_metrics.wrmse = np.sqrt(self.wSSE / self.N)\n\n        self.mean_loss = mean_loss\n        self.loss = mean_loss * self.N\n        self.TSS = TSS\n\n        self.settings = settings\n\n        self.jac = []\n        self.cov = []\n        self.hess = []\n        self.hess_inv = []\n        self.x_unc = np.ones_like(x) * -1\n\n        self._prediction_uncertainty()\n\n        if jac is not None:  # for future uncertainty calculations\n            self.jac = jac\n            self.hess = jac.T * jac\n\n            try:\n                self.hess_inv = np.linalg.inv(self.hess)\n            except:  # if unable to calculate inverse use Moore-Penrose pseudo-inverse\n                self.hess_inv = np.linalg.pinv(self.hess)\n\n            MSE = np.mean(resid**2)\n            self.cov = MSE * self.hess_inv\n\n            unc_alpha = self.settings.uncertainty_alpha\n            self.x_unc = np.sqrt(np.diag(self.cov)) * unc_factor(\n                self.DoF + 1, interval=\"PI\", alpha=unc_alpha\n            )\n\n            print()\n            print(self.jac)\n            print(\", \".join([f\"{val:.3e}\" for val in self.x]))\n            print(\", \".join([f\"{val:.3e}\" for val in self.x_unc]))\n            print(f\"full fcn: {self.f_unc:.2f}\")\n            print()\n\n        self.success = success\n        self.message = message\n        self.nfev = nfev\n        self.njev = -1\n        self.nhev = -1\n        self.nit = -1\n        self.time_elapsed = time_elapsed * 1e3\n\n        self._set_model_key()\n        self._refine_model()\n\n        self.named_coeffs = ModelCoefficients.from_np_arrays(self.x, self.coef_id)\n\n        self.x = np.array(self.x)\n\n    def _prediction_uncertainty(self):  # based on std\n        \"\"\"\n        Calculate the prediction uncertainty based on the standard deviation of residuals.\n        \"\"\"\n\n        self.DoF = self.baseline_metrics.ddof_autocorr\n\n        alpha = self.settings.uncertainty_alpha\n        f_unc = np.std(self.resid) * unc_factor(\n            self.DoF + 1, interval=\"PI\", alpha=alpha\n        )\n        self.f_unc = f_unc\n\n    def _set_model_key(self):\n        \"\"\"\n        Set the model key based on the coefficient names.\n        \"\"\"\n\n        if self.coef_id == [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"hdd_k\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"cdd_k\",\n            \"intercept\",\n        ]:\n            self.model_key = \"hdd_tidd_cdd_smooth\"\n        elif self.coef_id == [\"hdd_bp\", \"hdd_beta\", \"cdd_bp\", \"cdd_beta\", \"intercept\"]:\n            self.model_key = \"hdd_tidd_cdd\"\n        elif self.coef_id == [\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"]:\n            self.model_key = \"c_hdd_tidd_smooth\"\n        elif self.coef_id == [\"c_hdd_bp\", \"c_hdd_beta\", \"intercept\"]:\n            self.model_key = \"c_hdd_tidd\"\n        elif self.coef_id == [\"intercept\"]:\n            self.model_key = \"tidd\"\n        else:\n            raise Exception(f\"Unknown model type in 'OptimizeResult'\")\n\n        self.model_name = copy(self.model_key)\n        if \"c_hdd\" in self.model_key:\n            if self.x[self.coef_id.index(\"c_hdd_beta\")] < 0:\n                self.model_name = self.model_name.replace(\"c_hdd\", \"hdd\")\n            else:\n                self.model_name = self.model_name.replace(\"c_hdd\", \"cdd\")\n\n    def _refine_model(self):\n        \"\"\"\n        Refine the model based on the model key and coefficients.\n        \"\"\"\n        # update coeffs based on model\n        x = get_full_model_x(\n            self.model_key,\n            self.x,\n            self.T_min,\n            self.T_max,\n            self.T_min_seg,\n            self.T_max_seg,\n        )\n\n        # reduce model\n        self.coef_id, self.x = reduce_model(\n            *x, self.T_min, self.T_max, self.T_min_seg, self.T_max_seg, self.model_key\n        )\n        self.num_coeffs = len(self.x)\n\n        self._set_model_key()\n\n    def eval(self, T):\n        \"\"\"\n        Evaluate the full model at given temperature inputs.\n\n        Parameters:\n            T (numpy.ndarray): Array of temperatures.\n\n        Returns:\n            Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]:\n                Tuple containing the following arrays:\n                - model: Array of model values.\n                - f_unc: Array of uncertainties.\n                - hdd_load: Array of heating degree day loads.\n                - cdd_load: Array of cooling degree day loads.\n        \"\"\"\n\n        # find if T is an array of floats, else convert to array\n        if isinstance(T, (int, float)):\n            T = np.array([T]).astype(float)\n        elif T.dtype != float:\n            T = T.astype(float)\n\n        x = get_full_model_x(\n            self.model_key,\n            self.x,\n            self.T_min,\n            self.T_max,\n            self.T_min_seg,\n            self.T_max_seg,\n        )\n\n        if self.model_key == \"hdd_tidd_cdd_smooth\":\n            [hdd_bp, hdd_beta, pct_hdd_k, cdd_bp, cdd_beta, pct_cdd_k, intercept] = x\n            [hdd_bp, hdd_k, cdd_bp, cdd_k] = get_smooth_coeffs(\n                hdd_bp, pct_hdd_k, cdd_bp, pct_cdd_k\n            )\n            x = [hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept]\n\n        hdd_bp, cdd_bp, intercept = x[0], x[3], x[6]\n        T_fit_bnds = np.array([self.T_min, self.T_max])\n\n        model = full_model(*x, T_fit_bnds, T)\n        f_unc = np.ones_like(model) * self.f_unc\n\n        load_only = model - intercept\n\n        hdd_load = np.zeros_like(model)\n        cdd_load = np.zeros_like(model)\n\n        hdd_idx = np.argwhere(T <= hdd_bp).flatten()\n        cdd_idx = np.argwhere(T >= cdd_bp).flatten()\n\n        hdd_load[hdd_idx] = load_only[hdd_idx]\n        cdd_load[cdd_idx] = load_only[cdd_idx]\n\n        return model, f_unc, hdd_load, cdd_load\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/parameters.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom enum import Enum\nfrom typing import Any, Dict, Optional\n\nimport numpy as np\nfrom pydantic import BaseModel, ConfigDict\n\n\nclass ModelType(str, Enum):\n    # Full model \\_/\n    HDD_TIDD_CDD_SMOOTH = \"hdd_tidd_cdd_smooth\"\n    HDD_TIDD_CDD = \"hdd_tidd_cdd\"\n\n    # Heating, temp independent \\__\n    HDD_TIDD_SMOOTH = \"hdd_tidd_smooth\"\n    HDD_TIDD = \"hdd_tidd\"\n\n    # Temp independent, cooling __/\n    TIDD_CDD_SMOOTH = \"tidd_cdd_smooth\"\n    TIDD_CDD = \"tidd_cdd\"\n\n    # Temp independent, ___\n    TIDD = \"tidd\"\n\n\nclass ModelCoefficients(BaseModel):\n    \"\"\"\n    A class used to represent the coefficients of a model.\n\n    Attributes\n    ----------\n    model_type : ModelType\n        The type of the model.\n    intercept : float\n        The intercept of the model.\n    hdd_bp : float | None\n        The heating degree days breakpoint of the model, if applicable.\n    hdd_beta : float | None\n        The heating degree days beta of the model, if applicable.\n    hdd_k : float | None\n        The heating degree days k of the model, if applicable.\n    cdd_bp : float | None\n        The cooling degree days breakpoint of the model, if applicable.\n    cdd_beta : float | None\n        The cooling degree days beta of the model, if applicable.\n    cdd_k : float | None\n        The cooling degree days k of the model, if applicable.\n\n    Methods\n    -------\n    from_np_arrays(coefficients, coefficient_ids)\n        Constructs a ModelCoefficients object from numpy arrays of coefficients and their corresponding ids.\n    to_np_array()\n        Converts the ModelCoefficients object to a numpy array.\n    \"\"\"\n\n    \"\"\"\n    A class used to represent the coefficients of a model.\n\n    Attributes\n    ----------\n    model_type : ModelType\n        The type of the model.\n    intercept : float\n        The intercept of the model.\n    hdd_bp : float | None\n        The heating degree days breakpoint of the model, if applicable.\n    hdd_beta : float | None\n        The heating degree days beta of the model, if applicable.\n    hdd_k : float | None\n        The heating degree days k of the model, if applicable.\n    cdd_bp : float | None\n        The cooling degree days breakpoint of the model, if applicable.\n    cdd_beta : float | None\n        The cooling degree days beta of the model, if applicable.\n    cdd_k : float | None\n        The cooling degree days k of the model, if applicable.\n\n    Methods\n    -------\n    from_np_arrays(coefficients, coefficient_ids)\n        Constructs a ModelCoefficients object from numpy arrays of coefficients and their corresponding ids.\n    to_np_array()\n        Converts the ModelCoefficients object to a numpy array.\n    \"\"\"\n\n    model_type: ModelType\n    intercept: float\n    hdd_bp: Optional[float] = None\n    hdd_beta: Optional[float] = None\n    hdd_k: Optional[float] = None\n    cdd_bp: Optional[float] = None\n    cdd_beta: Optional[float] = None\n    cdd_k: Optional[float] = None\n\n    # suppress namespace warning for model_type\n    model_config = ConfigDict(protected_namespaces=())\n\n    @property\n    def is_smooth(self):\n        return self.model_type in [\n            ModelType.HDD_TIDD_CDD_SMOOTH,\n            ModelType.HDD_TIDD_SMOOTH,\n            ModelType.TIDD_CDD_SMOOTH,\n        ]\n\n    @property\n    def model_key(self):\n        \"\"\"Used inside OptimizedResult when reducing model\"\"\"\n        if self.model_type == ModelType.HDD_TIDD_CDD_SMOOTH:\n            return \"hdd_tidd_cdd_smooth\"\n        elif self.model_type == ModelType.HDD_TIDD_CDD:\n            return \"hdd_tidd_cdd\"\n        elif self.model_type in [ModelType.HDD_TIDD_SMOOTH, ModelType.TIDD_CDD_SMOOTH]:\n            return \"c_hdd_tidd_smooth\"\n        elif self.model_type in [ModelType.HDD_TIDD, ModelType.TIDD_CDD]:\n            return \"c_hdd_tidd\"\n        elif self.model_type == ModelType.TIDD:\n            return \"tidd\"\n\n    @classmethod\n    def from_np_arrays(cls, coefficients, coefficient_ids):\n        \"\"\"\n        This class method creates a ModelCoefficients instance from numpy arrays of coefficients and their corresponding ids.\n\n        Args:\n            cls (class): The class to which this class method belongs.\n            coefficients (np.array): A numpy array of coefficients.\n            coefficient_ids (list): A list of coefficient ids.\n\n        Returns:\n            ModelCoefficients: An instance of ModelCoefficients class.\n\n        Raises:\n            ValueError: If the coefficient_ids do not match any of the expected patterns.\n\n        The method matches the coefficient_ids to predefined patterns and based on the match,\n        it initializes a ModelCoefficients instance with the corresponding model_type and coefficients.\n        If the coefficient_ids do not match any of the predefined patterns, it raises a ValueError.\n        \"\"\"\n\n        if coefficient_ids == [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"hdd_k\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"cdd_k\",\n            \"intercept\",\n        ]:\n            hdd_bp = coefficients[0]\n            hdd_beta = coefficients[1]\n            hdd_k = coefficients[2]\n            cdd_bp = coefficients[3]\n            cdd_beta = coefficients[4]\n            cdd_k = coefficients[5]\n            if cdd_bp < hdd_bp:\n                hdd_bp, cdd_bp = cdd_bp, hdd_bp\n                hdd_beta, cdd_beta = cdd_beta, hdd_beta\n                hdd_k, cdd_k = cdd_k, hdd_k\n            return ModelCoefficients(\n                model_type=ModelType.HDD_TIDD_CDD_SMOOTH,\n                hdd_bp=hdd_bp,\n                hdd_beta=hdd_beta,\n                hdd_k=hdd_k,\n                cdd_bp=cdd_bp,\n                cdd_beta=cdd_beta,\n                cdd_k=cdd_k,\n                intercept=coefficients[6],\n            )\n        elif coefficient_ids == [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"intercept\",\n        ]:\n            hdd_bp = coefficients[0]\n            hdd_beta = coefficients[1]\n            cdd_bp = coefficients[2]\n            cdd_beta = coefficients[3]\n            if cdd_bp < hdd_bp:\n                hdd_bp, cdd_bp = cdd_bp, hdd_bp\n                hdd_beta, cdd_beta = cdd_beta, hdd_beta\n            return ModelCoefficients(\n                model_type=ModelType.HDD_TIDD_CDD,\n                hdd_bp=hdd_bp,\n                hdd_beta=hdd_beta,\n                cdd_bp=cdd_bp,\n                cdd_beta=cdd_beta,\n                intercept=coefficients[4],\n            )\n        elif coefficient_ids == [\n            \"c_hdd_bp\",\n            \"c_hdd_beta\",\n            \"c_hdd_k\",\n            \"intercept\",\n        ]:\n            if coefficients[1] < 0:  # model is heating dependent\n                hdd_bp = coefficients[0]\n                hdd_beta = coefficients[1]\n                hdd_k = coefficients[2]\n                cdd_bp = cdd_beta = cdd_k = None\n                model_type = ModelType.HDD_TIDD_SMOOTH\n            else:  # model is cooling dependent\n                cdd_bp = coefficients[0]\n                cdd_beta = coefficients[1]\n                cdd_k = coefficients[2]\n                hdd_bp = hdd_beta = hdd_k = None\n                model_type = ModelType.TIDD_CDD_SMOOTH\n            return ModelCoefficients(\n                model_type=model_type,\n                hdd_bp=hdd_bp,\n                hdd_beta=hdd_beta,\n                hdd_k=hdd_k,\n                cdd_bp=cdd_bp,\n                cdd_beta=cdd_beta,\n                cdd_k=cdd_k,\n                intercept=coefficients[3],\n            )\n        elif coefficient_ids == [\n            \"c_hdd_bp\",\n            \"c_hdd_beta\",\n            \"intercept\",\n        ]:\n            if coefficients[1] < 0:  # model is heating dependent\n                hdd_bp = coefficients[0]\n                hdd_beta = coefficients[1]\n                cdd_bp = cdd_beta = None\n                model_type = ModelType.HDD_TIDD\n            else:  # model is cooling dependent\n                cdd_bp = coefficients[0]\n                cdd_beta = coefficients[1]\n                hdd_bp = hdd_beta = None\n                model_type = ModelType.TIDD_CDD\n            return ModelCoefficients(\n                model_type=model_type,\n                hdd_bp=hdd_bp,\n                hdd_beta=hdd_beta,\n                cdd_bp=cdd_bp,\n                cdd_beta=cdd_beta,\n                intercept=coefficients[2],\n            )\n        elif coefficient_ids == [\n            \"intercept\",\n        ]:\n            return ModelCoefficients(\n                model_type=ModelType.TIDD,\n                intercept=coefficients[0],\n            )\n        else:\n            raise ValueError\n\n    def to_np_array(self):\n        \"\"\"\n        This method converts the model parameters into a numpy array based on the model type.\n\n        The model type determines which parameters are included in the array. The parameters are:\n        - hdd_bp: The base point for heating degree days (HDD)\n        - hdd_beta: The beta coefficient for HDD\n        - hdd_k: The smoothing parameter for HDD\n        - cdd_bp: The base point for cooling degree days (CDD)\n        - cdd_beta: The beta coefficient for CDD\n        - cdd_k: The smoothing parameter for CDD\n        - intercept: The model's intercept\n\n        Returns:\n            np.array: A numpy array containing the relevant parameters for the model type.\n        \"\"\"\n\n        if self.model_type == ModelType.HDD_TIDD_CDD_SMOOTH:\n            return np.array(\n                [\n                    self.hdd_bp,\n                    self.hdd_beta,\n                    self.hdd_k,\n                    self.cdd_bp,\n                    self.cdd_beta,\n                    self.cdd_k,\n                    self.intercept,\n                ]\n            )\n        elif self.model_type == ModelType.HDD_TIDD_CDD:\n            return np.array(\n                [\n                    self.hdd_bp,\n                    self.hdd_beta,\n                    self.cdd_bp,\n                    self.cdd_beta,\n                    self.intercept,\n                ]\n            )\n        elif self.model_type == ModelType.HDD_TIDD_SMOOTH:\n            return np.array([self.hdd_bp, self.hdd_beta, self.hdd_k, self.intercept])\n        elif self.model_type == ModelType.TIDD_CDD_SMOOTH:\n            return np.array([self.cdd_bp, self.cdd_beta, self.cdd_k, self.intercept])\n        elif self.model_type == ModelType.HDD_TIDD:\n            return np.array([self.hdd_bp, self.hdd_beta, self.intercept])\n        elif self.model_type == ModelType.TIDD_CDD:\n            return np.array([self.cdd_bp, self.cdd_beta, self.intercept])\n        elif self.model_type == ModelType.TIDD:\n            return np.array([self.intercept])\n\n\nclass DailySubmodelParameters(BaseModel):\n    coefficients: ModelCoefficients\n    temperature_constraints: Dict[str, float]\n    f_unc: float\n\n    @property\n    def model_type(self):\n        return self.coefficients.model_type\n\n\nclass DailyModelParameters(BaseModel):\n    submodels: Dict[str, DailySubmodelParameters]\n    info: Optional[Dict[str, Any]]\n    settings: Optional[Dict[str, Any]]\n\n    @classmethod\n    def from_2_0_params(cls, data):\n        model_2_0 = data.get(\"model_type\")\n        if model_2_0 == \"intercept_only\":\n            model_type = ModelType.TIDD\n        elif model_2_0 == \"hdd_only\":\n            model_type = ModelType.HDD_TIDD\n        elif model_2_0 == \"cdd_only\":\n            model_type = ModelType.TIDD_CDD\n        elif model_2_0 == \"cdd_hdd\":\n            model_type = ModelType.HDD_TIDD_CDD\n        elif model_2_0 is None:\n            raise ValueError(\"Missing model type\")\n        else:\n            raise ValueError(f\"Unknown model type: {model_2_0}\")\n        params = data[\"model_params\"]\n\n        hdd_beta = params.get(\"beta_hdd\")\n        if model_type == ModelType.HDD_TIDD:\n            # sign is reversed for heating-only model\n            hdd_beta *= -1\n\n        daily_coeffs = ModelCoefficients(\n            model_type=model_type,\n            intercept=params.get(\"intercept\"),\n            hdd_bp=params.get(\"heating_balance_point\"),\n            hdd_beta=hdd_beta,\n            cdd_bp=params.get(\"cooling_balance_point\"),\n            cdd_beta=params.get(\"beta_cdd\"),\n        )\n        submodel_params = DailySubmodelParameters(\n            coefficients=daily_coeffs,\n            temperature_constraints={\n                \"T_min\": -100,\n                \"T_min_seg\": -100,\n                \"T_max\": 200,\n                \"T_max_seg\": 200,\n            },\n            f_unc=np.inf,\n        )\n        return cls(\n            # TODO handle settings correctly with something in config.py\n            settings={\"from 2.0 - will fail if attempting from_dict()\": True},\n            info={},\n            submodels={\n                # no splits, full calendar\n                \"fw-su_sh_wi\": submodel_params,\n            },\n        )\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/plot.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport colorsys\nfrom copy import deepcopy as copy\n\nimport matplotlib as mpl\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfrom opendsm.common.stats.outliers import IQR_outlier\nfrom opendsm.eemeter.models.daily.utilities.ellipsoid_test import (\n    robust_confidence_ellipse,\n)\n\nfontsize = 14\nmpl.rc(\"font\", family=\"sans-serif\")\nc = [\"tab:blue\", \"tab:green\", \"tab:purple\"]\n\n\ndef adjust_lightness(color, amount=1.0):\n    try:\n        c = mpl.colors.cnames[color]\n    except:\n        c = color\n\n    c = colorsys.rgb_to_hls(*mpl.colors.to_rgb(c))\n\n    return colorsys.hls_to_rgb(c[0], max(0, min(1, amount * c[1])), c[2])\n\n\ndef plot(\n    fit,\n    meter_eval,\n    include_resid=False,\n    plot_gaussian_ellipses=False,\n    plot_outliers=True,\n):\n    # sort meter_eval by temperature\n    meter_eval = meter_eval.sort_values(by=\"temperature\")\n    meter_eval = meter_eval.dropna(subset=[\"temperature\", \"predicted\"])\n\n    fig = plt.figure(figsize=(14, 4), dpi=300)\n    if include_resid:\n        gs = fig.add_gridspec(2, hspace=0, height_ratios=[2.5, 1])\n        ax = gs.subplots()\n    else:\n        ax = [fig.subplots()]\n\n    # Plot scatter and Gaussian ellipses\n    for n, season in enumerate([\"summer\", \"shoulder\", \"winter\"]):\n        for day_type, day_num in enumerate([[0, 1, 2, 3, 4], [5, 6]]):\n            if day_type == 0:\n                color = c[n]\n                marker = \"o\"\n                s = 7**2\n                label = f\"{season} weekday\"\n            else:\n                color = adjust_lightness(copy(c[n]), amount=0.8)\n                marker = \"D\"\n                s = 5.5**2\n                label = f\"{season} weekend\"\n\n            meter_season = meter_eval[\n                (meter_eval[\"season\"] == season) & (meter_eval[\"observed\"].notna())\n            ]\n            meter_season = meter_season[meter_season[\"day_of_week\"].isin(day_num)]\n\n            T = meter_season[\"temperature\"].values\n            obs = meter_season[\"observed\"].values\n            model = meter_season[\"predicted\"].values\n            resid = obs - model\n\n            ax[0].scatter(T, obs, color=color, marker=marker, s=s, label=label)\n            if include_resid:\n                ax[1].scatter(T, resid, color=color, marker=marker, s=s)\n\n            if not plot_gaussian_ellipses:\n                continue\n\n            std_sqr = std = np.array(fit.model_settings.reduce_splits_num_std)[:, None]\n            std_sqr = std.T * std\n\n            mu, cov, a, b, phi = robust_confidence_ellipse(T, obs, std_sqr)\n\n            ell = mpl.patches.Ellipse(\n                mu, 2 * a, 2 * b, np.degrees(phi), color=color, zorder=10\n            )\n            ell.set_clip_box(ax[0].bbox)\n            ell.set_alpha(0.3)\n            ax[0].add_artist(ell)\n\n    # Plot models\n    for split in meter_eval[\"model_split\"].unique():\n        meter_segment = meter_eval[meter_eval[\"model_split\"] == split]\n\n        name = f\"{split}__{meter_segment['model_type'].iloc[0]}\"\n        ax[0].plot(\n            meter_segment[\"temperature\"],\n            meter_segment[\"predicted\"],\n            color=\"tab:orange\",\n            label=f\"{name}\",\n        )\n\n    # ax[0].plot(T, model[\"c_hdd_baseline\"].model, color=\"tab:red\", label=f\"c_hdd_baseline\")\n\n    if include_resid:\n        ax[1].axhline(y=0, linestyle=(0, (5, 1)), linewidth=1.5, color=(0.4, 0.4, 0.4))\n        ax[0].get_shared_x_axes().join(ax[0], ax[1])\n        ax[1].set_xlabel(\"Temperature\", labelpad=10, fontsize=fontsize)\n        ax[1].set_ylabel(\"Resid\", labelpad=10, fontsize=fontsize)\n\n    else:\n        ax[0].set_xlabel(\"Temperature\", labelpad=10, fontsize=fontsize)\n\n    # ax.plot(hours, meter[:,2], linewidth=1.5, linestyle=(0, (6, 1)), color='firebrick')\n    # ax.plot(hours, meter[:,2], linewidth=2.0, linestyle='-.')\n    # ax.fill_between(hours, cg_lb, cg_ub, alpha=0.3, facecolor='peru')\n\n    # ax.set_xlim([T[0], T[-1]])\n    # ax.set_xticks(np.arange(0, 505, 168))\n    ax[0].tick_params(axis=\"both\", which=\"major\", labelsize=0.85 * fontsize)\n\n    if not plot_outliers:\n        # Ignores crazy points when plotting based on iqr\n        ylim = IQR_outlier(\n            meter_eval[\"observed\"].values, sigma_threshold=1.0, quantile=0.025\n        )\n        ylim_idx = [\n            np.argmin(np.abs(x - meter_eval[\"observed\"].values), axis=0) for x in ylim\n        ]\n        ylim = meter_eval[\"observed\"].values[ylim_idx]\n    else:\n        ylim = np.quantile(meter_eval[\"observed\"], [0, 1])\n\n    ylim_border = 0.1 * (ylim[1] - ylim[0])\n    ax[0].set_ylim([ylim[0] - ylim_border, ylim[1] + ylim_border])\n    # ax.xaxis.set_minor_locator(mpl.ticker.AutoMinorLocator(7))\n    # ax.tick_params(axis='both', which='major', labelsize=0.85*fontsize)\n    # ax.yaxis.set_tick_params(which='minor', left=False)\n    ax[0].set_ylabel(\"Usage\", labelpad=10, fontsize=fontsize)\n\n    legend = ax[0].legend(framealpha=0.0, fontsize=0.5 * fontsize)\n    # legend._legend_box.align = 'left'\n\n    plt.show()\n\n    # if figsize is None:\n    #     figsize = (10, 4)\n\n    # if ax is None:\n    #     fig, ax = plt.subplots(figsize=figsize)\n\n    # color = \"C1\"\n    # alpha = 1\n\n    # temp_min, temp_max = (30, 90) if temp_range is None else temp_range\n\n    # temps = np.arange(temp_min, temp_max)\n\n    # prediction_index = pd.date_range(\n    #     \"2017-01-01T00:00:00Z\", periods=len(temps), freq=\"D\"\n    # )\n\n    # temps_daily = pd.Series(temps, index=prediction_index).resample(\"D\").mean()\n    # prediction = self._predict(temps_daily).model\n\n    # plot_kwargs = {\"color\": color, \"alpha\": alpha or 0.3}\n    # ax.plot(temps, prediction, **plot_kwargs)\n\n    # if title is not None:\n    #     ax.set_title(title)\n\n    return ax\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/base_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numba\nimport numpy as np\nfrom scipy.optimize import minimize_scalar\nfrom scipy.special import lambertw\nfrom scipy.stats import linregress, theilslopes\n\nfrom opendsm.common.utils import OoM_numba, log_cosh\n\n\ndef get_intercept(y, alpha=2):\n    \"\"\"\n    Calculates the intercept of a linear regression model.\n\n    Parameters:\n    -----------\n    y : array-like\n        Dependent variable.\n    alpha : float, optional\n        Significance level for the Theil-Sen estimator. Default is 2.\n\n    Returns:\n    --------\n    intercept : float\n        Intercept of the linear regression model.\n    \"\"\"\n\n    if alpha == 2:\n        intercept = np.mean(y)\n    else:\n        intercept = np.median(y)\n\n    return intercept\n\n\ndef get_slope(x, y, x_bp, intercept, alpha=2):\n    def slope_fcn_dec(x, y, x_bp, intercept, alpha):\n        def slope_fcn(slope):  # TODO: This function could be numba'd\n            model = slope * (x - x_bp) + intercept\n            resid = y - model\n\n            if alpha == 2:\n                obj = np.sqrt(np.sum(resid**2))\n            else:\n                # obj = np.sum(np.abs(resid)) # MAE\n                obj = np.sum(log_cosh(resid))                   \n\n            return obj\n\n        return slope_fcn\n\n    opt_fcn = slope_fcn_dec(x, y, x_bp, intercept, alpha)\n\n    slope = minimize_scalar(opt_fcn, method=\"brent\", tol=0.1).x\n\n    return slope\n\n\ndef linear_fit(x, y, alpha):\n    if len(set(x)) == 1:\n        slope = 0\n        intercept = x[0]\n    elif alpha == 2:\n        res = linregress(x, y)\n\n        slope = res.slope\n        intercept = res.intercept\n    else:\n        slope, intercept, _, _ = theilslopes(y, x, alpha=0.95)\n\n    return slope, intercept\n\n\n# smoothed curve will match unsmoothed at the perc_match% decay of the exp\n# if pct_match == 1.0, then it will converge at - or + inf\ndef get_smooth_coeffs(hdd_bp, pct_hdd_k, cdd_bp, pct_cdd_k, min_pct_k=0.01):\n    if (pct_hdd_k < min_pct_k) and (pct_cdd_k < min_pct_k):\n        return np.array([hdd_bp, 0, cdd_bp, 0])\n\n    pct_match = 1\n    hdd_w = cdd_w = 0\n    if pct_match < 1:\n        hdd_w = lambertw((1 - pct_match) / np.exp(1)).real\n        # cdd_w = lambertw(-(1 - pct_match)/np.exp(1)).real\n        cdd_w = -hdd_w\n\n    pct_k_sum = pct_hdd_k + pct_cdd_k\n    if pct_k_sum > 1:\n        pct_hdd_k /= pct_k_sum\n        pct_cdd_k /= pct_k_sum\n\n    # calculate the smoothing parameter as a percentage of the maximum allowed k\n    hdd_k = pct_hdd_k * (cdd_bp - hdd_bp) / (1 - hdd_w)\n    cdd_k = pct_cdd_k * (cdd_bp - hdd_bp) / (1 + cdd_w)\n\n    # move breakpoints based on k\n    hdd_bp = hdd_bp + hdd_k * (1 - hdd_w)\n    cdd_bp = cdd_bp - cdd_k * (1 + cdd_w)\n\n    return np.array([hdd_bp, hdd_k, cdd_bp, cdd_k])\n\n\n@numba.jit(nopython=True, error_model=\"numpy\", cache=True)\ndef fix_identical_bnds(bnds):\n    for i in np.argwhere(bnds[:, 0] == bnds[:, 1]):\n        bnds[i, :] = bnds[i, :] + np.array([-1.0, 1.0]) * 10 ** OoM_numba(\n            bnds[i, 0], method=\"floor\"\n        )\n\n    return bnds\n\n\ndef get_T_bnds(T, settings):\n    n_min_seg = settings.segment_minimum_count\n\n    T_min = np.min(T)\n    T_max = np.max(T)\n    T_min_seg = np.partition(T, n_min_seg)[n_min_seg]\n    T_max_seg = np.partition(T, -n_min_seg)[-n_min_seg]\n\n    return [T_min, T_max], [T_min_seg, T_max_seg]\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/const.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom enum import Enum\n\n\n# TODO: this is copy-pasted from gridmeter branch, need to merge\n\n\n\"\"\"data_settings constants\"\"\"\ndefault_season_def = {\n    \"options\": [\"summer\", \"shoulder\", \"winter\"],\n    \"January\": \"winter\",\n    \"February\": \"winter\",\n    \"March\": \"shoulder\",\n    \"April\": \"shoulder\",\n    \"May\": \"shoulder\",\n    \"June\": \"summer\",\n    \"July\": \"summer\",\n    \"August\": \"summer\",\n    \"September\": \"summer\",\n    \"October\": \"shoulder\",\n    \"November\": \"winter\",\n    \"December\": \"winter\",\n}\n\n\ndefault_weekday_weekend_def = {\n    \"options\": [\"weekday\", \"weekend\"],\n    \"Monday\": \"weekday\",\n    \"Tuesday\": \"weekday\",\n    \"Wednesday\": \"weekday\",\n    \"Thursday\": \"weekday\",\n    \"Friday\": \"weekday\",\n    \"Saturday\": \"weekend\",\n    \"Sunday\": \"weekend\",\n}\n\n\nseason_num = {\n    \"january\": 1,\n    \"february\": 2,\n    \"march\": 3,\n    \"april\": 4,\n    \"may\": 5,\n    \"june\": 6,\n    \"july\": 7,\n    \"august\": 8,\n    \"september\": 9,\n    \"october\": 10,\n    \"november\": 11,\n    \"december\": 12,\n}\n\n\nweekday_num = {\n    \"monday\": 1,\n    \"tuesday\": 2,\n    \"wednesday\": 3,\n    \"thursday\": 4,\n    \"friday\": 5,\n    \"saturday\": 6,\n    \"sunday\": 7,\n}"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/ellipsoid_test.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nfrom scipy.linalg import eigh\nfrom scipy.ndimage import median_filter\nfrom scipy.optimize import minimize_scalar\n\n\ndef ellipsoid_intersection_test(mu_A, mu_B, cov_A, cov_B):\n    \"\"\"\n    Tests whether two ellipsoids intersect or not. The ellipsoids are defined by their mean vectors and covariance matrices.\n    The function uses the K-function to calculate the intersection of the ellipsoids. If the K-function is greater than or equal to 0,\n    then the ellipsoids intersect, otherwise they do not.\n\n    Parameters:\n    mu_A (numpy.ndarray): Mean vector of the first ellipsoid.\n    mu_B (numpy.ndarray): Mean vector of the second ellipsoid.\n    cov_A (numpy.ndarray): Covariance matrix of the first ellipsoid.\n    cov_B (numpy.ndarray): Covariance matrix of the second ellipsoid.\n\n    Returns:\n    bool: True if the ellipsoids intersect, False otherwise.\n    \"\"\"\n\n    # Fix if all values are the same in 1 direction, \"brent\" doesn't work well with this\n    if cov_A[1, 1] == 0:\n        cov_A[1, 1] = 1e-14\n\n    if cov_B[1, 1] == 0:\n        cov_B[1, 1] = 1e-14\n\n    lambdas, phi = eigh(cov_A, b=cov_B)\n    v_squared = np.dot(phi.T, mu_A - mu_B) ** 2\n\n    res = minimize_scalar(\n        ellipsoid_K_function,\n        #   bracket = [0.0, 0.5, 1.0],\n        bounds=[0.0, 1.0],\n        args=(lambdas, v_squared),\n        method=\"bounded\",\n    )\n\n    if res.fun[0] >= 0:\n        return True\n    return False\n\n\ndef ellipsoid_K_function(ss, lambdas, v_squared):\n    \"\"\"\n    The K-function is a measure of spatial point pattern, often used in spatial statistics\n    to analyze the clustering or dispersion of points in a dataset. The formula used in this\n    code is a specific calculation for an ellipsoid.\n\n    Parameters:\n    ss (float): A scalar value between 0 and 1.\n    lambdas (numpy.ndarray): A 1D numpy array of eigenvalues of the covariance matrix.\n    v_squared (numpy.ndarray): A 1D numpy array of squared differences between the means of two ellipsoids.\n\n    Returns:\n    float: The value of the K-function for the given input values.\n    \"\"\"\n    ss = np.array(ss).reshape((-1, 1))\n    lambdas = np.array(lambdas).reshape((1, -1))\n    v_squared = np.array(v_squared).reshape((1, -1))\n\n    return 1 - np.sum(v_squared * ((ss * (1 - ss)) / (1 + ss * (lambdas - 1))), axis=1)\n\n\ndef confidence_ellipse(x, y, var=np.ones([2, 2]) * 1.96):\n    \"\"\"\n    Compute the confidence ellipse for a 2D dataset.\n\n    Parameters:\n    x (numpy.ndarray): The x-coordinates of the data points.\n    y (numpy.ndarray): The y-coordinates of the data points.\n    var (numpy.ndarray): The variance of the data points. Default is 1.96.\n\n    Returns:\n    list: A list containing the mean, covariance, major and minor axis lengths, and rotation angle of the ellipse.\n\n    \"\"\"\n\n    # Applying a median filter to help with outliers\n    idx_sorted = np.argsort(x).flatten()\n    idx_original = np.argsort(idx_sorted).flatten()\n\n    # size could be changed with justification\n    y = median_filter(y[idx_sorted], size=5)[idx_original]\n\n    # Computing the covariance and ellipse parameter values\n    cov = np.cov(x, y) * var  # scale covariances by std choice\n    ab_sqr, v = np.linalg.eig(cov)\n    [a, b] = np.sqrt(ab_sqr)\n    phi = np.arctan2(*v[:, 0][::-1])\n\n    mu = np.array([np.mean(x), np.mean(y)])\n\n    return mu, cov, a, b, phi\n\n\ndef robust_confidence_ellipse(x, y, var=np.ones([2, 2]) * 1.96, outlier_std=3, N=3):\n    \"\"\"\n    Computes a robust confidence ellipse for a set of points.\n\n    Parameters:\n    x (numpy.ndarray): Array of x-coordinates of the points.\n    y (numpy.ndarray): Array of y-coordinates of the points.\n    var (numpy.ndarray): Variance-covariance matrix. Default is a 2x2 matrix with 1.96 in the diagonal.\n    outlier_std (float): Standard deviation for outlier detection. Default is 3.\n    N (int): Number of iterations for outlier removal. Default is 3.\n\n    Returns:\n    list: A list containing the mean, covariance matrix, major and minor axis lengths, and rotation angle of the ellipse.\n    \"\"\"\n\n    var_outlier = np.ones([2, 2]) * outlier_std**2\n\n    # remove outliers in N iterations\n    for n in range(N):\n        if len(x) <= 1 or np.all(x == x[0]) or np.all(y == y[0]):\n            break\n\n        mu, cov, a, b, phi = confidence_ellipse(x, y, var_outlier)\n\n        if a == 0 or b == 0:\n            break\n\n        # Center points\n        xc = x - mu[0]\n        yc = y - mu[1]\n\n        # Rotate points so ellipse is aligned with axes\n        phi *= -1\n        xct = xc * np.cos(phi) - yc * np.sin(phi)\n        yct = xc * np.sin(phi) + yc * np.cos(phi)\n\n        # normalize to a circle of radius 1\n        r = (xct / a) ** 2 + (yct / b) ** 2\n\n        idx = np.argwhere(r <= 1).flatten()  # non-outlier points\n\n        # if all outliers, break\n        if len(idx) < 3:\n            break\n\n        # if no outliers, break\n        if len(idx) == len(x):\n            break\n\n        x = x[idx]\n        y = y[idx]\n\n    if (len(x) < 3) or np.all(x == x[0]) or np.all(y == y[0]):\n        mu = cov = a = b = phi = None\n        return [mu, cov, a, b, phi]\n\n    return confidence_ellipse(x, y, var)\n\n\ndef ellipsoid_split_filter(meter, n_std=[1.4, 1.4]):\n    \"\"\"\n    Filters a set of points based on a robust confidence ellipse. The points are split into groups using robust ellipses computed\n    and then tested for intersection. This determines whether separate keys are needed for different seasons and day types.\n\n    Parameters:\n    meter (pandas.DataFrame): Dataframe containing the points to be filtered.\n    n_std (float or list): Standard deviation for outlier detection. Default is [1.4, 1.4].\n\n    Returns:\n    dict: A dictionary containing the filtered points for each season and day type.\n    \"\"\"\n\n    if isinstance(n_std, float):\n        var = np.ones([2, 2]) * n_std**2\n    else:\n        std = np.array(n_std)[:, None]\n        var = std.T * std\n\n    cluster_ellipse = {}\n    for season in [\"summer\", \"shoulder\", \"winter\"]:\n        for day_type, day_num in enumerate([[1, 2, 3, 4, 5], [6, 7]]):\n            if day_type == 0:\n                key = f\"wd-{season[:2]}\"\n            else:\n                key = f\"we-{season[:2]}\"\n\n            meter_season = meter[\n                (meter[\"season\"] == season) & (meter[\"observed\"].notna())\n            ]\n            meter_season = meter_season[meter_season[\"day_of_week\"].isin(day_num)]\n            meter_season = meter_season.sort_values(by=[\"temperature\"])\n\n            T = meter_season[\"temperature\"].values\n            obs = meter_season[\"observed\"].values\n\n            if (len(T) < 3) or (len(obs) < 3):\n                mu = cov = a = b = phi = None\n            else:\n                mu, cov, a, b, phi = robust_confidence_ellipse(\n                    T, obs, var, outlier_std=3.6, N=3\n                )\n                # mu, cov, a, b, phi = confidence_ellipse(T, obs, std_sqr)\n\n            cluster_ellipse[key] = {\"mu\": mu, \"cov\": cov, \"a\": a, \"b\": b, \"phi\": phi}\n\n    combos = {\n        \"summer\": [\n            [[\"wd-su\", \"wd-sh\"], [\"we-su\", \"we-sh\"]],\n            [[\"wd-su\", \"wd-wi\"], [\"we-su\", \"we-wi\"]],\n        ],\n        \"shoulder\": [\n            [[\"wd-su\", \"wd-sh\"], [\"we-su\", \"we-sh\"]],\n            [[\"wd-sh\", \"wd-wi\"], [\"we-sh\", \"we-wi\"]],\n        ],\n        \"winter\": [\n            [[\"wd-sh\", \"wd-wi\"], [\"we-sh\", \"we-wi\"]],\n            [[\"wd-su\", \"wd-wi\"], [\"we-su\", \"we-wi\"]],\n        ],\n        \"weekday_weekend\": [\n            [[\"wd-su\", \"we-su\"], [\"wd-sh\", \"we-sh\"], [\"wd-wi\", \"we-wi\"]]\n        ],\n    }\n\n    ellipse_overlap = {}\n    allow_separate = {\n        \"summer\": [False, False],\n        \"shoulder\": [False, False],\n        \"winter\": [False, False],\n        \"weekday_weekend\": [False],\n    }\n    for key in allow_separate.keys():\n        for i, season_wd_we in enumerate(combos[key]):\n            for combo in season_wd_we:\n                combo_str = \"__\".join(combo)\n\n                if combo_str not in ellipse_overlap:\n                    mu_A = cluster_ellipse[combo[0]][\"mu\"]\n                    cov_A = cluster_ellipse[combo[0]][\"cov\"]\n                    mu_B = cluster_ellipse[combo[1]][\"mu\"]\n                    cov_B = cluster_ellipse[combo[1]][\"cov\"]\n\n                    if all([coef is not None for coef in [mu_A, mu_B, cov_A, cov_B]]):\n                        ellipse_overlap[combo_str] = ellipsoid_intersection_test(\n                            mu_A, mu_B, cov_A, cov_B\n                        )\n                    else:\n                        ellipse_overlap[combo_str] = False\n\n                if not ellipse_overlap[combo_str]:\n                    allow_separate[key][i] = True\n                    break\n\n        allow_separate[key] = all(allow_separate[key])\n\n    return allow_separate\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/opt_settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nfrom __future__ import annotations\n\nimport pydantic\n\nfrom enum import Enum\nfrom typing import Optional\n\nfrom opendsm.common.base_settings import BaseSettings, CustomField\n\n\nclass AlgorithmChoice(str, Enum):\n    # SciPy scalar optimization algorithms\n    SCIPY_BRENT = \"scipy_brent\"\n    SCIPY_BOUNDED = \"scipy_bounded\"\n    SCIPY_GOLDEN = \"scipy_golden\"\n\n    # SciPy local optimization algorithms\n    SCIPY_NELDERMEAD = \"scipy_nelder-mead\"\n    SCIPY_L_BFGS_B = \"scipy_l-bfgs-b\"\n    SCIPY_TNC = \"scipy_tnc\"\n    SCIPY_COBYLA = \"scipy_cobyla\"\n    SCIPY_COBYQA = \"scipy_cobyqa\"\n    SCIPY_SLSQP = \"scipy_slsqp\"\n    SCIPY_POWELL = \"scipy_powell\"\n    SCIPY_TRUST_CONSTR = \"scipy_trust-constr\"\n\n    # SciPy global optimization algorithms\n    SCIPY_DIRECT = \"scipy_direct\"\n\n    # nlopt-based algorithms\n    NLOPT_DIRECT = \"nlopt_direct\"\n    NLOPT_DIRECT_NOSCAL = \"nlopt_direct_noscal\"\n    NLOPT_DIRECT_L = \"nlopt_direct_l\"\n    NLOPT_DIRECT_L_RAND = \"nlopt_direct_l_rand\"\n    NLOPT_DIRECT_L_NOSCAL = \"nlopt_direct_l_noscal\"\n    NLOPT_DIRECT_L_RAND_NOSCAL = \"nlopt_direct_l_rand_noscal\"\n    NLOPT_ORIG_DIRECT = \"nlopt_orig_direct\"\n    NLOPT_ORIG_DIRECT_L = \"nlopt_orig_direct_l\"\n    NLOPT_CRS2_LM = \"nlopt_crs2_lm\"\n    NLOPT_MLSL_LDS = \"nlopt_mlsl_lds\"\n    NLOPT_MLSL = \"nlopt_mlsl\"\n    NLOPT_STOGO = \"nlopt_stogo\"\n    NLOPT_STOGO_RAND = \"nlopt_stogo_rand\"\n    NLOPT_AGS = \"nlopt_ags\"\n    NLOPT_ISRES = \"nlopt_isres\"\n    NLOPT_ESCH = \"nlopt_esch\"\n    NLOPT_COBYLA = \"nlopt_cobyla\"\n    NLOPT_BOBYQA = \"nlopt_bobyqa\"\n    NLOPT_NEWUOA = \"nlopt_newuoa\"\n    NLOPT_NEWUOA_BOUND = \"nlopt_newuoa_bound\"\n    NLOPT_PRAXIS = \"nlopt_praxis\"\n    NLOPT_NELDERMEAD = \"nlopt_neldermead\"\n    NLOPT_SBPLX = \"nlopt_sbplx\"\n    NLOPT_MMA = \"nlopt_mma\"\n    NLOPT_CCSAQ = \"nlopt_ccsaq\"\n    NLOPT_SLSQP = \"nlopt_slsqp\"\n    NLOPT_L_BFGS = \"nlopt_lbfgs\"\n    NLOPT_TNEWTON = \"nlopt_tnewton\"\n    NLOPT_TNEWTON_PRECOND = \"nlopt_tnewton_precond\"\n    NLOPT_TNEWTON_RESTART = \"nlopt_tnewton_restart\"\n    NLOPT_TNEWTON_PRECOND_RESTART = \"nlopt_tnewton_precond_restart\"\n    NLOPT_VAR1 = \"nlopt_var1\"\n    NLOPT_VAR2 = \"nlopt_var2\"\n\n\nclass StopCriteriaChoice(str, Enum):\n    ITERATION_MAXIMUM = \"iteration maximum\"\n    MAXIMUM_TIME = \"maximum time [min]\"\n\n\nclass OptimizationSettings(BaseSettings):\n    algorithm: AlgorithmChoice = CustomField(\n        default=AlgorithmChoice.NLOPT_SBPLX,\n        description=\"Optimization algorithm choice\",\n    )\n\n    stop_criteria_type: StopCriteriaChoice = CustomField(\n        default=StopCriteriaChoice.ITERATION_MAXIMUM,\n        description=\"Stopping criteria\",\n    )\n\n    stop_criteria_value: float = CustomField(\n        default=2000,\n        gt=0,\n        description=\"Stopping criteria value for the optimization algorithm\",\n    )\n    \n    initial_step: Optional[float] = CustomField(\n        default=0.1,\n        description=\"Initial step size for the optimization algorithm\",\n    )\n\n    x_tol_rel: float = CustomField(\n        default=1e-5,\n        gt=0,\n        description=\"Relative cutoff X tolerance for the optimization algorithm\",\n    )\n\n    f_tol_rel: float = CustomField(\n        default=1e-5,\n        gt=0,\n        description=\"Relative cutoff function tolerance for the optimization algorithm\",\n    )\n\n    initial_population_multiplier: Optional[float] = CustomField(\n        default=None,\n        description=\"Initial population multiplier for the optimization algorithm\",\n    )\n\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_population_multiplier(self):\n        if self.initial_population_multiplier is None:\n            if self.algorithm == AlgorithmChoice.NLOPT_ISRES:\n                raise ValueError(\"INITIAL_POPULATION_MULTIPLIER must be > 1 for nlopt_ISRES\")\n        else:\n            if self.algorithm == AlgorithmChoice.NLOPT_ISRES:\n                if self.initial_population_multiplier <= 1:\n                    raise ValueError(\"INITIAL_POPULATION_MULTIPLIER must be > 1 for nlopt_ISRES\")\n            else:\n                raise ValueError(\"INITIAL_POPULATION_MULTIPLIER must be None for all algorithms except nlopt_ISRES\")\n\n        return self"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/selection_criteria.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\n\nimport numpy as np\n\n\ndef neg_log_likelihood(loss, N):\n    \"\"\"\n    This function calculates the negative log likelihood for least squares fitting.\n\n    Parameters:\n    loss (float): The sum of squared residuals.\n    N (int): The number of data points.\n\n    Returns:\n    float: The negative log likelihood of the least squares fit.\n    \"\"\"\n\n    # log likelihood for n independent identical normal distributions:\n    # log_likelihood = -n/2*np.log(2*np.pi) - n/2*np.log(sigma**2) - 1/(2*sigma**2)*np.sum((x - mu)**2)\n\n    if loss <= 0 or N <= 0:\n        return np.inf\n\n    # log likelihood for for least squares fitting:\n\n    res = -N / 2 * (np.log(2 * np.pi) + np.log(loss / N) + 1)\n\n    return res\n\n\ndef selection_criteria(\n    loss,\n    TSS,\n    N,\n    num_coeffs,\n    model_selection_criteria=\"bic\",\n    penalty_multiplier=1.0,\n    penalty_power=1.0,\n):\n    \"\"\"\n    This function calculates the selection criteria for a given model. There are different criteria that can be used,\n    and the default is the Bayesian information criterion (BIC).\n\n    Parameters:\n    loss (float): The loss of the model.\n    TSS (float): Total sum of squares.\n    N (int): The number of observations.\n    num_coeffs (int): The number of coefficients in the model.\n    model_selection_criteria (str): The model selection criteria to use. Default is \"bic\".\n    penalty_multiplier (float): The penalty multiplier. Default is 1.0.\n    penalty_power (float): The penalty power. Default is 1.0.\n\n    Returns:\n    float: The calculated selection criteria.\n\n    Raises:\n    NotImplementedError: If the model selection criteria is \"dic\", \"waic\", or \"wbic\", as these are not implemented.\n    \"\"\"\n\n    K = num_coeffs  # total number of coefficients\n    c0 = penalty_multiplier\n    d0 = penalty_power\n\n    df_penalized = N - K - 1\n    if df_penalized <= 0:\n        df_penalized = 1e-6\n\n    # Root-mean-square error adjusted\n    if model_selection_criteria.lower() == \"rmse\":\n        criteria = np.sqrt(loss / N)\n\n    # Root-mean-square error adjusted\n    elif model_selection_criteria.lower() == \"rmse_adj\":\n        criteria = np.sqrt(loss / df_penalized)\n\n    # 1 - R_squared (because we minimize)\n    elif model_selection_criteria.lower() == \"r_squared\":\n        r_squared = 1 - loss / TSS\n\n        criteria = (1 - r_squared) * 100\n\n    elif model_selection_criteria.lower() == \"r_squared_adj\":\n        r_squared = 1 - loss / TSS\n        r_squared_adj = 1 - (1 - r_squared) * ((N - 1) / df_penalized)\n        criteria = (1 - r_squared_adj) * 100\n\n    # Final prediction error\n    elif model_selection_criteria.lower() == \"fpe\":\n        criteria = loss * (N + K + 1) / df_penalized\n        # penalized_loss = np.exp(-2/N*log_likelihood)*(N + K)/(N - K)\n\n    # Akaike (ah-kah-ee-kay)\n    # Akaike information criterion - Akaike (1973, 1974, 1981)\n    elif model_selection_criteria.lower() == \"aic\":\n        criteria = -2 * neg_log_likelihood(loss, N) + c0 * 2 * K**d0\n\n    # Akaike information criterion corrected - Hurvich and Tsai (1989)\n    elif model_selection_criteria.lower() == \"aicc\":\n        criteria = (\n            -2 * neg_log_likelihood(loss, N)\n            + c0 * (2 * K + (2 * K * (K + 1) / df_penalized)) ** d0\n        )\n\n    # Consistent Akaike information criterion - Bozdogan (1987)\n    elif model_selection_criteria.lower() == \"caic\":\n        criteria = -2 * neg_log_likelihood(loss, N) + c0 * K * (np.log(N) + 1) ** d0\n\n    # Bayesian information criterion\n    elif model_selection_criteria.lower() == \"bic\":\n        # if c0 = 0.299 and d0 = 2.1, this is the same as Liu, We, Zidek\n        criteria = -2 * neg_log_likelihood(loss, N) + c0 * K * np.log(N) ** d0\n\n    # Sample-size adjusted Bayesian information criteria\n    elif model_selection_criteria.lower() == \"sabic\":\n        criteria = (\n            -2 * neg_log_likelihood(loss, N) + c0 * K * np.log((N + 2) / 24) ** d0\n        )\n\n    # Deviance information criterion\n    elif model_selection_criteria.lower() == \"dic\":\n        raise NotImplementedError(\n            \"DIC has not been implmented as a model selection criterion\"\n        )\n\n    # Widely applicable (or Watanabe-Akaike) information criterion\n    elif model_selection_criteria.lower() == \"waic\":\n        raise NotImplementedError(\n            \"WAIC has not been implmented as a model selection criterion\"\n        )\n\n    # Widely applicable (or Watanabe) Bayesian information criterion\n    elif model_selection_criteria.lower() == \"wbic\":\n        raise NotImplementedError(\n            \"WBIC has not been implmented as a model selection criterion\"\n        )\n\n    if model_selection_criteria.lower() not in [\"rmse\", \"rmse_adj\"]:\n        criteria /= N  # Normalize to number of datapoints\n\n    return criteria\n"
  },
  {
    "path": "opendsm/eemeter/models/daily/utilities/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nfrom __future__ import annotations\n\nimport pydantic\n\nfrom enum import Enum\nfrom typing import Optional, Literal, Union\n\nfrom opendsm.common.base_settings import BaseSettings, CustomField\nimport opendsm.eemeter.models.daily.utilities.const as _const\nfrom opendsm.eemeter.models.daily.utilities.opt_settings import AlgorithmChoice\n\n\n# region option definitions\nclass AlphaFinalType(str, Enum):\n    ALL = \"all\"\n    LAST = \"last\"\n\n\nclass ModelSelectionCriteria(str, Enum):\n    RMSE = \"rmse\"\n    RMSE_ADJ = \"rmse_adj\"\n    R_SQUARED = \"r_squared\"\n    R_SQUARED_ADJ = \"r_squared_adj\"\n    AIC = \"aic\"\n    AICC = \"aicc\"\n    CAIC = \"caic\"\n    BIC = \"bic\"\n    SABIC = \"sabic\"\n    FPE = \"fpe\"\n\n    # Maybe these will be implemented one day\n    # DIC = \"dic\"\n    # WAIC = \"waic\"\n    # WBIC = \"wbic\"\n\n\nclass FullModelSelection(str, Enum):\n    HDD_TIDD_CDD = \"hdd_tidd_cdd\"\n    C_HDD_TIDD = \"c_hdd_tidd\"\n    TIDD = \"tidd\"\n\n# endregion\n\n\nclass Season_Definition(BaseSettings):\n    january: str = CustomField(default=\"winter\")\n    february: str = CustomField(default=\"winter\")\n    march: str = CustomField(default=\"shoulder\")\n    april: str = CustomField(default=\"shoulder\")\n    may: str = CustomField(default=\"shoulder\")\n    june: str = CustomField(default=\"summer\")\n    july: str = CustomField(default=\"summer\")\n    august: str = CustomField(default=\"summer\")\n    september: str = CustomField(default=\"summer\")\n    october: str = CustomField(default=\"shoulder\")\n    november: str = CustomField(default=\"winter\")\n    december: str = CustomField(default=\"winter\")\n\n    options: list[str] = CustomField(default=[\"summer\", \"shoulder\", \"winter\"])\n\n    \"\"\"Set dictionaries of seasons\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def set_numeric_dict(self) -> Season_Definition:\n        season_dict = {}\n        for month, num in _const.season_num.items():\n            val = getattr(self, month.lower())\n            if val not in self.options:\n                raise ValueError(f\"SeasonDefinition: {val} is not a valid option. Valid options are {self.options}\")\n            \n            season_dict[num] = val\n        \n        self._month_index = _const.season_num\n        self._num_dict = season_dict\n        self._order = {val: i for i, val in enumerate(self.options)}\n\n        return self\n\n\nclass Weekday_Weekend_Definition(BaseSettings):\n    monday: str = CustomField(default=\"weekday\")\n    tuesday: str = CustomField(default=\"weekday\")\n    wednesday: str = CustomField(default=\"weekday\")\n    thursday: str = CustomField(default=\"weekday\")\n    friday: str = CustomField(default=\"weekday\")\n    saturday: str = CustomField(default=\"weekend\")\n    sunday: str = CustomField(default=\"weekend\")\n\n    options: list[str] = CustomField(default=[\"weekday\", \"weekend\"])\n\n    \"\"\"Set dictionaries of weekday/weekend\"\"\"\n    @pydantic.model_validator(mode=\"after\")\n    def set_numeric_dict(self) -> Weekday_Weekend_Definition:\n        weekday_dict = {}\n        for day, num in _const.weekday_num.items():\n            val = getattr(self, day.lower())\n            if val not in self.options:\n                raise ValueError(f\"WeekdayWeekendDefinition: {val} is not a valid option. Valid options are {self.options}\")\n            \n            weekday_dict[num] = val\n        \n        self._day_index = _const.weekday_num\n        self._num_dict = weekday_dict\n        self._order = {val: i for i, val in enumerate(self.options)}\n\n        return self\n    \n\nclass Split_Selection_Definition(BaseSettings):\n    criteria: ModelSelectionCriteria = CustomField(\n        default=ModelSelectionCriteria.BIC,\n        developer=True,\n        description=\"What selection criteria is used to select data splits of models\",\n    )\n\n    penalty_multiplier: float = CustomField(\n        default=0.24,\n        ge=0,\n        developer=True,\n        description=\"Penalty multiplier for split selection criteria\",\n    )\n\n    penalty_power: float = CustomField(\n        default=2.061,\n        ge=1,\n        developer=True,\n        description=\"What power should the penalty of the selection criteria be raised to\",\n    )\n\n    allow_separate_summer: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Allow summer to be modeled separately\",\n    )\n\n    allow_separate_shoulder: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Allow shoulder to be modeled separately\",\n    )\n\n    allow_separate_winter: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Allow winter to be modeled separately\",\n    )\n\n    allow_separate_weekday_weekend: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Allow weekdays and weekends to be modeled separately\",\n    )\n\n    reduce_splits_by_gaussian: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Reduces splits by fitting with multivariate Gaussians and testing for overlap\",\n    )\n\n    reduce_splits_num_std: Optional[list[float]] = CustomField(\n        default=[1.4, 0.89],\n        developer=True,\n        description=\"Number of standard deviations to use with Gaussians\",\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_reduce_splits_num_std(self):\n        if self.reduce_splits_num_std is not None:\n            if len(self.reduce_splits_num_std) != 2:\n                raise ValueError(\"`REDUCE_SPLITS_NUM_STD` must be a list of length 2\")\n            \n            if self.reduce_splits_num_std[0] <= 0 or self.reduce_splits_num_std[1] <= 0:\n                raise ValueError(\"`REDUCE_SPLITS_NUM_STD` entries must be > 0\")\n            \n        return self\n\n\ndef _check_developer_mode(cls):  \n    for k, v in type(cls).model_fields.items():\n        if isinstance(getattr(cls, k), BaseSettings):\n            _check_developer_mode(getattr(cls, k))\n\n        elif v.json_schema_extra[\"developer\"] and getattr(cls, k) != v.default:\n            raise ValueError(f\"Developer mode is not enabled. Cannot change {k} from default value.\")\n\n    return cls\n\n\nclass DailySettings(BaseSettings):\n    \"\"\"Settings for creating the daily model.\n\n    These settings should be converted to a dictionary before being passed to the DailyModel class.\n    Be advised that any changes to the default settings deviates from OpenEEmeter standard methods and should be used with caution.\n\n    Attributes:\n        developer_mode (bool): Allows changing of developer settings\n        algorithm_choice (str): Optimization algorithm choice. Developer mode only.\n        initial_guess_algorithm_choice (str): Initial guess optimization algorithm choice. Developer mode only.\n        full_model (str): The largest model allowed. Developer mode only.\n        smoothed_model (bool): Allow smoothed models.\n        allow_separate_summer (bool): Allow summer to be modeled separately.\n        allow_separate_shoulder (bool): Allow shoulder to be modeled separately.\n        allow_separate_winter (bool): Allow winter to be modeled separately.\n        allow_separate_weekday_weekend (bool): Allow weekdays and weekends to be modeled separately.\n        reduce_splits_by_gaussian (bool): Reduces splits by fitting with multivariate Gaussians and testing for overlap.\n        reduce_splits_num_std (list[float]): Number of standard deviations to use with Gaussians.\n        alpha_minimum (float): Alpha where adaptive robust loss function is Welsch loss.\n        alpha_selection (float): Specified alpha to evaluate which is the best model type.\n        alpha_final_type (str): When to use 'alpha_final: 'all': on every model, 'last': on final model, 'None': don't use.\n        alpha_final (float | str | None): Specified alpha or 'adaptive' for adaptive loss in model evaluation.\n        final_bounds_scalar (float | None): Scalar for calculating bounds of 'alpha_final'.\n        regularization_alpha (float): Alpha for elastic net regularization.\n        regularization_percent_lasso (float): Percent lasso vs (1 - perc) ridge regularization.\n        segment_minimum_count (int): Minimum number of data points for HDD/CDD.\n        maximum_slope_OoM_scalar (float): Scaler for initial slope to calculate bounds based on order of magnitude.\n        initial_smoothing_parameter (float | None): Initial guess for the smoothing parameter.\n        initial_step_percentage (float | None): Initial step-size for relevant algorithms.\n        split_selection_criteria (str): What selection criteria is used to select data splits of models.\n        split_selection_penalty_multiplier (float): Penalty multiplier for split selection criteria.\n        split_selection_penalty_power (float): What power should the penalty of the selection criteria be raised to.\n        season (Dict[int, str]): Dictionary of months and their associated season (January is 1).\n        is_weekday (Dict[int, bool]): Dictionary of days (1 = Monday) and if that day is a weekday (True/False).\n        uncertainty_alpha (float): Significance level used for uncertainty calculations (0 < float < 1).\n        cvrmse_threshold (float): Threshold for the CVRMSE to disqualify a model.\n\n    \"\"\"\n\n    developer_mode: bool = CustomField(\n        default=False,\n        developer=False,\n        description=\"Developer mode flag\",\n    )\n\n    silent_developer_mode: bool = CustomField(\n        default=False,\n        developer=False,\n        exclude=True,\n        repr=False,\n    )\n\n    algorithm_choice: Optional[AlgorithmChoice] = CustomField(\n        default=AlgorithmChoice.NLOPT_SBPLX,\n        developer=True,\n        description=\"Optimization algorithm choice\",\n    )\n\n    initial_guess_algorithm_choice: Optional[AlgorithmChoice] = CustomField(\n        default=AlgorithmChoice.NLOPT_DIRECT,\n        developer=True,\n        description=\"Initial guess optimization algorithm choice\",\n    )\n\n    full_model: Optional[FullModelSelection] = CustomField(\n        default=FullModelSelection.HDD_TIDD_CDD,\n        developer=True,\n        description=\"The largest model allowed\",\n    )\n\n    allow_smooth_model: bool = CustomField(\n        default=True,\n        developer=True,\n        description=\"Allow smoothed models\",\n    )\n\n    alpha_minimum: float = CustomField(\n        default=-100,\n        le=-10,\n        developer=True,\n        description=\"Alpha where adaptive robust loss function is Welsch loss\",\n    )\n\n    alpha_selection: float = CustomField(\n        default=2,\n        ge=-10,\n        le=2,\n        developer=True,\n        description=\"Specified alpha to evaluate which is the best model type\",\n    )\n\n    alpha_final_type: Optional[AlphaFinalType] = CustomField(\n        default=AlphaFinalType.LAST,\n        developer=True,\n        description=\"When to use 'alpha_final: 'all': on every model, 'last': on final model, 'None': don't use\",\n    )\n\n    alpha_final: Optional[Union[float, Literal[\"adaptive\"]]] = CustomField(\n        default=\"adaptive\",\n        developer=True,\n        description=\"Specified alpha or 'adaptive' for adaptive loss in model evaluation\",\n    )\n\n    final_bounds_scalar: Optional[float] = CustomField(\n        default=1,\n        developer=True,\n        description=\"Scalar for calculating bounds of 'alpha_final'\",\n    )\n\n    regularization_alpha: float = CustomField(\n        default=0.001,\n        ge=0,\n        developer=True,\n        description=\"Alpha for elastic net regularization\",\n    )\n\n    regularization_percent_lasso: float = CustomField(\n        default=1,\n        ge=0,\n        le=1,\n        developer=True,\n        description=\"Percent lasso vs (1 - perc) ridge regularization\",\n    )\n\n    segment_minimum_count: int = CustomField(\n        default=6,\n        ge=3,\n        developer=True,\n        description=\"Minimum number of data points for HDD/CDD\",\n    )\n\n    maximum_slope_oom_scalar: float = CustomField(\n        default=2,\n        ge=1,\n        developer=True,\n        description=\"Scaler for initial slope to calculate bounds based on order of magnitude\",\n    )\n\n    initial_step_percentage: Optional[float] = CustomField(\n        default=0.1,\n        developer=True,\n        description=\"Initial step-size for relevant algorithms\",\n    )\n\n    split_selection: Split_Selection_Definition = CustomField(\n        default_factory=Split_Selection_Definition,\n        developer=True,\n        description=\"Settings for split selection\",\n    )\n\n    season: Season_Definition = CustomField(\n        default_factory=Season_Definition,\n        developer=False,\n        description=\"Dictionary of months and their associated season (January is 1)\",\n    )\n\n    weekday_weekend: Weekday_Weekend_Definition = CustomField(\n        default_factory=Weekday_Weekend_Definition,\n        developer=False,\n        description=\"Dictionary of days (1 = Monday) and if that day is a weekday (True/False)\",\n    )\n\n    uncertainty_alpha: float = CustomField(\n        default=0.1,\n        ge=0,\n        le=1,\n        developer=False,\n        description=\"Significance level used for uncertainty calculations\",\n    )\n\n    cvrmse_threshold: float = CustomField(\n        default=1,\n        ge=0,\n        developer=True,\n        description=\"Threshold for the CVRMSE to disqualify a model\",\n    )\n\n    pnrmse_threshold: float = CustomField(\n        default=1.6,\n        ge=0,\n        developer=True,\n        description=\"Threshold for the PNRMSE to disqualify a model\",\n    )\n\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_developer_mode(self):\n        if self.developer_mode:\n            if not self.silent_developer_mode:\n                print(\"Warning: Daily model is nonstandard and should be explicitly stated in any derived work\")\n\n            return self\n        \n        _check_developer_mode(self)\n\n        return self\n\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_alpha_final(self):\n        if self.alpha_final is None:\n            if self.alpha_final_type != None:\n                raise ValueError(\"`ALPHA_FINAL` must be set if `ALPHA_FINAL_TYPE` is not None\")\n            \n        elif isinstance(self.alpha_final, float):\n            if (self.alpha_minimum > self.alpha_final) or (self.alpha_final > 2.0):\n                raise ValueError(\n                    f\"`ALPHA_FINAL` must be `adaptive` or `ALPHA_MINIMUM` <= float <= 2\"\n                )\n\n        elif isinstance(self.alpha_final, str):\n            if self.alpha_final != \"adaptive\":\n                raise ValueError(\n                    f\"ALPHA_FINAL must be `adaptive` or `ALPHA_MINIMUM` <= float <= 2\"\n            )\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_final_bounds_scalar(self):\n        if self.final_bounds_scalar is not None:\n            if self.final_bounds_scalar <= 0:\n                raise ValueError(\"`FINAL_BOUNDS_SCALAR` must be > 0\")\n            \n            if self.alpha_final_type is None:\n                raise ValueError(\"`FINAL_BOUNDS_SCALAR` must be None if `ALPHA_FINAL` is None\")\n            \n        else:\n            if self.alpha_final_type is not None:\n                raise ValueError(\"`FINAL_BOUNDS_SCALAR` must be > 0 if `ALPHA_FINAL` is not None\")\n\n        return self\n\n    \n    @pydantic.model_validator(mode=\"after\")\n    def _check_initial_step_percentage(self):\n        if self.initial_step_percentage is not None:\n            if self.initial_step_percentage <= 0 or self.initial_step_percentage > 0.5:\n                raise ValueError(\"`INITIAL_STEP_PERCENTAGE` must be None or 0 < float <= 0.5\")\n            \n        else:\n            if self.algorithm_choice[:5] in [\"nlopt\"]:\n                raise ValueError(\"`INITIAL_STEP_PERCENTAGE` must be specified if `ALGORITHM_CHOICE` is from Nlopt\")\n            \n        return self\n            \n    \n    def __repr__(self):\n        text_all = []\n        text_all.append(type(self).__name__)\n\n        # get all keys\n        keys = list(type(self).model_fields.keys())\n\n        # print away\n        key_max = max([len(k) for k in keys]) + 2\n        for key in keys:\n            if not type(self).model_fields[key].repr:\n                continue\n\n            val = getattr(self, key)\n\n            if isinstance(val, dict):\n                v_max = max([len(str(v)) for v in list(val.values())])\n                k_max = max([len(str(k)) for k in list(val.keys())])\n                if k_max == 1:\n                    k_max = 2\n\n                for n, (k, v) in enumerate(val.items()):\n                    if n == 0:\n                        text_all.append(f\"{key:>{key_max}s}: {str(k):>{k_max}s}: {v}\")\n\n                    elif n < len(val) - 1:\n                        text_all.append(f\"{'':>{key_max}s}   {str(k):>{k_max}s}: {v}\")\n\n                    else:\n                        text_all.append(\n                            f\"{'':>{key_max}s}   {str(k):>{k_max}s}: {str(v):{v_max}s}\"\n                        )\n\n            else:\n                if isinstance(val, str):\n                    val = f\"'{val}'\"\n\n                text_all.append(f\"{key:>{key_max}s}: {val}\")\n\n        return \"\\n\".join(text_all)\n    \n\nclass Split_Selection_Legacy_Definition(Split_Selection_Definition):\n    allow_separate_summer: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Allow summer to be modeled separately\",\n    )\n\n    allow_separate_shoulder: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Allow shoulder to be modeled separately\",\n    )\n\n    allow_separate_winter: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Allow winter to be modeled separately\",\n    )\n\n    allow_separate_weekday_weekend: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Allow weekdays and weekends to be modeled separately\",\n    )\n\n    reduce_splits_by_gaussian: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Reduces splits by fitting with multivariate Gaussians and testing for overlap\",\n    )\n\n    reduce_splits_num_std: Optional[list[float]] = CustomField(\n        default=None,\n        developer=True,\n        description=\"Number of standard deviations to use with Gaussians\",\n    )\n\n\nclass DailyLegacySettings(DailySettings):\n    allow_smooth_model: bool = CustomField(\n        default=False,\n        developer=True,\n        description=\"Allow smoothed models\",\n    )\n\n    alpha_final: Optional[Union[float, Literal[\"adaptive\"]]] = CustomField(\n        default=2.0,\n        developer=True,\n        description=\"Specified alpha or 'adaptive' for adaptive loss in model evaluation\",\n    )\n\n    segment_minimum_count: int = CustomField(\n        default=10,\n        ge=3,\n        developer=True,\n        description=\"Minimum number of data points for HDD/CDD\",\n    )\n\n    split_selection: Split_Selection_Legacy_Definition = CustomField(\n        default_factory=Split_Selection_Legacy_Definition,\n        developer=True,\n        description=\"Settings for split selection\",\n    )\n\n\ndef update_daily_settings(settings, update_dict):\n    if not isinstance(settings, DailySettings):\n        raise TypeError(\"settings must be an instance of 'Daily_Settings'\")\n\n    # update settings with update_dict\n    settings_dict = settings.model_dump()\n    settings_dict.update(update_dict)\n\n    if isinstance(settings, DailyLegacySettings):\n        return DailyLegacySettings(**settings_dict)\n     \n    return DailySettings(**settings_dict)\n\n\n# TODO: deprecate\ndef default_settings(**kwargs) -> DailySettings:\n    \"\"\"\n    Returns default settings.\n    \"\"\"\n    return DailySettings(**kwargs)\n\n# TODO: deprecate\ndef caltrack_legacy_settings(**kwargs) -> DailyLegacySettings:\n    \"\"\"\n    Returns CalTRACK legacy settings.\n    \"\"\"\n    return DailyLegacySettings(**kwargs)"
  },
  {
    "path": "opendsm/eemeter/models/hourly/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .settings import (\n    HourlyNonSolarSettings,\n    HourlySolarSettings,\n)\nfrom .data import HourlyBaselineData, HourlyReportingData\nfrom .model import HourlyModel\n\n__all__ = (\n    \"HourlyNonSolarSettings\",\n    \"HourlySolarSettings\",\n    \"HourlyBaselineData\",\n    \"HourlyReportingData\",\n    \"HourlyModel\",\n)\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nfrom pathlib import Path\nimport copy\nfrom datetime import date\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.common.data_processor_utilities import (\n    remove_duplicates,\n)\nfrom opendsm.common.hourly_interpolation import interpolate\nfrom opendsm.eemeter.common.data_settings import HourlyDataSettings\nfrom opendsm.eemeter.common.sufficiency_criteria import HourlySufficiencyCriteria\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\n\nclass NREL_Weather_API:\n    api_key = (\n        \"---\"  # get your own key from https://developer.nrel.gov/signup/  #Required\n    )\n    name = \"---\"  # required\n    email = \"---\"  # required\n    interval = \"60\"  # required\n\n    attributes = \"ghi,dhi,dni,wind_speed,air_temperature,cloud_type,dew_point,clearsky_dhi,clearsky_dni,clearsky_ghi\"  # not required\n    leap_year = \"false\"  # not required\n    utc = \"false\"  # not required\n    reason_for_use = \"---\"  # not required\n    your_affiliation = \"---\"  # not required\n    mailing_list = \"false\"  # not required\n\n    # cache = Path(\"/app/.recurve_cache/data/MCE/MCE_weather_stations\")\n    cache = Path(\"/app/.recurve_cache/data/MCE/Weather_stations\")\n\n    use_cache = True\n\n    round_minutes_method = \"floor\"  # [None, floor, ceil, round]\n\n    def __init__(self, **kwargs):\n        self.__dict__.update(kwargs)\n        self.cache.mkdir(parents=True, exist_ok=True)\n\n    def get_data(self, lat, lon, years=[2017, 2021]):\n        data_path = self.cache / f\"{lat}_{lon}.pkl\"\n        if data_path.exists() and self.use_cache:\n            df = pd.read_pickle(data_path)\n\n        else:\n            years = list(range(min(years), max(years) + 1))\n\n            df = self.query_API(lat, lon, years)\n\n            df.columns = [x.lower().replace(\" \", \"_\") for x in df.columns]\n\n            if self.round_minutes_method == \"floor\":\n                df[\"datetime\"] = df[\"datetime\"].dt.floor(\"h\")\n            elif self.round_minutes_method == \"ceil\":\n                df[\"datetime\"] = df[\"datetime\"].dt.ceil(\"h\")\n            elif self.round_minutes_method == \"round\":\n                df[\"datetime\"] = df[\"datetime\"].dt.round(\"h\")\n\n            df = df.set_index(\"datetime\")\n\n            if self.use_cache:\n                df.to_pickle(data_path)\n\n        return df\n\n    def query_API(self, lat, lon, years):\n        leap_year = self.leap_year\n        interval = self.interval\n        utc = self.utc\n        api_key = self.api_key\n        name = self.name\n        email = self.email\n\n        year_df = []\n        for year in years:\n            year = str(year)\n\n            url = self._generate_url(\n                lat, lon, year, leap_year, interval, utc, api_key, name, email\n            )\n            df = pd.read_csv(url, skiprows=2)\n\n            # Set the time index in the pandas dataframe:\n            # set datetime using the year, month, day, and hour\n            df[\"datetime\"] = pd.to_datetime(\n                df[[\"Year\", \"Month\", \"Day\", \"Hour\", \"Minute\"]]\n            )\n\n            df = df.drop(columns=[\"Year\", \"Month\", \"Day\", \"Hour\", \"Minute\"])\n            df = df.dropna()\n\n            year_df.append(df)\n\n        # merge the dataframes for different years\n        df = pd.concat(year_df, axis=0)\n\n        return df\n\n    def _generate_url(\n        self, lat, lon, year, leap_year, interval, utc, api_key, name, email\n    ):\n        query = f\"?wkt=POINT({lon}%20{lat})&names={year}&interval={interval}&api_key={api_key}&full_name={name}&email={email}&utc={utc}\"\n\n        if year == \"2021\":\n            # details: https://developer.nrel.gov/docs/solar/nsrdb/psm3-2-2-download/\n            url = f\"https://developer.nrel.gov/api/nsrdb/v2/solar/psm3-2-2-download.csv{query}\"\n\n        elif year in [str(i) for i in range(1998, 2021)]:\n            # details: https://developer.nrel.gov/docs/solar/nsrdb/psm3-download/\n            url = f\"https://developer.nrel.gov/api/nsrdb/v2/solar/psm3-download.csv{query}\"\n\n        else:\n            print(\"Year must be between 1998 and 2021\")\n            url = None\n\n        return url\n\n\nclass _HourlyData:\n    \"\"\"Private base class for hourly baseline and reporting data\n\n    Will raise exception during data sufficiency check if instantiated\n    \"\"\"\n\n    _settings_class = HourlyDataSettings\n\n    def __init__(\n        self,\n        df: pd.DataFrame,\n        is_electricity_data: bool,\n        pv_start: date | str | None = None,\n        settings: dict | None = None,\n        **kwargs: dict,\n    ):\n        self._df = None\n        self.is_electricity_data = is_electricity_data\n        self.tz = None\n\n        self.warnings = []\n        self.disqualification = []\n\n        # TODO copied from HourlyData\n        self._to_be_interpolated_columns = []\n        self._outputs = []\n\n        self.pv_start = None\n        if pv_start is not None:\n            self.pv_start = pd.to_datetime(pv_start).date()\n\n        # Initialize settings\n        if settings is None:\n            self.settings = HourlyDataSettings()\n        elif isinstance(settings, dict):\n            self.settings = HourlyDataSettings(**settings)\n\n        self.settings.is_electricity_data = is_electricity_data\n\n        # TODO not sure why we're keeping this copy\n        self._kwargs = copy.deepcopy(kwargs)\n        if \"outputs\" in self._kwargs:\n            self._outputs = copy.deepcopy(self._kwargs[\"outputs\"])\n        else:\n            self._outputs = [\"temperature\", \"observed\"]\n\n        self._df = self._set_data(df)\n        disqualification, warnings = self._check_data_sufficiency()\n\n        self.disqualification += disqualification\n        self.warnings += warnings\n        self.log_warnings()\n\n    @property\n    def df(self):\n        \"\"\"Get the corrected input data stored in the class. The actual dataframe is immutable, this returns a copy.\"\"\"\n        if self._df is None:\n            return None\n        else:\n            return self._df.copy()\n\n    def log_warnings(self):\n        \"\"\"\n        Logs the warnings and disqualifications associated with the data.\n        \"\"\"\n        for warning in self.warnings + self.disqualification:\n            warning.warn()\n\n    def _get_contiguous_datetime(self, df):\n        # get earliest datetime and latest datetime\n        # make earliest start at 0 and latest end at 23, this ensures full days\n        earliest_datetime = df.index.min().replace(\n            hour=0, minute=0, second=0, microsecond=0\n        )\n        latest_datetime = df.index.max().replace(\n            hour=23, minute=0, second=0, microsecond=0\n        )\n\n        # create a new index with all the hours between the earliest and latest datetime\n        complete_dt = pd.date_range(\n            start=earliest_datetime, end=latest_datetime, freq=\"h\"\n        )\n\n        # merge meter data with complete_dt\n        df = df.reindex(complete_dt)\n\n        df[\"date\"] = df.index.date\n        df[\"hour_of_day\"] = df.index.hour\n\n        return df\n\n    def _interpolate(self, df):\n        # make column of interpolated boolean if any observed or temperature is nan\n        # check if in each row of the columns in output has nan values, the interpolated column will be true\n        if \"to_be_interpolated_columns\" in self._kwargs:\n            self._to_be_interpolated_columns = self._kwargs[\n                \"to_be_interpolated_columns\"\n            ].copy()\n            self._outputs += [\n                f\"{col}\"\n                for col in self._to_be_interpolated_columns\n                if col not in self._outputs\n            ]\n        else:\n            self._to_be_interpolated_columns = [\"temperature\", \"observed\"]\n            if \"ghi\" in df.columns:\n                self._to_be_interpolated_columns.append(\"ghi\")\n\n        for col in self._to_be_interpolated_columns:\n            if f\"interpolated_{col}\" in df.columns:\n                continue\n            self._outputs += [f\"interpolated_{col}\"]\n\n        # we can add kwargs to the interpolation class like: inter_kwargs = {\"n_cor_idx\": self.kwargs[\"n_cor_idx\"]}\n        df = interpolate(df, columns=self._to_be_interpolated_columns)\n\n        return df\n\n    def _add_pv_start_date(self, df, model_type=\"TS\"):\n        if self.pv_start is None:\n            self.pv_start = df.index.date.min()\n\n        if \"ts\" in model_type.lower() or \"time\" in model_type.lower():\n            df[\"has_pv\"] = 0\n            df.loc[df[\"date\"] >= self.pv_start, \"has_pv\"] = 1\n\n        else:\n            df[\"has_pv\"] = False\n            df.loc[df[\"date\"] >= self.pv_start, \"has_pv\"] = True\n        return df\n\n    def _merge_meter_temp(self, meter, temp):\n        df = meter.merge(\n            temp, left_index=True, right_index=True, how=\"left\"\n        ).tz_convert(meter.index.tz)\n        return df\n\n    def _check_data_sufficiency(self):\n        raise NotImplementedError(\n            \"Can't instantiate class _HourlyData, use HourlyBaselineData or HourlyReportingData.\"\n        )\n\n    def _set_data(self, data: pd.DataFrame):\n        df = data.copy()\n        expected_columns = [\n            \"observed\",\n            \"temperature\",\n            # \"ghi\",\n        ]\n        if not set(expected_columns).issubset(set(df.columns)):\n            # show the columns that are missing\n            raise ValueError(\n                \"Data is missing required columns: {}\".format(\n                    set(expected_columns) - set(df.columns)\n                )\n            )\n\n        # Check that the datetime index is timezone aware timestamp\n        if not isinstance(df.index, pd.DatetimeIndex) and \"datetime\" not in df.columns:\n            raise ValueError(\"Index is not datetime and datetime not provided\")\n\n        elif \"datetime\" in df.columns:\n            if df[\"datetime\"].dt.tz is None:\n                raise ValueError(\"Datatime is missing timezone information\")\n            df[\"datetime\"] = pd.to_datetime(df[\"datetime\"])\n            df.set_index(\"datetime\", inplace=True)\n\n        elif df.index.tz is None:\n            raise ValueError(\"Datatime is missing timezone information\")\n        elif str(df.index.tz) == \"UTC\":\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.data_quality.utc_index\",\n                    description=(\n                        \"Datetime index is in UTC. Use tz_localize() with the local timezone to ensure correct aggregations\"\n                    ),\n                    data={},\n                )\n            )\n        self.tz = df.index.tz\n        self.settings.time_zone = self.tz\n\n        # prevent later issues when merging on generated datetimes, which default to ns precision\n        # there is almost certainly a smoother way to accomplish this conversion, but this works\n        if df.index.dtype.unit != \"ns\":\n            utc_index = df.index.tz_convert(\"UTC\")\n            ns_index = utc_index.astype(\"datetime64[ns, UTC]\")\n            df.index = ns_index.tz_convert(self.tz)\n\n        # Convert electricity data having 0 meter values to NaNs\n        if self.is_electricity_data:\n            df.loc[df[\"observed\"] == 0, \"observed\"] = np.nan\n\n        # Caltrack 2.3.2 - Drop duplicates\n        df = remove_duplicates(df)\n\n        df = self._get_contiguous_datetime(df)\n        df = self._interpolate(df)\n        df = self._add_pv_start_date(df)\n\n        return df\n\n\nclass HourlyBaselineData(_HourlyData):\n    \"\"\"Data class to represent Hourly Baseline Data.\n\n    Only baseline data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            Optionally, column 'ghi' can be included in order to fit on solar data.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of issues with the data, but none that will severely reduce the quality of the model built.\n        pv_start (datetime.date): Solar install date. If left unset, assumed to be at beginning of data.\n\n    \"\"\"\n\n    def _check_data_sufficiency(self):\n        data = _create_sufficiency_df(self.df)\n        hsc = HourlySufficiencyCriteria(\n            data=data, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=False,\n            settings=self.settings.sufficiency,\n        )\n        hsc.check_sufficiency_baseline()\n        disqualification = hsc.disqualification\n        warnings = hsc.warnings\n\n        return disqualification, warnings\n\n\nclass HourlyReportingData(_HourlyData):\n    \"\"\"Data class to represent Hourly Reporting Data.\n\n    Only reporting data should go into the dataframe input, no blackout data should be input.\n    Checks sufficiency for the data provided as input depending on OpenEEMeter specifications and populates disqualifications and warnings based on it.\n\n    Meter data input is optional for the reporting class.\n\n    Args:\n        df (DataFrame): A dataframe having a datetime index or a datetime column with the timezone also being set.\n            It also requires 2 more columns - 'observed' for meter data, and 'temperature' for temperature data.\n            If GHI was provided during the baseline period, it should also be supplied for the reporting period with column name 'ghi'.\n            The temperature column should have values in Fahrenheit. Please convert your temperatures accordingly.\n\n        is_electricity_data (bool): Flag to ascertain if this is electricity data or not. Electricity data values of 0 are set to NaN.\n\n    Attributes:\n        df (DataFrame): Immutable dataframe that contains the meter and temperature values for the baseline data period.\n        disqualification (list[EEMeterWarning]): A list of serious issues with the data that can degrade the quality of the model. If you want to go ahead with building the model while ignoring them, set the ignore_disqualification = True flag in the model. By default disqualifications are not ignored.\n        warnings (list[EEMeterWarning]): A list of issues with the data, but none that will severely reduce the quality of the model built.\n        pv_start (datetime.date): Solar install date. If left unset, assumed to be at beginning of data.\n    \"\"\"\n\n    def __init__(\n        self,\n        df: pd.DataFrame,\n        is_electricity_data: bool,\n        pv_start: date | str | None = None,\n        settings: dict | None = None,\n        **kwargs: dict,\n    ):\n        df = df.copy()\n        if \"observed\" not in df.columns:\n            df[\"observed\"] = np.nan\n\n        super().__init__(df, is_electricity_data, pv_start, settings, **kwargs)\n\n    def _check_data_sufficiency(self):\n        data = _create_sufficiency_df(self.df)\n        hsc = HourlySufficiencyCriteria(\n            data=data, \n            is_electricity_data=self.is_electricity_data,\n            is_reporting_data=True,\n            settings=self.settings.sufficiency,\n        )\n        hsc.check_sufficiency_reporting()\n        disqualification = hsc.disqualification\n        warnings = hsc.warnings\n\n        return disqualification, warnings\n\n\ndef _create_sufficiency_df(df: pd.DataFrame):\n    \"\"\"Creates dataframe equivalent to legacy hourly input\"\"\"\n    df.loc[df[\"interpolated_observed\"] == 1, \"observed\"] = np.nan\n    df.loc[df[\"interpolated_temperature\"] == 1, \"temperature\"] = np.nan\n    if \"ghi\" in df.columns:\n        df.loc[df[\"interpolated_ghi\"] == 1, \"ghi\"] = np.nan\n    # set temperature_not_null  to 1.0 if temperature is not null\n    df[\"temperature_not_null\"] = df[\"temperature\"].notnull().astype(float)\n    df[\"temperature_null\"] = df[\"temperature\"].isnull().astype(float)\n    return df\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport os\nimport warnings\n\nos.environ[\"OMP_NUM_THREADS\"] = \"1\"\nos.environ[\"MKL_NUM_THREADS\"] = \"1\"\nos.environ[\"OPENBLAS_NUM_THREADS\"] = \"1\"\n\nimport re\n\nfrom pydantic import BaseModel, ConfigDict\n\nimport numpy as np\nimport pandas as pd\n\nfrom copy import deepcopy as copy\n\nimport sklearn\n\nsklearn.set_config(\n    assume_finite=True, skip_parameter_validation=True\n)  # Faster, we do checking\n\nfrom scipy.sparse import csr_matrix\n\nfrom scipy.spatial.distance import cdist\n\nfrom sklearn.linear_model import ElasticNet, LinearRegression, Ridge, Lasso\nfrom sklearn.kernel_ridge import KernelRidge\nfrom sklearn.preprocessing import StandardScaler, RobustScaler\n\nfrom timeit import default_timer as timer\n\nimport json\n\nfrom opendsm.eemeter.models.hourly import settings as _settings\nfrom opendsm.eemeter.models.hourly import HourlyBaselineData, HourlyReportingData\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\nfrom opendsm.common.clustering.cluster import cluster_features\nfrom opendsm.common.stats.adaptive_loss import adaptive_weights\nfrom opendsm.common.metrics import BaselineMetrics, BaselineMetricsFromDict, ReportingMetrics\nfrom opendsm import __version__\n\n\n\nclass AdaptiveElasticNetRegressor:  \n    def __init__(self, base_model, settings):\n        self.settings = settings\n\n        self.base_model = base_model\n        self.base_model.warm_start = True\n\n        self._hour_model = copy(self.base_model)\n\n    \n    def fit(self, X, y, sample_weight=None):\n        \"\"\"\n        Fit the model with X, y data.\n        \n        Parameters:\n        -----------\n        X : array-like of shape (n_samples, n_features)\n            The training input samples.\n        y : array-like of shape (n_samples,) or (n_samples, n_targets)\n            The target values.\n        sample_weight : array-like of shape (n_samples,), default=None\n            Sample weights.\n            \n        Returns:\n        --------\n        self : returns an instance of self.\n        \"\"\"\n        settings = self.settings.adaptive_weights\n        window_size = self.settings.adaptive_weights.window_size - 1\n        tol = self.settings.adaptive_weights.tol\n\n        num_hours = y.shape[1]\n\n        # fit the base model as an initial guess\n        self.base_model.fit(X, y, sample_weight=sample_weight)\n\n        if sample_weight is None:\n            weights = np.ones((X.shape[0], num_hours))\n        else:\n            weights = sample_weight\n\n        hour_fit = [False for _ in range(num_hours)]\n        alpha_prior = np.array([2.0 for _ in range(num_hours)])\n        alpha_min = alpha_prior.copy()\n        for i in range(settings.max_iter):\n            if all(hour_fit):\n                i -= 1\n                break\n\n            # get prediction and residuals for all hours\n            y_fit = self.base_model.predict(X)\n            resid = y - y_fit\n\n            for hour in range(num_hours):\n                # if hour_fit[hour]:\n                #     continue\n\n                # Update weights\n                # Calculate weights using window of hours\n                window_idx = np.arange(hour - window_size, hour + window_size + 1)\n\n                # if idx_i < 0, roll to the end or if idx_i >= num_hours, roll to the beginning \n                for idx_i in range(len(window_idx)):\n                    if window_idx[idx_i] < 0:\n                        window_idx[idx_i] = num_hours + window_idx[idx_i]\n\n                    if window_idx[idx_i] >= num_hours:\n                        window_idx[idx_i] = window_idx[idx_i] - num_hours\n\n                # unique values in idx only\n                window_idx = list(set(window_idx))\n  \n                # calculate weights\n                weights_update, _, alpha = adaptive_weights(\n                    resid[:,window_idx].flatten(), \n                    alpha=\"adaptive\", \n                    sigma=settings.sigma, \n                    quantile=0.25, \n                    min_weight=0.0,\n                    C_algo=settings.c_algo,\n                )\n\n                # break criteria\n                if (alpha == 2) or (np.abs(alpha - alpha_prior[hour]) <= tol):\n                    hour_fit[hour] = True\n                    continue\n                else:\n                    hour_fit[hour] = False\n\n                # update weights and alpha_prior\n                alpha_prior[hour] = alpha\n                alpha_min[hour] = min(alpha_min[hour], alpha)\n\n                # trim weights to hour size\n                if window_size > 0:\n                    # get index of hour in window_idx\n                    idx = window_idx.index(hour)\n                    hour_len = int(len(weights_update)/len(window_idx))\n\n                    weights_update = weights_update[idx*hour_len:(idx+1)*hour_len]\n\n                weights[:, hour] *= weights_update\n                \n                # update hour model from base model\n                self._hour_model.coef_ = self.base_model.coef_[hour,:]\n                self._hour_model.intercept_ = self.base_model.intercept_[hour]\n\n                # fit\n                self._hour_model.fit(\n                    X, \n                    y[:, hour], \n                    sample_weight=weights[:, hour]\n                )\n                \n                # update base model from refit hour model            \n                self.base_model.coef_[hour,:] = self._hour_model.coef_\n                self.base_model.intercept_[hour] = self._hour_model.intercept_\n\n        # save info to base_model\n        self.base_model.adaptive_iterations = i\n        self.base_model.adaptive_alpha = alpha_min\n        self.base_model.adaptive_weights = weights\n           \n        return self\n\n    @property\n    def is_fit(self):\n        \"\"\"Check if the model is fitted.\"\"\"\n        is_fit = True\n\n        if not hasattr(self.base_model, \"coef_\"):\n            is_fit = False\n\n        if not hasattr(self.base_model, \"intercept_\"):\n            is_fit = False\n\n        return is_fit\n    \n    def predict(self, X):\n        \"\"\"\n        Predict using the model.\n        \n        Parameters:\n        -----------\n        X : array-like of shape (n_samples, n_features)\n            The input samples.\n            \n        Returns:\n        --------\n        y : array of shape (n_samples,) or (n_samples, n_targets)\n            The predicted values.\n        \"\"\"\n        if not self.is_fit:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n\n        y = self.base_model.predict(X)\n\n        return y\n    \n    @property\n    def coef_(self):\n        \"\"\"Get model coefficients.\"\"\"\n        if not hasattr(self.base_model, \"coef_\"):\n            raise RuntimeError(\"Model coefficients must be set before accessed.\")\n        \n        return self.base_model.coef_\n        \n    @coef_.setter\n    def coef_(self, val):\n        self.base_model.coef_ = val\n\n    @property\n    def intercept_(self):\n        \"\"\"Get model intercepts.\"\"\"\n        if not hasattr(self.base_model, \"intercept_\"):\n            raise RuntimeError(\"Model intercepts must be set before accessed.\")\n\n        return self.base_model.intercept_\n        \n    @intercept_.setter\n    def intercept_(self, val):\n        \"\"\"Set model intercepts\"\"\"\n        self.base_model.intercept_ = val\n\n\nclass HourlyModel:\n    \"\"\"\n    A class to fit a model to the input meter data.\n\n    Attributes:\n        settings (dict): A dictionary of settings.\n        baseline_metrics (dict): A dictionary of metrics based on input baseline data and model fit.\n    \"\"\"\n    \n    # thresholds for switching model types\n    _alpha_model_threshold = 1E-5\n    _l1_ratio_model_threshold = 1E-4\n    _model_warning = EEMeterWarning\n    _base_settings = _settings\n\n    # set priority columns for sorting\n    # this is critical for ensuring predict column order matches fit column order\n    _priority_cols = {\n        \"ts\": [\"temporal_cluster\", \"temp_bin\", \"temperature\", \"ghi\"],\n        \"cat\": [\"temporal_cluster\", \"temp_bin\"],\n    }\n\n    _temporal_cluster_cols = [\"month\", \"day_of_week\"]\n\n    \"\"\"Note:\n        Despite the temporal clusters, we can view all models created as a subset of the same full model.\n        The temporal clusters would simply have the same coefficients within the same days/month combinations.\n    \"\"\"\n\n    def __init__(\n        self,\n        settings: dict | _settings.BaseHourlySettings | None = None,\n    ):\n        \"\"\"\n        Args:\n            settings: HourlySettings to use (generally left default). Will default to solar model if GHI is given to the fit step.\n        \"\"\"\n\n        # TODO move this logic into HourlySettings init\n        if isinstance(settings, dict):\n            if features := settings.get(\"train_features\"):\n                if \"ghi\" in features:\n                    settings = _settings.HourlySolarSettings(**settings)\n                else:\n                    settings = _settings.HourlyNonSolarSettings(**settings)\n            else:\n                settings = _settings.BaseHourlySettings(**settings)\n\n        # Initialize settings\n        if settings is None:\n            self.settings = _settings.BaseHourlySettings()\n        else:\n            self.settings = settings\n\n        # Initialize model\n        self._set_scalers()\n        self._model = self._set_model()\n\n        self._T_bin_edges = None\n        self._T_edge_bin_coeffs = None\n        self._T_edge_bin_rate = None\n        self._df_temporal_clusters = None\n        self._categorical_features = None\n        self._ts_feature_norm = None\n\n        self._ts_features = []\n        if self.settings.train_features:\n            self._ts_features = self.settings.train_features.copy()\n\n        self._is_fit = False\n        self.baseline_metrics = None\n        self.baseline_hour_metrics = None\n\n        self.warnings = []\n        self.disqualification = []\n\n        self.baseline_timezone = None\n        self.version = __version__\n\n    \n    def _set_scalers(self):\n        # set scalers\n        if self.settings.scaling_method == _settings.ScalingChoice.STANDARD_SCALER:\n            self._feature_scaler = StandardScaler()\n            self._y_scaler = StandardScaler()\n        elif self.settings.scaling_method == _settings.ScalingChoice.ROBUST_SCALER:\n            self._feature_scaler = RobustScaler(unit_variance=True)\n            self._y_scaler = RobustScaler(unit_variance=True)\n\n\n    def _set_model(self):\n        # set base model\n        if self.settings.base_model == _settings.BaseModel.ELASTICNET:\n            settings = self.settings.elasticnet\n            if settings.alpha <= self._alpha_model_threshold:\n                model = LinearRegression(\n                    fit_intercept=settings.fit_intercept\n                )\n            else:\n                if settings.l1_ratio < self._l1_ratio_model_threshold:\n                    base_model = Ridge\n                elif settings.l1_ratio > (1 - self._l1_ratio_model_threshold):\n                    base_model = Lasso\n                else:\n                    base_model = ElasticNet\n\n                model = base_model(\n                    alpha=settings.alpha,\n                    fit_intercept=settings.fit_intercept,\n                    max_iter=settings.max_iter,\n                    tol=settings.tol,\n                    random_state=settings._seed,\n                )\n                \n                if not isinstance(model, Ridge):\n                    model.precompute = settings.precompute\n                    model.selection = settings.selection\n                    model.warm_start = settings.warm_start\n\n                if isinstance(model, ElasticNet):\n                    model.l1_ratio = settings.l1_ratio\n\n            if self.settings.adaptive_weights.enabled:\n                model = AdaptiveElasticNetRegressor(model, self.settings)\n                \n        elif self.settings.base_model == _settings.BaseModel.KERNEL_RIDGE:\n            settings = self.settings.kernel_ridge\n            model = KernelRidge(\n                alpha=settings.alpha,\n                kernel=settings.kernel,\n                gamma=settings.gamma,\n            )\n\n        return model\n\n\n    def fit(\n        self, baseline_data: HourlyBaselineData, ignore_disqualification: bool = False\n    ) -> HourlyModel:\n        \"\"\"Fit the model using baseline data.\n\n        Args:\n            baseline_data: HourlyBaselineData object.\n            ignore_disqualification: Whether to ignore disqualification errors / warnings.\n\n        Returns:\n            The fitted model.\n\n        Raises:\n            TypeError: If baseline_data is not an HourlyBaselineData object.\n            DataSufficiencyError: If the model can't be fit on disqualified baseline data.\n        \"\"\"\n        if not isinstance(baseline_data, HourlyBaselineData):\n            raise TypeError(\"baseline_data must be an HourlyBaselineData object\")\n        baseline_data.log_warnings()\n        if baseline_data.disqualification and not ignore_disqualification:\n            raise DataSufficiencyError(\"Can't fit model on disqualified baseline data\")\n        if \"ghi\" in self._ts_features and not \"ghi\" in baseline_data.df.columns:\n            raise ValueError(\n                \"Model was explicitly set to use GHI, but baseline data does not contain GHI.\"\n            )\n\n        self.warnings = baseline_data.warnings\n        self.disqualification = baseline_data.disqualification\n\n        if not self._ts_features:\n            self.settings = self.settings.add_default_features(baseline_data.df.columns)\n            self._ts_features = self.settings.train_features.copy()\n\n        if \"ghi\" in baseline_data.df.columns and not \"ghi\" in self._ts_features:\n            model_mismatch_warning = self._model_warning(\n                qualified_name=\"eemeter.potential_model_mismatch\",\n                description=(\n                    \"Model was explicitly set to ignore GHI, but baseline period contained a GHI column.\"\n                ),\n                data={},\n            )\n            model_mismatch_warning.warn()\n            self.warnings.append(model_mismatch_warning)\n        \n        self._fit(baseline_data)\n        self._check_model_fit()\n\n        return self\n\n    def _fit(self, meter_data):\n        self._is_fit = False\n\n        # Initialize dataframe\n        df_meter = meter_data.df  # used to have a copy here\n        self.baseline_timezone = meter_data.tz\n\n        # Prepare feature arrays/matrices\n        X, y, fit_mask = self._prepare_features(df_meter)\n        X_fit = X[fit_mask, :]\n        y_fit = y[fit_mask]\n\n        # fit the model\n        self._model.fit(X_fit, y_fit)\n        self._is_fit = True\n\n        # get model prediction of baseline\n        df_meter = self._predict(meter_data, X=X)\n\n        self._set_baseline_metrics(df_meter)\n\n        return self\n\n    def predict(\n        self,\n        reporting_data,\n        ignore_disqualification=False,\n    ) -> pd.DataFrame:\n        \"\"\"Predicts the energy consumption using the fitted model.\n\n        Args:\n            reporting_data (Union[HourlyBaselineData, HourlyReportingData]): The data used for prediction.\n            ignore_disqualification (bool, optional): Whether to ignore model disqualification. Defaults to False.\n\n        Returns:\n            Dataframe with input data along with predicted energy consumption.\n\n        Raises:\n            RuntimeError: If the model is not fitted.\n            DisqualifiedModelError: If the model is disqualified and ignore_disqualification is False.\n            TypeError: If the reporting data is not of type HourlyBaselineData or HourlyReportingData.\n        \"\"\"\n        if not self._is_fit:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n\n        if missing_features := (\n            set(self._ts_features) - set(reporting_data.df.columns)\n        ):\n            raise ValueError(\n                f\"Reporting data is missing the following features: {missing_features}\"\n            )\n\n        if \"ghi\" in reporting_data.df.columns and not \"ghi\" in self._ts_features:\n            model_mismatch_warning = self._model_warning(\n                qualified_name=\"eemeter.potential_model_mismatch\",\n                description=(\n                    \"Reporting data contains GHI, but model was fit without GHI.\"\n                ),\n                data={},\n            )\n            model_mismatch_warning.warn()\n            self.warnings.append(model_mismatch_warning)\n\n        if str(self.baseline_timezone) != str(reporting_data.tz):\n            raise ValueError(\n                \"Reporting data must use the same timezone that the model was initially fit on.\"\n            )\n\n        if self.disqualification and not ignore_disqualification:\n            raise DisqualifiedModelError(\n                \"Attempting to predict using disqualified model without setting ignore_disqualification=True\"\n            )\n\n        if not isinstance(reporting_data, (HourlyBaselineData, HourlyReportingData)):\n            raise TypeError(\n                \"reporting_data must be a HourlyBaselineData or HourlyReportingData object\"\n            )\n\n        return self._predict(reporting_data)\n\n    def _predict(self, eval_data, X=None):\n        \"\"\"\n        Makes model prediction on given temperature data.\n\n        Parameters:\n            df_eval (pandas.DataFrame): The evaluation dataframe.\n\n        Returns:\n            pandas.DataFrame: The evaluation dataframe with model predictions added.\n        \"\"\"\n\n        df_eval = eval_data.df  # used to have a copy here\n        dst_indices = _get_dst_indices(df_eval)\n        datetime_original = eval_data.df.index\n        # # get list of columns to keep in output\n        columns = df_eval.columns.tolist()\n        if \"datetime\" in columns:\n            columns.remove(\"datetime\")  # index in output, not column\n\n        if X is None:\n            X, _, _ = self._prepare_features(df_eval)\n\n        y_predict_scaled = self._model.predict(X)\n        y_predict = self._y_scaler.inverse_transform(y_predict_scaled)\n        y_predict = y_predict.flatten()\n\n        y_predict = _transform_dst(y_predict, dst_indices)\n\n        df_eval[\"predicted\"] = y_predict\n        df_eval = self._calculate_predicted_uncertianty(df_eval)\n\n        # # remove columns not in original columns and predicted\n        df_eval = df_eval[[*columns, \"predicted\", \"predicted_unc\"]]\n\n        # reindex to original datetime index\n        df_eval = df_eval.reindex(datetime_original)\n\n        return df_eval\n\n    def _prepare_features(self, meter_data):\n        \"\"\"\n        Initializes the meter data by performing the following operations:\n        - Renames the 'model' column to 'model_old' if it exists\n        - Converts the index to a DatetimeIndex if it is not already\n        - Adds a 'season' column based on the month of the index using the settings.season dictionary\n        - Adds a 'day_of_week' column based on the day of the week of the index\n        - Removes any rows with NaN values in the 'temperature' or 'observed' columns\n        - Sorts the data by the index\n        - Reorders the columns to have 'season' and 'day_of_week' first, followed by the remaining columns\n\n        Parameters:\n        - meter_data: A pandas DataFrame containing the meter data\n\n        Returns:\n        - A pandas DataFrame containing the initialized meter data\n        \"\"\"\n        dst_indices = _get_dst_indices(meter_data)\n        meter_data = self._add_categorical_features(meter_data)\n        self._add_supplemental_features(meter_data)\n\n        self._ts_features, self._categorical_features = self._sort_features(\n            self._ts_features, self._categorical_features\n        )\n\n        meter_data = self._daily_fitting_sufficiency(meter_data)\n        meter_data = self._normalize_features(meter_data)\n        meter_data = self._add_temperature_interactions(meter_data)\n\n        # save actual df used for later inspection\n        self._ts_feature_norm, _ = self._sort_features(self._ts_feature_norm)\n        selected_features = self._ts_feature_norm + self._categorical_features\n        if \"observed_norm\" in meter_data.columns:\n            selected_features += [\"observed_norm\"]\n        self._processed_meter_data_full = meter_data\n        self._processed_meter_data = self._processed_meter_data_full[selected_features]\n\n        # get feature matrices\n        X, y, fit_mask = self._get_feature_matrices(meter_data, dst_indices)\n\n        # Convert to sparse matrix\n        X = csr_matrix(X.astype(float))\n\n        return X, y, fit_mask\n\n    def _add_temperature_bins(self, df):\n        # TODO: do we need to do something about empty bins in prediction? I think not but maybe\n        settings = self.settings.temperature_bin\n\n        # add temperature bins based on temperature\n        if not self._is_fit:\n            if settings.method == \"equal_sample_count\":\n                T_bin_edges = pd.qcut(\n                    df[\"temperature\"], q=settings.n_bins, labels=False\n                )\n\n            elif settings.method == \"equal_bin_width\":\n                T_bin_edges = pd.cut(\n                    df[\"temperature\"], bins=settings.n_bins, labels=False\n                )\n\n            elif settings.method == \"set_bin_width\":\n                bin_width = settings.bin_width\n\n                min_temp = np.floor(df[\"temperature\"].min())\n                max_temp = np.ceil(df[\"temperature\"].max())\n\n                if not settings.include_edge_bins:\n                    step_num = (\n                        np.round((max_temp - min_temp) / bin_width).astype(int) + 1\n                    )\n\n                    # T_bin_edges = np.arange(min_temp, max_temp + bin_width, bin_width)\n                    T_bin_edges = np.linspace(min_temp, max_temp, step_num)\n\n                else:\n                    set_edge_bin_width = False\n                    if set_edge_bin_width:\n                        edge_bin_width = bin_width * 1 / 2\n\n                        bin_range = [\n                            min_temp + edge_bin_width,\n                            max_temp - edge_bin_width,\n                        ]\n\n                    else:\n                        edge_bin_count = int(len(df) * settings.edge_bin_percent)\n\n                        # get 5th smallest and 5th largest temperatures\n                        sorted_temp = np.sort(df[\"temperature\"])\n                        min_temp_reg_bin = np.ceil(sorted_temp[edge_bin_count])\n                        max_temp_reg_bin = np.floor(sorted_temp[-edge_bin_count])\n\n                        bin_range = [min_temp_reg_bin, max_temp_reg_bin]\n\n                    step_num = (\n                        np.round((bin_range[1] - bin_range[0]) / bin_width).astype(int)\n                        + 1\n                    )\n\n                    # create bins with set width\n                    T_bin_edges = np.array(\n                        [min_temp, *np.linspace(*bin_range, step_num), max_temp]\n                    )\n\n            elif settings.method == \"fixed_bins\":\n                temp =  df[\"temperature\"].values\n                min_temp = np.floor(np.min(temp))\n                max_temp = np.ceil(np.max(temp))\n\n                T_bin_edges = np.array(settings.fixed_bins)\n                T_bin_edges = np.array([-np.inf, *T_bin_edges, np.inf])\n                \n                def _merge_bins(bin_edges, temp, min_bin_count):\n                    # if less than 20 values from df[\"temperature\"] are in a bin, \n                    # remove bin edge starting from edges and moving inwards\n                    def _eliminate_empty_bins(bin_edges, temp):\n                        valid_bin_edges = [-np.inf, ]\n                        for i in range(len(bin_edges) - 1):\n                            bin_count = ((temp >= bin_edges[i]) & (temp < bin_edges[i + 1])).sum()\n                            if bin_count > 0:\n                                valid_bin_edges.append(bin_edges[i + 1])\n                        valid_bin_edges[-1] = np.inf\n\n                        return np.array(valid_bin_edges)   \n                    \n                    bin_edges = _eliminate_empty_bins(bin_edges, temp)\n                    for i in range(int(np.ceil(len(bin_edges)/2)) - 1):\n                        if bin_edges[i+1] == bin_edges[-(i + 2)]:\n                            continue # only 1 bin edge left\n\n                        # left side\n                        bin_count = ((temp >= bin_edges[i]) & (temp < bin_edges[i+1])).sum()\n                        if bin_count < min_bin_count:\n                            bin_edges[i+1] = bin_edges[i]\n\n                        # right side\n                        bin_count = ((temp >= bin_edges[-(i + 2)]) & (temp < bin_edges[-(i + 1)])).sum()\n                        if bin_count < min_bin_count:\n                            bin_edges[-(i + 2)] = bin_edges[-(i + 1)]\n\n                    return np.unique(bin_edges)\n\n                if temp.size < settings.min_bin_count:\n                    raise ValueError(\"Not enough data to form temperature bins\")\n                elif temp.size < settings.min_bin_count*2:\n                    T_bin_edges = np.array([-np.inf, np.inf])\n                else:\n                    T_bin_edges = _merge_bins(T_bin_edges, temp, settings.min_bin_count)\n\n            else:\n                raise ValueError(\"Invalid temperature binning method\")\n\n            # set the first and last bin to -inf and inf\n            T_bin_edges[0] = -np.inf\n            T_bin_edges[-1] = np.inf\n\n            # store bin edges for prediction\n            self._T_bin_edges = T_bin_edges\n\n        T_bins = pd.cut(df[\"temperature\"], bins=self._T_bin_edges, labels=False)\n\n        df[\"temp_bin\"] = T_bins\n\n        # Create dummy variables for temperature bins\n        bin_dummies = pd.get_dummies(\n            pd.Categorical(\n                df[\"temp_bin\"], categories=range(len(self._T_bin_edges) - 1)\n            ),\n            prefix=\"temp_bin\",\n        )\n        bin_dummies.index = df.index\n\n        col_names = bin_dummies.columns.tolist()\n        df = pd.merge(df, bin_dummies, how=\"left\", left_index=True, right_index=True)\n\n        return df, col_names\n\n    def _add_categorical_features(self, df):\n        def set_initial_temporal_clusters(df):\n            fit_df_grouped = (\n                df.groupby(self._temporal_cluster_cols + [\"hour_of_day\"])[\"observed\"]\n                .agg(self.settings.temporal_cluster_aggregation)\n                .reset_index()\n            )\n            # pivot table to get 2D array of observed values\n            fit_df_grouped = fit_df_grouped.pivot_table(\n                index=self._temporal_cluster_cols,\n                columns=\"hour_of_day\",\n                values=\"observed\",\n            )\n\n            labels = cluster_features(\n                fit_df_grouped,\n                self.settings.temporal_cluster\n            )\n\n            df_temporal_clusters = pd.DataFrame(\n                labels,\n                columns=[\"temporal_cluster\"],\n                index=fit_df_grouped.index,\n            )\n\n            return df_temporal_clusters\n\n        def correct_missing_temporal_clusters(df):\n            # check and match any missing temporal combinations\n\n            # get all unique combinations of month and day_of_week in df\n            df_temporal = df[self._temporal_cluster_cols].drop_duplicates()\n            df_temporal = df_temporal.sort_values(self._temporal_cluster_cols)\n            df_temporal_index = df_temporal.set_index(self._temporal_cluster_cols).index\n            available_combinations = df_temporal_index\n\n            # reindex self.df_temporal_clusters to df_temporal_index\n            df_temporal_clusters = self._df_temporal_clusters.reindex(df_temporal_index)\n\n            # get index of any nan values in df_temporal_clusters\n            missing_combinations = df_temporal_clusters[\n                df_temporal_clusters[\"temporal_cluster\"].isna()\n            ].index\n            if not missing_combinations.empty:\n                if missing_combinations == available_combinations:\n                    raise ValueError(\n                        f\"Data does not have known temporal clusters of {self._temporal_cluster_cols}. Can't assign missing temporal clusters\"\n                    )\n                elif \"observed\" in df.columns and not df[\"observed\"].isnull().all():\n                    # filter df to only include missing combinations\n                    df_missing = df[\n                        df.set_index(self._temporal_cluster_cols).index.isin(\n                            missing_combinations\n                        )\n                    ]\n\n                    df_missing_grouped = (\n                        df_missing.groupby(\n                            self._temporal_cluster_cols + [\"hour_of_day\"]\n                        )[\"observed\"]\n                        .agg(self.settings.temporal_cluster_aggregation)\n                        .reset_index()\n                    )\n                    df_missing_grouped = df_missing_grouped.pivot_table(\n                        index=self._temporal_cluster_cols,\n                        columns=\"hour_of_day\",\n                        values=\"observed\",\n                    )\n                    X = df_missing_grouped.values\n\n                    # calculate average observed for known clusters\n                    # join df_temporal_clusters to df on month and day_of_week\n                    df = pd.merge(\n                        df,\n                        df_temporal_clusters,\n                        how=\"left\",\n                        left_on=self._temporal_cluster_cols,\n                        right_index=True,\n                    )\n\n                    df_known = df[\n                        ~df.set_index(self._temporal_cluster_cols).index.isin(\n                            missing_combinations\n                        )\n                    ]\n\n                    df_known_mean = (\n                        df_known.groupby(self._temporal_cluster_cols + [\"hour_of_day\"])[\n                            \"observed\"\n                        ]\n                        .mean()\n                        .reset_index()\n                    )\n                    df_known_mean = df_known_mean.pivot_table(\n                        index=self._temporal_cluster_cols,\n                        columns=\"hour_of_day\",\n                        values=\"observed\",\n                    )\n                    X_known = df_known_mean.values\n\n                    # get smallest distance between X and X_known\n                    dist = cdist(X, X_known, metric=\"euclidean\")\n                    min_dist_idx = np.argmin(dist, axis=1)\n\n                    # get temporal clusters df_known\n                    temporal_clusters = df_known.groupby(self._temporal_cluster_cols)[\n                        \"temporal_cluster\"\n                    ].first()\n                    temporal_clusters = temporal_clusters.reindex(df_known_mean.index)\n\n                    # set labels to minimum distance of known clusters\n                    labels = temporal_clusters.iloc[min_dist_idx].values\n                    df_temporal_clusters.loc[\n                        missing_combinations, \"temporal_cluster\"\n                    ] = labels\n\n                    self._df_temporal_clusters = df_temporal_clusters\n\n                else:\n                    # TODO: There's better ways of handling this\n                    # unstack and fill missing days in each month\n                    # assuming months more important than days\n                    df_temporal_clusters = df_temporal_clusters.unstack()\n\n                    # fill missing days in each month\n                    df_temporal_clusters = df_temporal_clusters.ffill(axis=1)\n                    df_temporal_clusters = df_temporal_clusters.bfill(axis=1)\n\n                    # fill missing months if any remaining empty\n                    df_temporal_clusters = df_temporal_clusters.ffill(axis=0)\n                    df_temporal_clusters = df_temporal_clusters.bfill(axis=0)\n\n                    df_temporal_clusters = df_temporal_clusters.stack()\n\n            return df_temporal_clusters\n\n        # assign basic temporal features\n        df[\"date\"] = df.index.date\n        df[\"month\"] = df.index.month\n        df[\"day_of_week\"] = df.index.dayofweek\n        df[\"hour_of_day\"] = df.index.hour\n\n        # assign temporal clusters\n        if not self._is_fit:\n            self._df_temporal_clusters = set_initial_temporal_clusters(df)\n            n_clusters = self._df_temporal_clusters[\"temporal_cluster\"].nunique()\n\n        else:\n            self._df_temporal_clusters = correct_missing_temporal_clusters(df)\n\n            # Get all unique temporal clusters from categorical features\n            temporal_cluster = []\n            for col in self._categorical_features:\n                if \"temporal_cluster\" in col:\n                    match = re.match(r'^temporal_cluster_(\\d+)*', col)\n                    if match and int(match.group(1)) not in temporal_cluster:\n                        temporal_cluster.append(int(match.group(1)))\n\n            n_clusters = len(temporal_cluster)\n\n        # join df_temporal_clusters to df\n        df = pd.merge(\n            df,\n            self._df_temporal_clusters,\n            how=\"left\",\n            left_on=self._temporal_cluster_cols,\n            right_index=True,\n        )\n\n        cluster_dummies = pd.get_dummies(\n            pd.Categorical(df[\"temporal_cluster\"], categories=range(n_clusters)),\n            prefix=\"temporal_cluster\",\n        )\n        cluster_dummies.index = df.index\n\n        cluster_cat = [f\"temporal_cluster_{i}\" for i in range(n_clusters)]\n        self._categorical_features = cluster_cat\n\n        df = pd.merge(\n            df, cluster_dummies, how=\"left\", left_index=True, right_index=True\n        )\n\n        if self.settings.temperature_bin is not None:\n            df, temp_bin_cols = self._add_temperature_bins(df)\n            self._categorical_features.extend(temp_bin_cols)\n\n        return df\n\n    def _add_supplemental_features(self, df):\n        # TODO: should either do upper or lower on all strs\n        if self.settings.supplemental_time_series_columns is not None:\n            for col in self.settings.supplemental_time_series_columns:\n                if (col in df.columns) and (col not in self._ts_features):\n                    self._ts_features.append(col)\n\n        if self.settings.supplemental_categorical_columns is not None:\n            for col in self.settings.supplemental_categorical_columns:\n                if (\n                    (col in df.columns)\n                    and (col not in self._ts_features)\n                    and (col not in self._categorical_features)\n                ):\n                    self._categorical_features.append(col)\n\n    def _sort_features(self, ts_features=None, cat_features=None):\n        features = {\"ts\": ts_features, \"cat\": cat_features}\n\n        # sort features\n        for _type in [\"ts\", \"cat\"]:\n            feat = features[_type]\n\n            if feat is not None:\n                sorted_cols = []\n                for col in self._priority_cols[_type]:\n                    cat_cols = [c for c in feat if c.startswith(col)]\n                    sorted_cols.extend(sorted(cat_cols))\n\n                # get all columns in self._categorical_feature not in sorted_cat_cols\n                leftover_cols = [c for c in feat if c not in sorted_cols]\n                if leftover_cols:\n                    sorted_cols.extend(sorted(leftover_cols))\n\n                features[_type] = sorted_cols\n\n        return features[\"ts\"], features[\"cat\"]\n\n    # TODO rename to avoid confusion with data sufficiency\n    def _daily_fitting_sufficiency(self, df):\n        # remove days with insufficient data\n        min_hours = self.settings.min_daily_training_hours\n\n        if min_hours > 0:\n            # find any rows with interpolated data\n            cols = [col for col in df.columns if col.startswith(\"interpolated_\")]\n            df[\"interpolated\"] = df[cols].any(axis=1)\n\n            # if row contains any null values, set interpolated to True\n            df[\"interpolated\"] = df[\"interpolated\"] | df.isnull().any(axis=1)\n\n            # count number of non interpolated hours per day\n            daily_hours = 24 - df.groupby(\"date\")[\"interpolated\"].sum()\n            sufficient_days = daily_hours[daily_hours >= min_hours].index\n\n            # set \"include_day\" column to True if day has sufficient hours\n            df[\"include_date\"] = df[\"date\"].isin(sufficient_days)\n\n        else:\n            df[\"include_date\"] = True\n\n        return df\n\n    def _normalize_features(self, df):\n        \"\"\" \"\"\"\n        train_features = self._ts_features\n        self._ts_feature_norm = [i + \"_norm\" for i in train_features]\n\n        # need to set scaler if not fit\n        if not self._is_fit:\n            self._feature_scaler.fit(df[train_features].values)\n            self._y_scaler.fit(df[\"observed\"].values.reshape(-1, 1))\n\n        data_transformed = self._feature_scaler.transform(df[train_features].values)\n        normalized_df = pd.DataFrame(\n            data_transformed, index=df.index, columns=self._ts_feature_norm\n        )\n\n        df = pd.concat([df, normalized_df], axis=1)\n\n        if \"observed\" in df.columns:\n            df[\"observed_norm\"] = self._y_scaler.transform(\n                df[\"observed\"].values.reshape(-1, 1)\n            )\n\n        if \"ghi\" in self._ts_features and \"ghi\" in df.columns:\n            df[\"ghi_norm\"] *= self.settings.ghi_scalar\n\n        return df\n    \n    def _add_extreme_temperature_bins(self, df, bin_range):\n        settings = self.settings.temperature_bin\n\n        def get_k(int_col, a, b):\n            k = []\n            for hour in range(24):\n                df_hour = df[df[\"hour_of_day\"] == hour]\n                df_hour = df_hour.sort_values(by=int_col)\n\n                x_data = a * df_hour[int_col].values + b\n                y_data = df_hour[\"observed\"].values\n\n                # Fit the model using robust least squares\n                try:\n                    params = _fit_exp_growth_decay(\n                        x_data, y_data, k_only=True, is_x_sorted=True\n                    )\n                    # save k for each hour\n                    k.append(params[2])\n                except:\n                    pass\n\n            k = np.abs(np.array(k))\n            k_valid = k[k < 5]\n\n            if len(k_valid) > 0:\n                k = np.mean(k_valid)\n            else:\n                k = 1 # if no valid k, set to 1\n\n            # if k is too small, set to minimum\n            k_min = 1/np.log(1E6)\n            if k < k_min:\n                k = k_min\n\n            return k\n\n        if self._T_edge_bin_coeffs is None:\n            self._T_edge_bin_coeffs = {}\n\n        cols = bin_range\n        # maybe add nonlinear terms to second and second to last columns?\n        # cols = [0, 1, last_temp_bin - 1, last_temp_bin]\n        # cols = list(set(cols))\n        # all columns?\n        # cols = range(cols[0], cols[1] + 1)\n\n        # Add all columns using col_dict at end\n        col_dict = {}\n        for n in cols:\n            base_col = f\"temp_bin_{n}\"\n            int_col = f\"{base_col}_ts\"\n            T_col = f\"{base_col}_T\"\n\n            # get k for exponential growth/decay\n            if not self._is_fit:\n                # determine temperature conversion for bin\n                range_offset = settings.edge_bin_temperature_range_offset\n                T_range = [\n                    df[int_col].min() - range_offset,\n                    df[int_col].max() + range_offset,\n                ]\n                new_range = [-1, 1]\n\n                T_a = (new_range[1] - new_range[0]) / (T_range[1] - T_range[0])\n                T_b = new_range[1] - T_a * T_range[1]\n\n                # The best rate for exponential\n                if settings.edge_bin_rate == \"heuristic\":\n                    k = get_k(int_col, T_a, T_b)\n                else:\n                    k = settings.edge_bin_rate\n\n                # get A for exponential\n                A = 1 / (np.exp(1 / k * new_range[1]) - 1)\n\n                self._T_edge_bin_coeffs[n] = {\n                    \"t_a\": float(T_a),\n                    \"t_b\": float(T_b),\n                    \"k\": float(k),\n                    \"a\": float(A),\n                }\n\n            T_a = self._T_edge_bin_coeffs[n][\"t_a\"]\n            T_b = self._T_edge_bin_coeffs[n][\"t_b\"]\n            k = self._T_edge_bin_coeffs[n][\"k\"]\n            A = self._T_edge_bin_coeffs[n][\"a\"]\n\n            col_dict[T_col] = np.where(\n                df[base_col].values, T_a * df[int_col].values + T_b, 0\n            )\n\n            for pos_neg in [\"pos\", \"neg\"]:\n                # if first or last column, add additional column\n                # testing exp, previously squaring worked well\n\n                s = 1\n                if \"neg\" in pos_neg:\n                    s = -1\n\n                # set rate exponential\n                ts_col = f\"{base_col}_{pos_neg}_exp_ts\"\n\n                col_dict[ts_col] = np.where(\n                    df[base_col].values, A * np.exp(s / k * col_dict[T_col]) - A, 0\n                )\n\n                self._ts_feature_norm.append(ts_col)\n\n        # create new df with col_dict\n        df = pd.concat([df, pd.DataFrame(col_dict, index=df.index)], axis=1)\n\n        return df\n\n    def _add_temperature_interactions(self, df):\n        settings = self.settings.temperature_bin\n\n        # TODO: if this permanent then it should not create, erase, make anew\n        self._ts_feature_norm.remove(\"temperature_norm\")\n\n        temp_bin_cols = [c for c in df.columns if re.match(r'^temp_bin_\\d+$', c)]\n        cluster_cols = [c for c in df.columns if re.match(r'^temporal_cluster_\\d+$', c)]\n\n        col_dict = {}\n\n        # add global temperature bins\n        for col in temp_bin_cols:\n            # splits temperature_norm into unique columns if that temp_bin column is True\n            ts_col = f\"{col}_ts\"\n            col_dict[ts_col] = df[\"temperature_norm\"] * df[col]\n\n            self._ts_feature_norm.append(ts_col)\n\n        # add temporal cluster interactions\n        # multiply each temp_bin by each temporal cluster\n        # get all columns that start with temp_bin_ and are a number\n        s = self.settings.interaction_scalar\n        for temporal_cluster_col in cluster_cols:\n            for temp_bin_col in temp_bin_cols:\n                # add intercept term\n                interaction_col = f\"{temporal_cluster_col}_{temp_bin_col}_interact\"\n                col_dict[interaction_col] = df[temp_bin_col] * df[temporal_cluster_col]\n\n                # add slope term\n                interaction_ts_col = f\"{interaction_col}_ts\"\n                # df[interaction_ts_col] = df[\"temperature_norm\"] * df[interaction_col]\n                col_dict[interaction_ts_col] = s*df[\"temperature_norm\"] * col_dict[interaction_col]\n\n                # add to feature lists\n                self._categorical_features.append(interaction_col)\n                self._ts_feature_norm.append(interaction_ts_col)\n\n        # concat df with col_dict\n        df = pd.concat([df, pd.DataFrame(col_dict, index=df.index)], axis=1)\n\n        # TODO: Model is better without this, but not sure why\n        # remove temporal cluster columns from categorical features\n        # cluster_cols = [c for c in df.columns if re.match(r'^temporal_cluster_\\d+(?!_)', c)]\n        # self._categorical_features = [c for c in self._categorical_features if c not in cluster_cols]\n\n        # add extreme temperature bins to global temperature bins\n        if settings.include_edge_bins:\n            bin_range = [0, len(temp_bin_cols) - 1]\n            df = self._add_extreme_temperature_bins(df, bin_range)\n\n        return df\n\n    def _get_feature_matrices(self, df, dst_indices):\n        # get aggregated features with agg function\n        agg_dict = {f: lambda x: list(x) for f in self._ts_feature_norm}\n\n        def correct_dst(agg):\n            \"\"\"interpolate or average hours to account for DST. modifies in place\"\"\"\n            interp, mean = dst_indices\n            for date, hour in interp:\n                for feature_idx, feature in enumerate(agg[date]):\n                    if hour == 0:\n                        # there are a handful of countries that use 0:00 as the DST transition\n                        interpolated = (\n                            agg[date - 1][feature_idx][-1] + feature[hour]\n                        ) / 2\n                    else:\n                        interpolated = (feature[hour - 1] + feature[hour]) / 2\n                    feature.insert(hour, interpolated)\n            for date, hour in mean:\n                for feature in agg[date]:\n                    mean = (feature[hour + 1] + feature.pop(hour)) / 2\n                    feature[hour] = mean\n\n        df_grouped = df.groupby(\"date\")\n        agg_x = df_grouped.agg(agg_dict).values.tolist()\n        correct_dst(agg_x)\n\n        # get the features and target for each day\n        ts_feature = np.array(agg_x)\n\n        ts_feature = ts_feature.reshape(\n            ts_feature.shape[0], ts_feature.shape[1] * ts_feature.shape[2]\n        )\n\n        # get the first categorical features for each day for each sample\n        unique_dummies = (\n            df[[\"date\"] + self._categorical_features].groupby(\"date\").first()\n        )\n\n        X = np.concatenate((ts_feature, unique_dummies), axis=1)\n\n        if not self._is_fit:\n            agg_y = (\n                df_grouped\n                .agg({\"observed_norm\": lambda x: list(x)})\n                .values.tolist()\n            )\n            correct_dst(agg_y)\n            y = np.array(agg_y)\n            y = y.reshape(y.shape[0], y.shape[1] * y.shape[2])\n\n            fit_mask = df_grouped[\"include_date\"].first().values\n        else:\n            y = None\n            fit_mask = None\n\n        return X, y, fit_mask\n\n    def _set_baseline_metrics(self, df_meter):\n        # get number of model parameters\n        if self.settings.base_model == _settings.BaseModel.ELASTICNET:\n            if self.settings.adaptive_weights.enabled:\n                self._model = self._model.base_model\n\n            num_parameters = np.count_nonzero(self._model.coef_)\n\n        elif self.settings.base_model == _settings.BaseModel.KERNEL_RIDGE:\n            num_parameters = np.count_nonzero(self._model.dual_coef_)\n\n        # calculate baseline metrics on non-interpolated data\n        # TODO: change interpolated to imputed\n        cols = [col for col in df_meter.columns if col.startswith(\"interpolated_\")]\n        interpolated = df_meter[cols].any(axis=1)\n\n        self.baseline_metrics = BaselineMetrics(\n            df=df_meter.loc[~interpolated], \n            num_model_params=num_parameters\n        )\n\n        # calculate baseline metrics per hour-of-day on non-interpolated data\n        self.baseline_hour_metrics = {}\n        for hour in range(24):\n            # get number of model parameters\n            if self.settings.base_model == _settings.BaseModel.ELASTICNET:\n                num_parameters = np.count_nonzero(self._model.coef_[hour]) \n\n            elif self.settings.base_model == _settings.BaseModel.KERNEL_RIDGE:\n                num_parameters = np.count_nonzero(self._model.dual_coef_[:,hour])   \n\n            hour_mask = df_meter.index.hour == hour\n            hour_data = df_meter.loc[hour_mask & ~interpolated]\n\n            self.baseline_hour_metrics[hour] = BaselineMetrics(\n                df=hour_data, \n                num_model_params=num_parameters\n            )\n\n    def _check_model_fit(self):\n        cvrmse = self.baseline_metrics.cvrmse_adj\n        pnrmse = self.baseline_metrics.pnrmse_adj\n\n        cvrmse_threshold = self.settings.cvrmse_threshold\n        pnrmse_threshold = self.settings.pnrmse_threshold\n\n        def _model_fit_is_acceptable(cvrmse, pnrmse):\n            # sufficient is (0 <= cvrmse <= threshold) or (0 <= pnrmse <= threshold)\n            if cvrmse is not None:\n                if (0 <= cvrmse) and (cvrmse <= cvrmse_threshold):\n                    return True\n                \n            if pnrmse is not None:\n                # less than 0 is not possible, but just in case\n                if (0 <= pnrmse) and (pnrmse <= pnrmse_threshold):\n                    return True\n\n            return False\n\n        if not _model_fit_is_acceptable(cvrmse, pnrmse):\n            model_fit_warning = self._model_warning(\n                qualified_name=\"eemeter.model_fit_metrics\",\n                description=\"Model disqualified due to poor fit.\",\n                data={\n                    \"cvrmse_threshold\": cvrmse_threshold,\n                    \"cvrmse_adj\": cvrmse,\n                    \"pnrmse_threshold\": pnrmse_threshold,\n                    \"pnrmse_adj\": pnrmse,\n                },\n            )\n            model_fit_warning.warn()\n            self.disqualification.append(model_fit_warning)\n\n    def _calculate_predicted_uncertianty(self, df_eval):\n        # initialize predicted_unc column with NaN\n        df_eval[\"predicted_unc\"] = np.nan\n\n        cols = [col for col in df_eval.columns if col.startswith(\"interpolated_\")]\n        interpolated = df_eval[cols].any(axis=1)\n\n        if self.baseline_metrics is None:\n            return df_eval\n\n        # calculate uncertainty using self.baseline_metrics\n        reporting_metrics = ReportingMetrics(\n            baseline_metrics=self.baseline_metrics,\n            reporting_df=df_eval[~interpolated],\n            data_frequency=\"hourly\",\n            confidence_level=self.settings.uncertainty_alpha,\n            t_tail=2,\n        )\n\n        df_eval[\"predicted_unc\"] = reporting_metrics.predicted_data_point_unc\n\n        # update uncertainties for each hour if available\n        # if self.baseline_hour_metrics is not None:\n        #     # calculate uncertainty using self.baseline_hour_metrics\n        #     for hour in range(24):\n        #         hour_mask = df_eval.index.hour == hour\n        #         hour_data = df_eval.loc[hour_mask & ~interpolated]\n\n        #         hour_reporting_metrics = ReportingMetrics(\n        #             baseline_metrics=self.baseline_hour_metrics[hour],\n        #             reporting_df=hour_data,\n        #             data_frequency=\"hourly\",\n        #             confidence_level=self.settings.uncertainty_alpha,\n        #             t_tail=2,\n        #         )\n\n        #         data_point_unc = hour_reporting_metrics.predicted_data_point_unc\n\n        #         if data_point_unc is not None:\n        #             df_eval.loc[hour_data.index, \"predicted_unc\"] = data_point_unc\n\n        return df_eval\n\n    def to_dict(self) -> dict:\n        \"\"\"Returns a dictionary of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        feature_scaler = {}\n        if self.settings.scaling_method == _settings.ScalingChoice.STANDARD_SCALER:\n            for i, key in enumerate(self._ts_features):\n                feature_scaler[key] = [\n                    self._feature_scaler.mean_[i],\n                    self._feature_scaler.scale_[i],\n                ]\n\n            y_scaler = [self._y_scaler.mean_.squeeze(), self._y_scaler.scale_.squeeze()]\n\n        elif self.settings.scaling_method == _settings.ScalingChoice.ROBUST_SCALER:\n            for i, key in enumerate(self._ts_features):\n                feature_scaler[key] = [\n                    self._feature_scaler.center_[i],\n                    self._feature_scaler.scale_[i],\n                ]\n\n            y_scaler = [\n                self._y_scaler.center_.squeeze(),\n                self._y_scaler.scale_.squeeze(),\n            ]\n\n        # convert self._df_temporal_clusters to list of lists\n        df_temporal_clusters = self._df_temporal_clusters.reset_index().values.tolist()\n\n        params = self._base_settings.SerializeModel(\n            settings=self.settings,\n            temporal_clusters=df_temporal_clusters,\n            temperature_bin_edges=self._T_bin_edges,\n            temperature_edge_bin_coefficients=self._T_edge_bin_coeffs,\n            ts_features=self._ts_features,\n            categorical_features=self._categorical_features,\n            coefficients=self._model.coef_.tolist(),\n            intercept=self._model.intercept_.tolist(),\n            feature_scaler=feature_scaler,\n            catagorical_scaler=None,\n            y_scaler=y_scaler,\n            baseline_metrics=self.baseline_metrics,\n            info=self._base_settings.ModelInfo(\n                disqualification=self.disqualification,\n                warnings=self.warnings,\n\n                baseline_timezone=str(self.baseline_timezone),\n                version=self.version,\n            ),\n        )\n\n        model_dict = params.model_dump()\n        return model_dict\n\n    def to_json(self) -> str:\n        \"\"\"Returns a JSON string of model parameters.\n\n        Returns:\n            Model parameters.\n        \"\"\"\n        return json.dumps(self.to_dict())\n\n    @classmethod\n    def from_dict(cls, data) -> HourlyModel:\n        \"\"\"Create a instance of the class from a dictionary (such as one produced from the to_dict method).\n\n        Args:\n            data (dict): The dictionary containing the model data.\n\n        Returns:\n            An instance of the class.\n        \"\"\"\n        # get settings\n        train_features = data.get(\"settings\").get(\"train_features\")\n\n        if \"ghi\" in train_features:\n            settings = _settings.HourlySolarSettings(**data.get(\"settings\"))\n        else:\n            settings = _settings.HourlyNonSolarSettings(**data.get(\"settings\"))\n\n        # initialize model class\n        model_cls = cls(settings=settings)\n\n        df_temporal_clusters = pd.DataFrame(\n            data.get(\"temporal_clusters\"),\n            columns=model_cls._temporal_cluster_cols + [\"temporal_cluster\"],\n        ).set_index(model_cls._temporal_cluster_cols)\n\n        model_cls._df_temporal_clusters = df_temporal_clusters\n        model_cls._T_bin_edges = np.array(data.get(\"temperature_bin_edges\"))\n        model_cls._T_edge_bin_coeffs = {\n            int(k): v for k, v in data.get(\"temperature_edge_bin_coefficients\").items()\n        }\n\n        model_cls._ts_features = data.get(\"ts_features\")\n        model_cls._categorical_features = data.get(\"categorical_features\")\n\n        # set scalers\n        feature_scaler_values = list(data.get(\"feature_scaler\").values())\n        feature_scaler_loc = [i[0] for i in feature_scaler_values]\n        feature_scaler_scale = [i[1] for i in feature_scaler_values]\n\n        y_scaler_values = data.get(\"y_scaler\")\n\n        if settings.scaling_method == _settings.ScalingChoice.STANDARD_SCALER:\n            model_cls._feature_scaler.mean_ = np.array(feature_scaler_loc)\n            model_cls._feature_scaler.scale_ = np.array(feature_scaler_scale)\n\n            model_cls._y_scaler.mean_ = np.array(y_scaler_values[0])\n            model_cls._y_scaler.scale_ = np.array(y_scaler_values[1])\n\n        elif settings.scaling_method == _settings.ScalingChoice.ROBUST_SCALER:\n            model_cls._feature_scaler.center_ = np.array(feature_scaler_loc)\n            model_cls._feature_scaler.scale_ = np.array(feature_scaler_scale)\n\n            model_cls._y_scaler.center_ = np.array(y_scaler_values[0])\n            model_cls._y_scaler.scale_ = np.array(y_scaler_values[1])\n\n        # set model\n        model_cls._model.coef_ = np.array(data.get(\"coefficients\"))\n        model_cls._model.intercept_ = np.array(data.get(\"intercept\"))\n\n        model_cls._is_fit = True\n\n        # set baseline metrics\n        model_cls.baseline_metrics = BaselineMetricsFromDict(\n            data.get(\"baseline_metrics\")\n        )\n\n        info = model_cls._base_settings.ModelInfo(**data.get(\"info\"))\n        model_cls.warnings = info.warnings\n        model_cls.disqualification = info.disqualification\n        model_cls.baseline_timezone = info.baseline_timezone\n        model_cls.version = info.version\n\n        return model_cls\n\n    @classmethod\n    def from_json(cls, str_data) -> HourlyModel:\n        \"\"\"Create an instance of the class from a JSON string.\n\n        Args:\n            str_data: The JSON string representing the object.\n\n        Returns:\n            An instance of the class.\n\n        \"\"\"\n        return cls.from_dict(json.loads(str_data))\n\n    def plot(\n        self,\n        df_eval: HourlyBaselineData | HourlyReportingData,\n    ):\n        \"\"\"Plot a model fit with baseline or reporting data.\n\n        Args:\n            df_eval: The baseline or reporting data object to plot.\n        \"\"\"\n        raise NotImplementedError\n\n\ndef _fit_exp_growth_decay(x, y, k_only=True, is_x_sorted=False):\n    # Courtsey: https://math.stackexchange.com/questions/1337601/fit-exponential-with-constant\n    #           https://www.scribd.com/doc/14674814/Regressions-et-equations-integrales\n    #           Jean Jacquelin\n\n    # fitting function is actual b*exp(c*x) + a\n\n    # sort x in order\n    x = np.array(x)\n    y = np.array(y)\n    n = len(x)\n\n    if not is_x_sorted:\n        sort_idx = np.argsort(x)\n        x = x[sort_idx]\n        y = y[sort_idx]\n\n    s = [0]\n    for i in range(1, len(x)):\n        s.append(s[i - 1] + 0.5 * (y[i] + y[i - 1]) * (x[i] - x[i - 1]))\n\n    s = np.array(s)\n\n    x_diff_sq = np.sum((x - x[0]) ** 2)\n    xs_diff = np.sum(s * (x - x[0]))\n    s_sq = np.sum(s**2)\n    xy_diff = np.sum((x - x[0]) * (y - y[0]))\n    ys_diff = np.sum(s * (y - y[0]))\n\n    A = np.array([[x_diff_sq, xs_diff], [xs_diff, s_sq]])\n    b = np.array([xy_diff, ys_diff])\n\n    _, c = np.linalg.solve(A, b)\n    with np.errstate(divide='ignore'):\n        k = 1 / c # ignore divide by zero, it will be filtered later\n\n    if k_only:\n        a, b = None, None\n    else:\n        theta_i = np.exp(c * x)\n\n        theta = np.sum(theta_i)\n        theta_sq = np.sum(theta_i**2)\n        y_sum = np.sum(y)\n        y_theta = np.sum(y * theta_i)\n\n        A = np.array([[n, theta], [theta, theta_sq]])\n        b = np.array([y_sum, y_theta])\n\n        a, b = np.linalg.solve(A, b)\n\n    return a, b, k\n\n\ndef _get_dst_indices(df):\n    \"\"\"\n    given a datetime-indexed dataframe,\n    return the indices which need to be interpolated and averaged\n    in order to ensure exact 24 hour slots\n    \"\"\"\n    # TODO test on baselines that begin/end on DST change\n    counts = df.groupby(df.index.date).count()\n    interp = counts[counts[\"observed\"] == 23]\n    mean = counts[counts[\"observed\"] == 25]\n\n    interp_idx = []\n    for idx in interp.index:\n        month = df.loc[idx.isoformat()]\n        date_idx = counts.index.get_loc(idx)\n        missing_hour = set(range(24)) - set(month.index.hour)\n        if len(missing_hour) != 1:\n            raise ValueError(\"too many missing hours\")\n        hour = missing_hour.pop()\n        interp_idx.append((date_idx, hour))\n\n    mean_idx = []\n    for idx in mean.index:\n        date_idx = counts.index.get_loc(idx)\n        month = df.loc[idx.isoformat()]\n        seen = set()\n        for i in month.index:\n            if i.hour in seen:\n                hour = i.hour\n                break\n            seen.add(i.hour)\n        mean_idx.append((date_idx, hour))\n\n    return interp_idx, mean_idx\n\n\ndef _transform_dst(prediction, dst_indices):\n    interp, mean = dst_indices\n\n    START_END = 0\n    REMOVE = 1\n    INTERPOLATE = 2\n\n    # get concrete indices\n    remove_idx = [(REMOVE, date * 24 + hour) for date, hour in interp]\n    interp_idx = [(INTERPOLATE, date * 24 + hour + 1) for date, hour in mean]\n\n    # these values will be inserted for the 25th hour\n    interpolated_vals = []\n    for _, idx in interp_idx:\n        interpolated = (prediction[idx - 1] + prediction[idx]) / 2\n        interpolated_vals.append(interpolated)\n    interpolation = iter(interpolated_vals)\n\n    # sort \"operations\" by index (can't assume a strict back-and-forth ordering)\n    ops = sorted(remove_idx + interp_idx, key=lambda t: t[1])\n\n    # create fenceposts where slices end\n    pairs = list(zip([(START_END, 0)] + ops, ops + [(START_END, None)]))\n    slices = []\n    for start, end in pairs:\n        start_i = start[1]\n        end_i = end[1]\n        if start[0] == REMOVE:\n            start_i += 1\n        if start[0] == INTERPOLATE:\n            slices.append([next(interpolation)])\n        slices.append(prediction[slice(start_i, end_i)])\n    return np.concatenate(slices)\n\n    ## the block above is equivalent to:\n    # shift = 0\n    # for op in ops:\n    #     if op[0] == REMOVE:\n    #         # delete artificial DST hour\n    #         idx = op[1] + shift\n    #         prediction = np.delete(prediction, idx)\n    #         shift -= 1\n    #     if op[0] == INTERPOLATE:\n    #         # interpolate missing DST hour\n    #         idx = op[1] + shift\n    #         interp = (prediction[idx - 1] + prediction[idx]) / 2\n    #         prediction = np.insert(prediction, idx, interp)\n    #         shift += 1\n    # return prediction\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly/settings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport numpy as np\nimport pandas as pd\n\nimport pydantic\n\nfrom enum import Enum\nfrom typing import Optional, Literal, Union, TypeVar, Dict\n\nimport pywt\n\nfrom opendsm.common.base_settings import BaseSettings\nfrom opendsm.common.clustering.settings import ClusteringSettings\nfrom opendsm.common.metrics import BaselineMetrics\nfrom opendsm.common.const import CAlgoChoice\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n# from opendsm.common.const import CountryCode\n\n\nclass SelectionChoice(str, Enum):\n    CYCLIC = \"cyclic\"\n    RANDOM = \"random\"\n\n\nclass ScalingChoice(str, Enum):\n    ROBUST_SCALER = \"robustscaler\"\n    STANDARD_SCALER = \"standardscaler\"\n\n\nclass BinningChoice(str, Enum):\n    EQUAL_SAMPLE_COUNT = \"equal_sample_count\"\n    EQUAL_BIN_WIDTH = \"equal_bin_width\"\n    SET_BIN_WIDTH = \"set_bin_width\"\n    FIXED_BINS = \"fixed_bins\"\n\n\nclass DefaultTrainingFeatures(str, Enum):\n    SOLAR = [\"temperature\", \"ghi\"]\n    NONSOLAR = [\"temperature\"]\n\n\nclass AggregationMethod(str, Enum):\n    MEAN = \"mean\"\n    MEDIAN = \"median\"\n\n\nclass BaseModel(str, Enum):\n    ELASTICNET = \"elasticnet\"\n    KERNEL_RIDGE = \"kernel_ridge\"\n\n\nclass TemperatureBinSettings(BaseSettings):\n    \"\"\"how to bin temperature data\"\"\"\n    method: BinningChoice = pydantic.Field(\n        default=BinningChoice.FIXED_BINS,\n    )\n\n    \"\"\"number of temperature bins\"\"\"\n    n_bins: Optional[int] = pydantic.Field(\n        default=None,\n        ge=1,\n    )\n\n    \"\"\"temperature bin width in fahrenheit\"\"\"\n    bin_width: Optional[float] = pydantic.Field(\n        default=25,\n        ge=1,\n    )\n\n    \"\"\"specified fixed temperature bins in fahrenheit\"\"\"\n    fixed_bins: Optional[list[float]] = pydantic.Field(\n        default=[10, 30, 50, 65, 75, 90, 105],\n    )\n\n    \"minimum bin count\"\n    min_bin_count: Optional[int] = pydantic.Field(\n        default=20,\n        ge=1,\n    )\n\n    \"\"\"use edge bins bool\"\"\"\n    include_edge_bins: bool = pydantic.Field(\n        default=True, \n    )\n\n    \"\"\"rate for edge temperature bins\"\"\"\n    edge_bin_rate: Optional[Union[float, Literal[\"heuristic\"]]] = pydantic.Field(\n        default=\"heuristic\",\n    )\n\n    \"\"\"percent of total data in edge bins\"\"\"\n    edge_bin_percent: Optional[float] = pydantic.Field(\n        default=None,\n        gt=0,\n        le=0.45,\n    )\n\n    \"\"\"offset normalized temperature range for edge bins (keeps exp from blowing up)\"\"\"\n    edge_bin_temperature_range_offset: Optional[float] = pydantic.Field(\n        default=1.0, # prior 1.0\n        ge=0,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_temperature_bins(self):\n        if self.method == BinningChoice.EQUAL_SAMPLE_COUNT:\n            if self.n_bins is None:\n                raise ValueError(\n                    \"'n_bins' must be specified if 'method' is 'equal_sample_count'.\"\n                )\n            if self.n_bins < 1:\n                raise ValueError(\"'n_bins' must be greater than 0.\")\n\n        elif self.method == BinningChoice.EQUAL_BIN_WIDTH:\n            if self.bin_width is None:\n                raise ValueError(\n                    \"'bin_width' must be specified if 'method' is 'equal_bin_width'.\"\n                )\n            if self.bin_width < 1:\n                raise ValueError(\"'bin_width' must be greater than 0.\")\n\n        elif self.method == BinningChoice.SET_BIN_WIDTH:\n            if self.bin_width is None:\n                raise ValueError(\n                    \"'bin_width' must be specified if 'method' is 'set_bin_width'.\"\n                )\n            if self.bin_width < 1:\n                raise ValueError(\"'bin_width' must be greater than 0.\")\n\n        elif self.method == BinningChoice.FIXED_BINS:\n            if self.fixed_bins is None:\n                raise ValueError(\n                    \"'fixed_bins' must be specified if 'method' is 'fixed_bins'.\"\n                )\n\n        else:\n            raise ValueError(f\"Invalid method: {self.method}\")\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_edge_bins(self):\n        if self.include_edge_bins:\n            if self.edge_bin_rate is None:\n                raise ValueError(\n                    \"'edge_bin_rate' must be specified if 'include_edge_bins' is True.\"\n                )\n            if self.edge_bin_percent is None and self.method != BinningChoice.FIXED_BINS:\n                raise ValueError(\n                    \"'edge_bin_days' must be specified if 'include_edge_bins' is True.\"\n                )\n            if self.edge_bin_temperature_range_offset is None:\n                raise ValueError(\n                    \"'edge_bin_temperature_range_offset' must be specified if 'include_edge_bins' is True.\"\n                )\n\n        else:\n            if self.edge_bin_rate is not None:\n                raise ValueError(\n                    \"'edge_bin_rate' must be None if 'include_edge_bins' is False.\"\n                )\n            if self.edge_bin_percent is not None:\n                raise ValueError(\n                    \"'edge_bin_days' must be None if 'include_edge_bins' is False.\"\n                )\n            if self.edge_bin_temperature_range_offset is not None:\n                raise ValueError(\n                    \"'edge_bin_temperature_range_offset' must be None if 'include_edge_bins' is False.\"\n                )\n\n        return self\n\n\nclass ElasticNetSettings(BaseSettings):\n    \"\"\"ElasticNet alpha parameter\"\"\"\n\n    alpha: float = pydantic.Field(\n        default=0.0139,\n        ge=0,\n    )\n\n    \"\"\"ElasticNet l1_ratio parameter\"\"\"\n    l1_ratio: float = pydantic.Field(\n        default=0.871,\n        ge=0,\n        le=1,\n    )\n\n    \"\"\"ElasticNet fit_intercept parameter\"\"\"\n    fit_intercept: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"ElasticNet parameter to precompute Gram matrix\"\"\"\n    precompute: bool = pydantic.Field(\n        default=False,\n    )\n\n    \"\"\"ElasticNet max_iter parameter\"\"\"\n    max_iter: int = pydantic.Field(\n        default=3000,\n        ge=1,\n        le=2**32 - 1,\n    )\n\n    \"\"\"ElasticNet copy_X parameter\"\"\"\n    copy_x: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"ElasticNet tol parameter\"\"\"\n    tol: float = pydantic.Field(\n        default=1e-3,\n        gt=0,\n    )\n\n    \"\"\"ElasticNet selection parameter\"\"\"\n    selection: SelectionChoice = pydantic.Field(\n        default=SelectionChoice.CYCLIC,\n    )\n\n    \"\"\"ElasticNet warm_start parameter\"\"\"\n    warm_start: bool = pydantic.Field(\n        default=False,\n    )\n\n\nclass KernelRidgeSettings(BaseSettings):\n    \"\"\"Kernel Ridge alpha parameter\"\"\"\n    alpha: float = pydantic.Field(\n        default=0.0425,\n        ge=0,\n    )\n\n    \"\"\"Kernel Ridge kernel parameter\"\"\"\n    kernel: str = pydantic.Field(\n        default=\"rbf\",\n    )\n\n    \"\"\"Kernel Ridge gamma parameter\"\"\"\n    gamma: Optional[float] = pydantic.Field(\n        default=None,\n        gt=0,\n    )\n\n\nclass AdaptiveWeightsSettings(BaseSettings):\n    \"\"\"Adaptive Weights for ElasticNet\"\"\"\n    enabled: bool = pydantic.Field(\n        default=True,\n    )\n\n    \"\"\"Sigma threshold for calculating C\"\"\"\n    sigma: Optional[float] = pydantic.Field(\n        default=4.55,\n        gt=0,\n    )\n\n    \"\"\"Adaptive weights window size\"\"\"\n    window_size: Optional[int] = pydantic.Field(\n        default=3,\n        ge=1,\n        le=12,\n    )\n\n    \"\"\"Algorithm to use for calculating C\"\"\"\n    c_algo: Optional[CAlgoChoice] = pydantic.Field(\n        default=CAlgoChoice.IQR,\n    )\n\n    \"\"\"Number of iterations to iterate weights\"\"\"\n    max_iter: Optional[int] = pydantic.Field(\n        default=100,   # Exits early based on tol\n        ge=1,\n    )\n\n    \"\"\"Relative difference in weights to stop iteration\"\"\"\n    tol: Optional[float] = pydantic.Field(\n        default=1E-3,   # Previously was using 1e-4\n        ge=0,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_adaptive_weights(self):\n        if self.enabled:\n            # iterate through all the parameters to check if they are set\n            # if any are None, raise an error\n            pass\n        else:\n            # iterate through all the parameters to check if they are set\n            # if any are not None, raise an error\n            pass\n\n        return self\n\n\nclass Criterion(str, Enum):\n    AIC = \"aic\"\n    BIC = \"bic\"\n\n\n# analytic_features = ['GHI', 'Temperature', 'DHI', 'DNI', 'Relative Humidity', 'Wind Speed', 'Clearsky DHI', 'Clearsky DNI', 'Clearsky GHI', 'Cloud Type']\nclass BaseHourlySettings(BaseSettings):\n    \"\"\"train features used within the model\"\"\"\n\n    train_features: Optional[list[str]] = None\n\n    \"\"\"CVRMSE threshold for model disqualification\"\"\"\n    cvrmse_threshold: float = pydantic.Field(\n        default=1.4,\n    )\n\n    \"\"\"PNRMSE threshold for model disqualification\"\"\"\n    pnrmse_threshold: float = pydantic.Field(\n        default=2.2,\n    )\n\n    \"\"\"minimum number of training hours per day below which a day is excluded\"\"\"\n    min_daily_training_hours: int = pydantic.Field(\n        default=12,\n        ge=0,\n        le=24,\n    )\n\n    \"\"\"temperature bin settings\"\"\"\n    temperature_bin: Optional[TemperatureBinSettings] = pydantic.Field(\n        default_factory=TemperatureBinSettings,\n    )\n\n    \"\"\"settings for temporal clustering\"\"\"\n    temporal_cluster: ClusteringSettings = pydantic.Field(\n        default_factory=ClusteringSettings,\n    )\n\n    \"\"\"temporal cluster aggregation method\"\"\"\n    temporal_cluster_aggregation: AggregationMethod = pydantic.Field(\n        default=AggregationMethod.MEDIAN,\n    )\n\n    \"\"\"temporal cluster/temperature bin/temperature interaction scalar\"\"\"\n    interaction_scalar: float = pydantic.Field(\n        default=0.524,\n        gt=0,\n    )\n\n    \"\"\"scalar for ghi feature\"\"\"\n    ghi_scalar: float = pydantic.Field(\n        default=1.0,\n        gt=0,\n    )\n\n    \"\"\"supplemental time series column names\"\"\"\n    supplemental_time_series_columns: Optional[list] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"supplemental categorical column names\"\"\"\n    supplemental_categorical_columns: Optional[list] = pydantic.Field(\n        default=None,\n    )\n\n    \"\"\"base model type\"\"\"\n    base_model: BaseModel = pydantic.Field(\n        default=BaseModel.ELASTICNET,\n    )\n\n    \"\"\"ElasticNet settings\"\"\"\n    elasticnet: Optional[ElasticNetSettings] = pydantic.Field(\n        default_factory=ElasticNetSettings,\n    )\n\n    \"\"\"Kernel Ridge settings\"\"\"\n    kernel_ridge: Optional[KernelRidgeSettings] = pydantic.Field(\n        default_factory=KernelRidgeSettings,\n    )\n\n    \"\"\"Adaptive Weights settings\"\"\"\n    adaptive_weights: AdaptiveWeightsSettings = pydantic.Field(\n        default_factory=AdaptiveWeightsSettings,\n    )\n\n    \"\"\"Feature scaling method\"\"\"\n    scaling_method: ScalingChoice = pydantic.Field(\n        default=ScalingChoice.STANDARD_SCALER,\n    )\n\n    \"\"\"Significance level used for uncertainty calculations\"\"\"\n    uncertainty_alpha: float = pydantic.Field(\n        default=0.1,\n        ge=0,\n        le=1,\n        description=\"Significance level used for uncertainty calculations\",\n    )\n\n    \"\"\"seed for any random state assignment (ElasticNet, Clustering)\"\"\"\n    seed: Optional[int] = pydantic.Field(\n        default=None,\n        ge=0,\n    )\n\n    @pydantic.model_validator(mode=\"after\")\n    def _check_seed(self):\n        if self.seed is None:\n            self._seed = np.random.randint(0, 2**32 - 1, dtype=np.int64)\n        else:\n            self._seed = self.seed\n\n        self.elasticnet._seed = self._seed\n        self.temporal_cluster._seed = self._seed\n\n        return self\n\n    @pydantic.model_validator(mode=\"after\")\n    def _remove_unselected_model_settings(self):\n        self.model_config[\"frozen\"] = False\n        \n        if self.base_model == BaseModel.ELASTICNET:\n            self.kernel_ridge = None\n        elif self.base_model == BaseModel.KERNEL_RIDGE:\n            self.elasticnet = None\n\n        self.model_config[\"frozen\"] = True\n\n        return self\n\n    def add_default_features(self, incoming_columns: list[str]):\n        \"\"\" \"called prior fit step to set default training features\"\"\"\n        if \"ghi\" in incoming_columns:\n            default_features = [\"temperature\", \"ghi\"]\n        else:\n            default_features = [\"temperature\"]\n        return self.model_copy(update={\"train_features\": default_features})\n\n\nclass HourlySolarSettings(BaseHourlySettings):\n    \"\"\"train features used within the model\"\"\"\n\n    train_features: list[str] = pydantic.Field(\n        default=[\"temperature\", \"ghi\"],\n    )\n\n    @pydantic.field_validator(\"train_features\", mode=\"after\")\n    def _add_required_features(cls, v):\n        required_features = [\"ghi\", \"temperature\"]\n        for feature in required_features:\n            if feature not in v:\n                v.insert(0, feature)\n        return v\n\n\nclass HourlyNonSolarSettings(BaseHourlySettings):\n    \"\"\"number of temperature bins\"\"\"\n\n    # TEMPERATURE_BIN_COUNT: Optional[int] = pydantic.Field(\n    #     default=10,\n    #     ge=1,\n    # )\n    train_features: list[str] = pydantic.Field(\n        default=[\"temperature\"],\n    )\n\n    @pydantic.field_validator(\"train_features\", mode=\"after\")\n    def _add_required_features(cls, v):\n        if \"temperature\" not in v:\n            v.insert(0, \"temperature\")\n        return v\n\n\nclass ModelInfo(pydantic.BaseModel):\n    \"\"\"additional information about the model\"\"\"\n\n    baseline_timezone: str\n    disqualification: list[EEMeterWarning]\n    warnings: list[EEMeterWarning]\n    version: str\n\n\nclass SerializeModel(BaseSettings):\n    model_config = pydantic.ConfigDict(arbitrary_types_allowed=True)\n\n    settings: Optional[BaseHourlySettings] = None\n    temporal_clusters: Optional[list[list[int]]] = None\n    temperature_bin_edges: Optional[list] = None\n    temperature_edge_bin_coefficients: Optional[Dict[int, Dict[str, float]]] = None\n    ts_features: Optional[list] = None\n    categorical_features: Optional[list] = None\n    feature_scaler: Optional[Dict[str, list[float]]] = None\n    catagorical_scaler: Optional[Dict[str, list[float]]] = None\n    y_scaler: Optional[list[float]] = None\n    coefficients: Optional[list[list[float]]] = None\n    intercept: Optional[list[float]] = None\n    baseline_metrics: Optional[BaselineMetrics] = None\n    info: ModelInfo\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .data import HourlyBaselineData, HourlyReportingData\nfrom .wrapper import HourlyModel\n\n__all__ = (\n    \"HourlyBaselineData\",\n    \"HourlyReportingData\",\n    \"HourlyModel\",\n)\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom typing import Optional, Union\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.common.data_processor_utilities import compute_minimum_granularity\nfrom opendsm.eemeter.common.features import compute_temperature_features, merge_features\nfrom opendsm.eemeter.models.hourly_caltrack.usage_per_day import (\n    caltrack_sufficiency_criteria,\n)\n\n\nclass HourlyReportingData:\n    def __init__(self, df: pd.DataFrame, is_electricity_data: bool):\n        if \"observed\" not in df.columns:\n            df[\"observed\"] = np.nan\n\n        if is_electricity_data:\n            df.loc[df[\"observed\"] == 0, \"observed\"] = np.nan\n\n        df = self._correct_frequency(df)\n\n        self.df = df\n        self.warnings = []\n        self.disqualification = []\n\n    def _correct_frequency(self, df: pd.DataFrame):\n        meter = df[\"observed\"]\n        temp = df[\"temperature\"]\n\n        # unknown for weirdly large frequencies. Anything higher frequency than hourly frequency still comes up as hourly\n        min_granularity = compute_minimum_granularity(meter.dropna().index, \"unknown\")\n\n        if meter.index.inferred_freq is None and min_granularity != \"hourly\":\n            raise ValueError(\n                f\"Meter Data must be atleast hourly, but is {min_granularity}.\"\n            )\n        else:\n            # TODO : Add the high frequency check for meter data\n            meter = meter.resample(\"h\").sum(min_count=1)\n            meter.index.freq = \"h\"\n\n        # TODO : Add the high frequency check for temperature data and add NaNs\n        temp = temp.resample(\"h\").mean()\n        temp.index.freq = \"h\"\n\n        return merge_features([meter, temp], keep_partial_nan_rows=True)\n\n    @classmethod\n    def from_series(\n        cls,\n        meter_data: Optional[pd.Series],\n        temperature_data: pd.Series,\n        is_electricity_data: bool,\n    ):\n        # TODO verify\n        if meter_data is None:\n            meter_data = temperature_data.copy().rename(\"observed\") * np.nan\n        df = merge_features([meter_data, temperature_data], keep_partial_nan_rows=True)\n        df = df.rename(\n            {\n                df.columns[0]: \"observed\",\n                df.columns[1]: \"temperature\",\n            },\n            axis=1,\n        )\n        return cls(df, is_electricity_data)\n\n\nclass HourlyBaselineData(HourlyReportingData):\n    def __init__(self, df: pd.DataFrame, is_electricity_data: bool):\n        if is_electricity_data:\n            df.loc[df[\"observed\"] == 0, \"observed\"] = np.nan\n\n        df = self._correct_frequency(df)\n\n        self.df = df\n        self.warnings = self._check_data_sufficiency()\n        self.disqualification = []\n\n    def _check_data_sufficiency(self):\n        meter = self.df[\"observed\"].rename(\"meter_value\")\n        temp = self.df[\"temperature\"]\n\n        temperature_features = compute_temperature_features(\n            meter.index,\n            temp,\n            data_quality=True,\n        )\n\n        sufficiency_df = merge_features([meter, temperature_features])\n        sufficiency = caltrack_sufficiency_criteria(\n            sufficiency_df, requested_start=None, requested_end=None\n        )\n        return sufficiency.warnings\n\n    @classmethod\n    def from_series(\n        cls,\n        meter_data: Union[pd.Series, pd.DataFrame],\n        temperature_data: Union[pd.Series, pd.DataFrame],\n        is_electricity_data: bool,\n    ):\n        if isinstance(meter_data, pd.Series):\n            meter_data = meter_data.to_frame()\n        if isinstance(temperature_data, pd.Series):\n            temperature_data = temperature_data.to_frame()\n        meter_data = meter_data.rename(columns={meter_data.columns[0]: \"observed\"})\n        temperature_data = temperature_data.rename(\n            columns={temperature_data.columns[0]: \"temperature\"}\n        )\n        temperature_data.index = temperature_data.index.tz_convert(\n            meter_data.index.tzinfo\n        )\n        df = pd.concat([meter_data, temperature_data], axis=1).dropna()\n        return cls(df, is_electricity_data)\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/derivatives.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom scipy.stats import t\n\nfrom opendsm.eemeter.models.daily.model import DailyModel\n\n__all__ = (\"metered_savings\", \"modeled_savings\")\n\n\ndef _compute_ols_error(\n    t_stat,\n    rmse_base_residuals,\n    post_obs,\n    base_obs,\n    base_avg,\n    post_avg,\n    base_var,\n    nprime,\n):\n    ols_model_agg_error = (\n        (t_stat * rmse_base_residuals * post_obs)\n        / (base_obs**0.5)\n        * (1.0 + ((base_avg - post_avg) ** 2.0 / base_var)) ** 0.5\n    )\n\n    ols_noise_agg_error = (\n        t_stat * rmse_base_residuals * (post_obs * base_obs / nprime) ** 0.5\n    )\n\n    ols_total_agg_error = (ols_model_agg_error**2.0 + ols_noise_agg_error**2.0) ** 0.5\n\n    return ols_total_agg_error, ols_model_agg_error, ols_noise_agg_error\n\n\ndef _compute_fsu_error(\n    t_stat,\n    interval,\n    post_obs,\n    total_base_energy,\n    rmse_base_residuals,\n    base_avg,\n    base_obs,\n    nprime,\n):\n    if interval.startswith(\"billing\"):\n        a_coeff = -0.00022\n        b_coeff = 0.03306\n        c_coeff = 0.94054\n        months_reporting = float(post_obs)\n    else:  # daily\n        a_coeff = -0.00024\n        b_coeff = 0.03535\n        c_coeff = 1.00286\n        months_reporting = float(post_obs) / 30.0\n\n    fsu_error_band = total_base_energy * (\n        t_stat\n        * (a_coeff * months_reporting**2.0 + b_coeff * months_reporting + c_coeff)\n        * (rmse_base_residuals / base_avg)\n        * ((base_obs / nprime) * (1.0 + (2.0 / nprime)) * (1.0 / post_obs)) ** 0.5\n    )\n\n    return fsu_error_band\n\n\ndef _compute_error_bands_metered_savings(\n    totals_metrics, results, interval, confidence_level\n):\n    num_parameters = float(totals_metrics.num_parameters)\n\n    base_obs = float(totals_metrics.observed_length)\n    if (interval.startswith(\"billing\")) & (len(results.dropna().index) > 0):\n        post_obs = float(round((results.index[-1] - results.index[0]).days / 30.0))\n    else:\n        post_obs = float(results[\"reporting_observed\"].dropna().shape[0])\n\n    degrees_of_freedom = float(base_obs - num_parameters)\n    single_tailed_confidence_level = 1 - ((1 - confidence_level) / 2)\n    t_stat = t.ppf(single_tailed_confidence_level, degrees_of_freedom)\n\n    rmse_base_residuals = float(totals_metrics.rmse_adj)\n    autocorr_resid = totals_metrics.autocorr_resid\n\n    base_avg = float(totals_metrics.observed_mean)\n    post_avg = float(results[\"reporting_observed\"].mean())\n\n    base_var = float(totals_metrics.observed_variance)\n\n    # these result in division by zero error for fsu_error_band\n    if (\n        post_obs == 0\n        or autocorr_resid is None\n        or abs(autocorr_resid) == 1\n        or base_obs == 0\n        or base_avg == 0\n        or base_var == 0\n    ):\n        return None\n\n    autocorr_resid = float(autocorr_resid)\n\n    nprime = float(base_obs * (1 - autocorr_resid) / (1 + autocorr_resid))\n\n    total_base_energy = float(base_avg * base_obs)\n\n    ols_total_agg_error, ols_model_agg_error, ols_noise_agg_error = _compute_ols_error(\n        t_stat,\n        rmse_base_residuals,\n        post_obs,\n        base_obs,\n        base_avg,\n        post_avg,\n        base_var,\n        nprime,\n    )\n\n    fsu_error_band = _compute_fsu_error(\n        t_stat,\n        interval,\n        post_obs,\n        total_base_energy,\n        rmse_base_residuals,\n        base_avg,\n        base_obs,\n        nprime,\n    )\n\n    return {\n        \"FSU Error Band\": fsu_error_band,\n        \"OLS Error Band\": ols_total_agg_error,\n        \"OLS Error Band: Model Error\": ols_model_agg_error,\n        \"OLS Error Band: Noise\": ols_noise_agg_error,\n    }\n\n\ndef metered_savings(\n    baseline_model,\n    reporting_meter_data,\n    temperature_data,\n    with_disaggregated=False,\n    confidence_level=0.90,\n    predict_kwargs=None,\n    degc: bool = False,\n    billing_data: bool = False,\n):\n    \"\"\"Compute metered savings, i.e., savings in which the baseline model\n    is used to calculate the modeled usage in the reporting period. This\n    modeled usage is then compared to the actual usage from the reporting period.\n    Also compute two measures of the uncertainty of the aggregate savings estimate,\n    a fractional savings uncertainty (FSU) error band and an OLS error band. (To convert\n    the FSU error band into FSU, divide by total estimated savings.)\n\n    Parameters\n    ----------\n    baseline_model : :any:`eemeter.CalTRACKUsagePerDayModelResults`\n        Object to use for predicting pre-intervention usage.\n    reporting_meter_data : :any:`pandas.DataFrame`\n        The observed reporting period data (totals). Savings will be computed for the\n        periods supplied in the reporting period data.\n    temperature_data : :any:`pandas.Series`\n        Hourly-frequency timeseries of temperature data during the reporting\n        period.\n    with_disaggregated : :any:`bool`, optional\n        If True, calculate baseline counterfactual disaggregated usage\n        estimates. Savings cannot be disaggregated for metered savings. For\n        that, use :any:`eemeter.modeled_savings`.\n    confidence_level : :any:`float`, optional\n        The two-tailed confidence level used to calculate the t-statistic used\n        in calculation of the error bands.\n\n        Ignored if not computing error bands.\n    predict_kwargs : :any:`dict`, optional\n        Extra kwargs to pass to the baseline_model.predict method.\n    degc : :any 'bool'\n        Relevant temperature units; defaults to False (i.e. Fahrenheit).\n\n    Returns\n    -------\n    results : :any:`pandas.DataFrame`\n        DataFrame with metered savings, indexed with\n        ``reporting_meter_data.index``. Will include the following columns:\n\n        - ``counterfactual_usage`` (baseline model projected into reporting period)\n        - ``reporting_observed`` (given by reporting_meter_data)\n        - ``metered_savings``\n\n        If `with_disaggregated` is set to True, the following columns will also\n        be in the results DataFrame:\n\n        - ``counterfactual_base_load``\n        - ``counterfactual_heating_load``\n        - ``counterfactual_cooling_load``\n\n    error_bands : :any:`dict`, optional\n        If baseline_model is an instance of CalTRACKUsagePerDayModelResults,\n        will also return a dictionary of FSU and OLS error bands for the\n        aggregated energy savings over the post period.\n    \"\"\"\n    if degc == True:\n        temperature_data = 32 + (temperature_data * 1.8)\n\n    if predict_kwargs is None:\n        predict_kwargs = {}\n\n    model_type = None\n    if isinstance(baseline_model, DailyModel):\n        raise NotImplementedError(\n            \"Use predict() with daily and billing models to compute metered savings.\"\n        )\n\n    prediction_index = reporting_meter_data.index\n    model_prediction = baseline_model.predict(\n        prediction_index, temperature_data, **predict_kwargs\n    )\n\n    predicted_baseline_usage = model_prediction.result\n\n    # CalTrack 3.5.1\n    counterfactual_usage = predicted_baseline_usage[\"predicted_usage\"].to_frame(\n        \"counterfactual_usage\"\n    )\n\n    reporting_observed = reporting_meter_data[\"value\"].to_frame(\"reporting_observed\")\n\n    def metered_savings_func(row):\n        return row.counterfactual_usage - row.reporting_observed\n\n    results = reporting_observed.join(counterfactual_usage).assign(\n        metered_savings=metered_savings_func\n    )\n\n    results = results.dropna().reindex(results.index)  # carry NaNs\n\n    # compute t-statistic associated with n degrees of freedom\n    # and a two-tailed confidence level.\n    error_bands = None\n    return results, error_bands\n\n\ndef _compute_error_bands_modeled_savings(\n    totals_metrics_baseline,\n    totals_metrics_reporting,\n    results,\n    interval_baseline,\n    interval_reporting,\n    confidence_level,\n):\n    num_parameters_baseline = float(totals_metrics_baseline.num_parameters)\n    num_parameters_reporting = float(totals_metrics_reporting.num_parameters)\n\n    base_obs_baseline = float(totals_metrics_baseline.observed_length)\n    base_obs_reporting = float(totals_metrics_reporting.observed_length)\n\n    if (interval_baseline.startswith(\"billing\")) & (len(results.dropna().index) > 0):\n        post_obs_baseline = float(\n            round((results.index[-1] - results.index[0]).days / 30.0)\n        )\n    else:\n        post_obs_baseline = float(results[\"modeled_baseline_usage\"].dropna().shape[0])\n\n    if (interval_reporting.startswith(\"billing\")) & (len(results.dropna().index) > 0):\n        post_obs_reporting = float(\n            round((results.index[-1] - results.index[0]).days / 30.0)\n        )\n    else:\n        post_obs_reporting = float(results[\"modeled_reporting_usage\"].dropna().shape[0])\n\n    degrees_of_freedom_baseline = float(base_obs_baseline - num_parameters_baseline)\n    degrees_of_freedom_reporting = float(base_obs_reporting - num_parameters_reporting)\n    single_tailed_confidence_level = 1 - ((1 - confidence_level) / 2)\n    t_stat_baseline = t.ppf(single_tailed_confidence_level, degrees_of_freedom_baseline)\n    t_stat_reporting = t.ppf(\n        single_tailed_confidence_level, degrees_of_freedom_reporting\n    )\n\n    rmse_base_residuals_baseline = float(totals_metrics_baseline.rmse_adj)\n    rmse_base_residuals_reporting = float(totals_metrics_reporting.rmse_adj)\n    autocorr_resid_baseline = totals_metrics_baseline.autocorr_resid\n    autocorr_resid_reporting = totals_metrics_reporting.autocorr_resid\n\n    base_avg_baseline = float(totals_metrics_baseline.observed_mean)\n    base_avg_reporting = float(totals_metrics_reporting.observed_mean)\n\n    # these result in division by zero error for fsu_error_band\n    if (\n        post_obs_baseline == 0\n        or autocorr_resid_baseline is None\n        or abs(autocorr_resid_baseline) == 1\n        or base_obs_baseline == 0\n        or base_avg_baseline == 0\n        or post_obs_reporting == 0\n        or autocorr_resid_reporting is None\n        or abs(autocorr_resid_reporting) == 1\n        or base_obs_reporting == 0\n        or base_avg_reporting == 0\n    ):\n        return None\n\n    autocorr_resid_baseline = float(autocorr_resid_baseline)\n    autocorr_resid_reporting = float(autocorr_resid_reporting)\n\n    nprime_baseline = float(\n        base_obs_baseline\n        * (1 - autocorr_resid_baseline)\n        / (1 + autocorr_resid_baseline)\n    )\n    nprime_reporting = float(\n        base_obs_reporting\n        * (1 - autocorr_resid_reporting)\n        / (1 + autocorr_resid_reporting)\n    )\n\n    total_base_energy_baseline = float(base_avg_baseline * base_obs_baseline)\n    total_base_energy_reporting = float(base_avg_reporting * base_obs_reporting)\n\n    fsu_error_band_baseline = _compute_fsu_error(\n        t_stat_baseline,\n        interval_baseline,\n        post_obs_baseline,\n        total_base_energy_baseline,\n        rmse_base_residuals_baseline,\n        base_avg_baseline,\n        base_obs_baseline,\n        nprime_baseline,\n    )\n\n    fsu_error_band_reporting = _compute_fsu_error(\n        t_stat_reporting,\n        interval_reporting,\n        post_obs_reporting,\n        total_base_energy_reporting,\n        rmse_base_residuals_reporting,\n        base_avg_reporting,\n        base_obs_reporting,\n        nprime_reporting,\n    )\n\n    return {\n        \"FSU Error Band: Baseline\": fsu_error_band_baseline,\n        \"FSU Error Band: Reporting\": fsu_error_band_reporting,\n        \"FSU Error Band\": (fsu_error_band_baseline**2.0 + fsu_error_band_reporting**2.0)\n        ** 0.5,\n    }\n\n\ndef modeled_savings(\n    baseline_model,\n    reporting_model,\n    result_index,\n    temperature_data,\n    with_disaggregated=False,\n    confidence_level=0.90,\n    predict_kwargs=None,\n    degc: bool = False,\n):\n    \"\"\"Compute modeled savings, i.e., savings in which baseline and reporting\n    usage values are based on models. This is appropriate for annualizing or\n    weather normalizing models.\n\n    Parameters\n    ----------\n    baseline_model : :any:`eemeter.CalTRACKUsagePerDayCandidateModel`\n        Model to use for predicting pre-intervention usage.\n    reporting_model : :any:`eemeter.CalTRACKUsagePerDayCandidateModel`\n        Model to use for predicting post-intervention usage.\n    result_index : :any:`pandas.DatetimeIndex`\n        The dates for which usage should be modeled.\n    temperature_data : :any:`pandas.Series`\n        Hourly-frequency timeseries of temperature data during the modeled\n        period.\n    with_disaggregated : :any:`bool`, optional\n        If True, calculate modeled disaggregated usage estimates and savings.\n    confidence_level : :any:`float`, optional\n        The two-tailed confidence level used to calculate the t-statistic used\n        in calculation of the error bands.\n\n        Ignored if not computing error bands.\n    predict_kwargs : :any:`dict`, optional\n        Extra kwargs to pass to the baseline_model.predict and\n        reporting_model.predict methods.\n    degc : :any 'bool'\n        Relevant temperature units; defaults to False (i.e. Fahrenheit).\n\n\n    Returns\n    -------\n    results : :any:`pandas.DataFrame`\n        DataFrame with modeled savings, indexed with the result_index. Will\n        include the following columns:\n\n        - ``modeled_baseline_usage``\n        - ``modeled_reporting_usage``\n        - ``modeled_savings``\n\n        If `with_disaggregated` is set to True, the following columns will also\n        be in the results DataFrame:\n\n        - ``modeled_baseline_base_load``\n        - ``modeled_baseline_cooling_load``\n        - ``modeled_baseline_heating_load``\n        - ``modeled_reporting_base_load``\n        - ``modeled_reporting_cooling_load``\n        - ``modeled_reporting_heating_load``\n        - ``modeled_base_load_savings``\n        - ``modeled_cooling_load_savings``\n        - ``modeled_heating_load_savings``\n    error_bands : :any:`dict`, optional\n        If baseline_model and reporting_model are instances of\n        CalTRACKUsagePerDayModelResults, will also return a dictionary of\n        FSU and error bands for the aggregated energy savings over the\n        normal year period.\n    \"\"\"\n    if degc == True:\n        temperature_data = 32 + (temperature_data * 1.8)\n\n    prediction_index = result_index\n\n    if predict_kwargs is None:\n        predict_kwargs = {}\n\n    model_type = None  # generic\n    if isinstance(baseline_model, DailyModel) or isinstance(\n        reporting_model, DailyModel\n    ):\n        raise NotImplementedError(\n            \"Use predict() with daily and billing models to compute modeled savings.\"\n        )\n\n    def _predicted_usage(model):\n        model_prediction = model.predict(\n            prediction_index, temperature_data, **predict_kwargs\n        )\n        predicted_usage = model_prediction.result\n        return predicted_usage\n\n    predicted_baseline_usage = _predicted_usage(baseline_model)\n    predicted_reporting_usage = _predicted_usage(reporting_model)\n    modeled_baseline_usage = predicted_baseline_usage[\"predicted_usage\"].to_frame(\n        \"modeled_baseline_usage\"\n    )\n    modeled_reporting_usage = predicted_reporting_usage[\"predicted_usage\"].to_frame(\n        \"modeled_reporting_usage\"\n    )\n\n    def modeled_savings_func(row):\n        return row.modeled_baseline_usage - row.modeled_reporting_usage\n\n    results = modeled_baseline_usage.join(modeled_reporting_usage).assign(\n        modeled_savings=modeled_savings_func\n    )\n\n    results = results.dropna().reindex(results.index)  # carry NaNs\n\n    error_bands = None\n    return results, error_bands\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/design_matrices.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\n\nfrom opendsm.eemeter.common.features import (\n    compute_temperature_features,\n    compute_time_features,\n    compute_usage_per_day_feature,\n    merge_features,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.model import (\n    caltrack_hourly_fit_feature_processor,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import (\n    iterate_segmented_dataset,\n)\n\n__all__ = (\n    \"create_caltrack_hourly_preliminary_design_matrix\",\n    \"create_caltrack_hourly_segmented_design_matrices\",\n    \"create_caltrack_daily_design_matrix\",\n    \"create_caltrack_billing_design_matrix\",\n)\n\n\ndef create_caltrack_hourly_preliminary_design_matrix(\n    meter_data, temperature_data, degc: bool = False\n):\n    \"\"\"A helper function which calls basic feature creation methods to create an\n    input suitable for use in the first step of creating a CalTRACK hourly model.\n\n    Parameters\n    ----------\n    meter_data : :any:`pandas.DataFrame`\n        Hourly meter data in eemeter format.\n    temperature_data : :any:`pandas.Series`\n        Hourly temperature data in eemeter format.\n    degc : :any 'bool'\n        Relevant temperature units; defaults to False (i.e. Fahrenheit).\n\n    Returns\n    -------\n    design_matrix : :any:`pandas.DataFrame`\n        A design matrix with meter_value, hour_of_week, hdd_(hbp_default), and cdd_(cbp_default) features.\n    \"\"\"\n\n    if degc == True:\n        temperature_data = 32 + (temperature_data * 1.8)\n\n    time_features = compute_time_features(\n        meter_data.index, hour_of_week=True, hour_of_day=False, day_of_week=False\n    )\n    temperature_features = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[50],\n        cooling_balance_points=[\n            65\n        ],  # note both HBP this will require further work in future iterations of eemeter - CBP in particular assumes all buildings have cooling, which is a strong assumption.\n        degree_day_method=\"hourly\",\n    )\n    design_matrix = merge_features(\n        [meter_data.value.to_frame(\"meter_value\"), temperature_features, time_features]\n    )\n    return design_matrix\n\n\ndef create_caltrack_billing_design_matrix(\n    meter_data, temperature_data, degc: bool = False\n):\n    \"\"\"A helper function which calls basic feature creation methods to create a\n    design matrix suitable for use with CalTRACK Billing methods.\n\n    Parameters\n    ----------\n    meter_data : :any:`pandas.DataFrame`\n        Monthly meter data in eemeter format.\n    temperature_data : :any:`pandas.Series`\n        Hourly temperature data in eemeter format.\n    degc : :any 'bool'\n        Relevant temperature units; defaults to Fahrenheit.\n\n    Returns\n    -------\n    design_matrix : :any:`pandas.DataFrame`\n        A design matrix with mean usage_per_day and temperature features.\n    \"\"\"\n    usage_per_day = compute_usage_per_day_feature(meter_data, series_name=\"meter_value\")\n    usage_per_day = usage_per_day.resample(\"D\").ffill()\n    if degc == True:\n        temperature_data = 32 + (temperature_data * 1.8)\n\n    temperature_features = compute_temperature_features(\n        usage_per_day.index,\n        temperature_data,\n        data_quality=True,\n        tolerance=pd.Timedelta(\n            \"35D\"\n        ),  # limit temperature data matching to periods of up to 35 days.\n    )\n    design_matrix = merge_features([usage_per_day, temperature_features])\n    return design_matrix\n\n\ndef create_caltrack_daily_design_matrix(\n    meter_data, temperature_data, degc: bool = False\n):\n    \"\"\"A helper function which calls basic feature creation methods to create a\n    design matrix suitable for use with CalTRACK daily methods.\n\n    Parameters\n    ----------\n    meter_data : :any:`pandas.DataFrame`\n        Daily meter data in eemeter format.\n    temperature_data : :any:`pandas.Series`\n        Hourly temperature data in eemeter format.\n     degc : :any 'bool'\n        Relevant temperature units; defaults to Fahrenheit.\n\n    Returns\n    -------\n    design_matrix : :any:`pandas.DataFrame`\n        A design matrix with mean usage_per_day and temperature features\n    \"\"\"\n    usage_per_day = compute_usage_per_day_feature(meter_data, series_name=\"meter_value\")\n    if degc == True:\n        temperature_data = 32 + (temperature_data * 1.8)\n\n    temperature_features = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        data_quality=True,\n    )\n    design_matrix = merge_features([usage_per_day, temperature_features])\n    return design_matrix\n\n\ndef create_caltrack_hourly_segmented_design_matrices(\n    preliminary_design_matrix,\n    segmentation,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    \"\"\"A helper function which calls basic feature creation methods to create a\n    design matrix suitable for use with segmented CalTRACK hourly models.\n    Parameters\n    ----------\n    preliminary_design_matrix : :any:`pandas.DataFrame`\n        A dataframe of the form returned by\n        :any:`eemeter.create_caltrack_hourly_preliminary_design_matrix`.\n    segmentation : :any:`pandas.DataFrame`\n        Weights for each segment. This is a dataframe of the form returned by\n        :any:`eemeter.segment_time_series` on the `preliminary_design_matrix`.\n    occupancy_lookup : any:`pandas.DataFrame`\n        Occupancy for each segment. This is a dataframe of the form returned by\n        :any:`eemeter.estimate_hour_of_week_occupancy`.\n    occupied_temperature_bins : :any:``\n        Occupied temperature bin settings for each segment. This is a dataframe of the\n        form returned by :any:`eemeter.fit_temperature_bins`.\n    unoccupied_temperature_bins : :any:``\n        Ditto, for unoccupied.\n    Returns\n    -------\n    design_matrix : :any:`dict` of :any:`pandas.DataFrame`\n        A dict of design matrixes created using the\n        :any:`eemeter.caltrack_hourly_fit_feature_processor`.\n    \"\"\"\n    return {\n        segment_name: segmented_data\n        for segment_name, segmented_data in iterate_segmented_dataset(\n            preliminary_design_matrix,\n            segmentation=segmentation,\n            feature_processor=caltrack_hourly_fit_feature_processor,\n            feature_processor_kwargs={\n                \"occupancy_lookup\": occupancy_lookup,\n                \"occupied_temperature_bins\": occupied_temperature_bins,\n                \"unoccupied_temperature_bins\": unoccupied_temperature_bins,\n            },\n        )\n    }\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/metrics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nfrom scipy.stats import t\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n__all__ = (\"ModelMetrics\",)\n\n\ndef _compute_r_squared(combined):\n    return combined[[\"predicted\", \"observed\"]].corr().iloc[0, 1] ** 2\n\n\ndef _compute_r_squared_adj(r_squared, length, num_parameters):\n    return 1 - (1 - r_squared) * (length - 1) / (length - num_parameters - 1)\n\n\ndef _compute_rmse(combined):\n    return (combined[\"residuals\"].astype(float) ** 2).mean() ** 0.5\n\n\ndef _compute_rmse_adj(combined, length, num_parameters):\n    if length > num_parameters:\n        return (\n            (combined[\"residuals\"].astype(float) ** 2).sum() / (length - num_parameters)\n        ) ** 0.5\n    else:\n        return np.nan\n\n\ndef _compute_cvrmse(rmse, observed_mean):\n    return rmse / observed_mean\n\n\ndef _compute_cvrmse_adj(rmse_adj, observed_mean):\n    return rmse_adj / observed_mean\n\n\ndef _compute_mape(combined):\n    return (combined[\"residuals\"] / combined[\"observed\"]).abs().mean()\n\n\ndef _compute_nmae(combined):\n    return (combined[\"residuals\"].astype(float).abs().sum()) / (\n        combined[\"observed\"].sum()\n    )\n\n\ndef _compute_nmbe(combined):\n    return combined[\"residuals\"].astype(float).sum() / combined[\"observed\"].sum()\n\n\ndef _compute_autocorr_resid(combined, autocorr_lags):\n    return combined[\"residuals\"].autocorr(lag=autocorr_lags)\n\n\ndef _json_safe_float(number):\n    \"\"\"\n    JSON serialization for infinity can be problematic.\n    See https://docs.python.org/2/library/json.html#basic-usage\n    This function returns None if `number` is infinity or negative infinity.\n\n    If the `number` cannot be converted to float, this will raise an exception.\n    \"\"\"\n    if number is None:\n        return None\n\n    if isinstance(number, float):\n        return None if np.isinf(number) or np.isnan(number) else number\n\n    # errors if number is not float compatible\n    return float(number)\n\n\nclass ModelMetricsFromJson(object):\n    def __init__(\n        self,\n        observed_length,\n        predicted_length,\n        merged_length,\n        num_parameters,\n        observed_mean,\n        predicted_mean,\n        observed_variance,\n        predicted_variance,\n        observed_skew,\n        predicted_skew,\n        observed_kurtosis,\n        predicted_kurtosis,\n        observed_cvstd,\n        predicted_cvstd,\n        r_squared,\n        r_squared_adj,\n        rmse,\n        rmse_adj,\n        cvrmse,\n        cvrmse_adj,\n        mape,\n        mape_no_zeros,\n        num_meter_zeros,\n        nmae,\n        nmbe,\n        autocorr_resid,\n        confidence_level,\n        n_prime,\n        single_tailed_confidence_level,\n        degrees_of_freedom,\n        t_stat,\n        cvrmse_auto_corr_correction,\n        approx_factor_auto_corr_correction,\n        fsu_base_term,\n    ):\n        self.observed_length = observed_length\n        self.predicted_length = predicted_length\n        self.merged_length = merged_length\n        self.num_parameters = num_parameters\n        self.observed_mean = observed_mean\n        self.predicted_mean = predicted_mean\n        self.observed_variance = observed_variance\n        self.predicted_variance = predicted_variance\n        self.observed_skew = observed_skew\n        self.predicted_skew = predicted_skew\n        self.observed_kurtosis = observed_kurtosis\n        self.predicted_kurtosis = predicted_kurtosis\n        self.observed_cvstd = observed_cvstd\n        self.predicted_cvstd = predicted_cvstd\n        self.r_squared = r_squared\n        self.r_squared_adj = r_squared_adj\n        self.rmse = rmse\n        self.rmse_adj = rmse_adj\n        self.cvrmse = cvrmse\n        self.cvrmse_adj = cvrmse_adj\n        self.mape = mape\n        self.mape_no_zeros = mape_no_zeros\n        self.num_meter_zeros = num_meter_zeros\n        self.nmae = nmae\n        self.nmbe = nmbe\n        self.autocorr_resid = autocorr_resid\n        self.confidence_level = confidence_level\n        self.n_prime = n_prime\n        self.single_tailed_confidence_level = single_tailed_confidence_level\n        self.degrees_of_freedom = degrees_of_freedom\n        self.t_stat = t_stat\n        self.cvrmse_auto_corr_correction = cvrmse_auto_corr_correction\n        self.approx_factor_auto_corr_correction = approx_factor_auto_corr_correction\n        self.fsu_base_term = fsu_base_term\n\n\nclass ModelMetrics(object):\n    \"\"\"Contains measures of model fit and summary statistics on the input series.\n\n    Parameters\n    ----------\n    observed_input : :any:`pandas.Series`\n        Series with :any:`pandas.DatetimeIndex` with a set of electricity or\n        gas meter values.\n    predicted_input : :any:`pandas.Series`\n        Series with :any:`pandas.DatetimeIndex` with a set of electricity or\n        gas meter values.\n    num_parameters : :any:`int`, optional\n        The number of parameters (excluding the intercept) used in the\n        regression from which the predictions were derived.\n    autocorr_lags : :any:`int`, optional\n        The number of lags to use when calculating the autocorrelation of the\n        residuals.\n    confidence_level : :any:`int`, optional\n        Confidence level used in fractional savings uncertainty computations.\n\n    Attributes\n    ----------\n    observed_length : :any:`int`\n        The length of the observed_input series.\n    predicted_length : :any:`int`\n        The length of the predicted_input series.\n    merged_length : :any:`int`\n        The length of the dataframe resulting from the inner join of the\n        observed_input series and the predicted_input series.\n    observed_mean : :any:`float`\n        The mean of the observed_input series.\n    predicted_mean : :any:`float`\n        The mean of the predicted_input series.\n    observed_skew : :any:`float`\n        The skew of the observed_input series.\n    predicted_skew : :any:`float`\n        The skew of the predicted_input series.\n    observed_kurtosis : :any:`float`\n        The excess kurtosis of the observed_input series.\n    predicted_kurtosis : :any:`float`\n        The excess kurtosis of the predicted_input series.\n    observed_cvstd : :any:`float`\n        The coefficient of standard deviation of the observed_input series.\n    predicted_cvstd : :any:`float`\n        The coefficient of standard deviation of the predicted_input series.\n    r_squared : :any:`float`\n        The r-squared of the model from which the predicted_input series was\n        produced.\n    r_squared_adj : :any:`float`\n        The r-squared of the predicted_input series relative to the\n        observed_input series, adjusted by the number of parameters in the model.\n    cvrmse : :any:`float`\n        The coefficient of variation (root-mean-squared error) of the\n        predicted_input series relative to the observed_input series.\n    cvrmse_adj : :any:`float`\n        The coefficient of variation (root-mean-squared error) of the\n        predicted_input series relative to the observed_input series, adjusted\n        by the number of parameters in the model.\n    mape : :any:`float`\n        The mean absolute percent error of the predicted_input series relative\n        to the observed_input series.\n    mape_no_zeros : :any:`float`\n        The mean absolute percent error of the predicted_input series relative\n        to the observed_input series, with all time periods dropped where the\n        observed_input series was not greater than zero.\n    num_meter_zeros : :any:`int`\n        The number of time periods for which the observed_input series was not\n        greater than zero.\n    nmae : :any:`float`\n        The normalized mean absolute error of the predicted_input series\n        relative to the observed_input series.\n    nmbe : :any:`float`\n        The normalized mean bias error of the predicted_input series relative\n        to the observed_input series.\n    autocorr_resid : :any:`float`\n        The autocorrelation of the residuals (where the residuals equal the\n        predicted_input series minus the observed_input series), measured\n        using a number of lags equal to autocorr_lags.\n    n_prime: :any:`float`\n        The number of baseline inputs corrected for autocorrelation -- used\n        in fractional savings uncertainty computation.\n    single_tailed_confidence_level: :any:`float`\n        The adjusted confidence level for use in single-sided tests.\n    degrees_of_freedom: :any:`float\n        Maxmimum number of independent variables which have the freedom to vary\n    t_stat: :any:`float\n        t-statistic, used for hypothesis testing\n    cvrmse_auto_corr_correction: :any:`float\n        Correctoin factor the apply to cvrmse to account for autocorrelation of inputs.\n    approx_factor_auto_corr_correction: :any:`float\n        Approximation factor used in ashrae 14 guideline for uncertainty computation.\n    fsu_base_term: :any:`float\n        Base term used in fractional savings uncertainty computation.\n\n    \"\"\"\n\n    def __init__(\n        self,\n        observed_input,\n        predicted_input,\n        num_parameters=1,\n        autocorr_lags=1,\n        confidence_level=0.90,\n    ):\n        if num_parameters < 0:\n            raise ValueError(\"num_parameters must be greater than or equal to zero\")\n        if autocorr_lags <= 0:\n            raise ValueError(\"autocorr_lags must be greater than zero\")\n        if (confidence_level <= 0) or (confidence_level >= 1):\n            raise ValueError(\"confidence_level must be between zero and one.\")\n\n        self.warnings = []\n\n        observed = observed_input.to_frame().dropna()\n        predicted = predicted_input.to_frame().dropna()\n        observed.columns = [\"observed\"]\n        predicted.columns = [\"predicted\"]\n\n        self.observed_length = observed.shape[0]\n        self.predicted_length = predicted.shape[0]\n\n        # Do an inner join on the two input series to make sure that we only\n        # use observations with the same time stamps.\n        combined = observed.merge(predicted, left_index=True, right_index=True)\n        self.merged_length = len(combined)\n\n        if self.observed_length != self.predicted_length:\n            self.warnings.append(\n                EEMeterWarning(\n                    qualified_name=\"eemeter.metrics.input_series_are_of_different_lengths\",\n                    description=\"Input series are of different lengths.\",\n                    data={\n                        \"observed_input_length\": len(observed_input),\n                        \"predicted_input_length\": len(predicted_input),\n                        \"observed_length_without_nan\": self.observed_length,\n                        \"predicted_length_without_nan\": self.predicted_length,\n                        \"merged_length\": self.merged_length,\n                    },\n                )\n            )\n\n        # Calculate residuals because these are an input for most of the metrics.\n        combined[\"residuals\"] = combined.predicted - combined.observed\n\n        self.num_parameters = num_parameters\n        self.autocorr_lags = autocorr_lags\n\n        # to account for solar usage the cvrmse should be calculated as the\n        # mean of the absolute value of observed.\n\n        self.observed_mean = abs(combined[\"observed\"]).mean()\n        self.predicted_mean = abs(combined[\"predicted\"]).mean()\n\n        self.observed_variance = combined[\"observed\"].var(ddof=0)\n        self.predicted_variance = combined[\"predicted\"].var(ddof=0)\n\n        self.observed_skew = combined[\"observed\"].skew()\n        self.predicted_skew = combined[\"predicted\"].skew()\n\n        self.observed_kurtosis = combined[\"observed\"].kurtosis()\n        self.predicted_kurtosis = combined[\"predicted\"].kurtosis()\n\n        self.r_squared = _compute_r_squared(combined)\n        self.r_squared_adj = _compute_r_squared_adj(\n            self.r_squared, self.merged_length, self.num_parameters\n        )\n\n        self.rmse = _compute_rmse(combined)\n        self.rmse_adj = _compute_rmse_adj(\n            combined, self.merged_length, self.num_parameters\n        )\n\n        with np.errstate(divide=\"ignore\", invalid=\"ignore\"):\n            self.observed_cvstd = combined[\"observed\"].std() / self.observed_mean\n            self.predicted_cvstd = combined[\"predicted\"].std() / self.predicted_mean\n\n            self.cvrmse = _compute_cvrmse(self.rmse, self.observed_mean)\n            self.cvrmse_adj = _compute_cvrmse_adj(self.rmse_adj, self.observed_mean)\n\n        # Create a new DataFrame with all rows removed where observed is\n        # zero, so we can calculate a version of MAPE with the zeros excluded.\n        # (Without the zeros excluded, MAPE becomes infinite when one observed\n        # value is zero.)\n        no_observed_zeros = combined[combined[\"observed\"] > 0]\n\n        self.mape = _compute_mape(combined)\n        self.mape_no_zeros = _compute_mape(no_observed_zeros)\n\n        self.num_meter_zeros = (self.merged_length) - no_observed_zeros.shape[0]\n\n        self.nmae = _compute_nmae(combined)\n\n        self.nmbe = _compute_nmbe(combined)\n\n        self.autocorr_resid = _compute_autocorr_resid(combined, autocorr_lags)\n\n        # ** Compute terms needed for fractional savings uncertainty computation.\n\n        self.confidence_level = confidence_level\n        self.n_prime = float(\n            self.observed_length * (1 - self.autocorr_resid) / (1 + self.autocorr_resid)\n        )\n        self.single_tailed_confidence_level = 1 - ((1 - self.confidence_level) / 2)\n\n        # convert to integer degrees of freedom, because n_prime could be non-integer\n        if pd.isnull(self.n_prime) or not np.isfinite(self.n_prime):\n            self.degrees_of_freedom = None\n            self.t_stat = None\n        else:\n            self.degrees_of_freedom = round(self.n_prime - self.num_parameters)\n            self.t_stat = t.ppf(\n                self.single_tailed_confidence_level, self.degrees_of_freedom\n            )\n\n        if (\n            self.n_prime == 0\n            or pd.isnull(self.n_prime)\n            or not np.isfinite(self.n_prime)\n            or self.n_prime - self.num_parameters == 0\n            or self.degrees_of_freedom < 1\n            or self.observed_length < self.num_parameters\n        ):\n            self.cvrmse_auto_corr_correction = None\n            self.approx_factor_auto_corr_correction = None\n            self.fsu_base_term = None\n        else:\n            # factor to correct cvrmse_adj for autocorrelation of inputs\n            # i.e., divide by (n' - n_param) instead of by (n - n_param)\n            self.cvrmse_auto_corr_correction = (\n                (self.observed_length - self.num_parameters)\n                / (self.n_prime - self.num_parameters)\n            ) ** 0.5\n\n            # part of approximation factor used in ashrae 14 guideline\n            self.approx_factor_auto_corr_correction = (\n                1.0 + (2.0 / self.n_prime)\n            ) ** 0.5\n\n            # all the following values are unitless\n            self.fsu_base_term = (\n                self.t_stat\n                * self.cvrmse_adj\n                * self.cvrmse_auto_corr_correction\n                * self.approx_factor_auto_corr_correction\n            )\n\n    def __repr__(self):\n        return (\n            \"ModelMetrics(merged_length={}, r_squared_adj={}, cvrmse_adj={}, \"\n            \"mape_no_zeros={}, nmae={}, nmbe={}, autocorr_resid={}, confidence_level={})\".format(\n                self.merged_length,\n                round(self.r_squared_adj, 3),\n                round(self.cvrmse_adj, 3),\n                round(self.mape_no_zeros, 3),\n                round(self.nmae, 3),\n                round(self.nmbe, 3),\n                round(self.autocorr_resid, 3),\n                round(self.confidence_level, 3),\n            )\n        )\n\n    def json(self):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n        return {\n            \"observed_length\": _json_safe_float(self.observed_length),\n            \"predicted_length\": _json_safe_float(self.predicted_length),\n            \"merged_length\": _json_safe_float(self.merged_length),\n            \"num_parameters\": _json_safe_float(self.num_parameters),\n            \"observed_mean\": _json_safe_float(self.observed_mean),\n            \"predicted_mean\": _json_safe_float(self.predicted_mean),\n            \"observed_variance\": _json_safe_float(self.observed_variance),\n            \"predicted_variance\": _json_safe_float(self.predicted_variance),\n            \"observed_skew\": _json_safe_float(self.observed_skew),\n            \"predicted_skew\": _json_safe_float(self.predicted_skew),\n            \"observed_kurtosis\": _json_safe_float(self.observed_kurtosis),\n            \"predicted_kurtosis\": _json_safe_float(self.predicted_kurtosis),\n            \"observed_cvstd\": _json_safe_float(self.observed_cvstd),\n            \"predicted_cvstd\": _json_safe_float(self.predicted_cvstd),\n            \"r_squared\": _json_safe_float(self.r_squared),\n            \"r_squared_adj\": _json_safe_float(self.r_squared_adj),\n            \"rmse\": _json_safe_float(self.rmse),\n            \"rmse_adj\": _json_safe_float(self.rmse_adj),\n            \"cvrmse\": _json_safe_float(self.cvrmse),\n            \"cvrmse_adj\": _json_safe_float(self.cvrmse_adj),\n            \"mape\": _json_safe_float(self.mape),\n            \"mape_no_zeros\": _json_safe_float(self.mape_no_zeros),\n            \"num_meter_zeros\": _json_safe_float(self.num_meter_zeros),\n            \"nmae\": _json_safe_float(self.nmae),\n            \"nmbe\": _json_safe_float(self.nmbe),\n            \"autocorr_resid\": _json_safe_float(self.autocorr_resid),\n            \"confidence_level\": _json_safe_float(self.confidence_level),\n            \"n_prime\": _json_safe_float(self.n_prime),\n            \"single_tailed_confidence_level\": _json_safe_float(\n                self.single_tailed_confidence_level\n            ),\n            \"degrees_of_freedom\": _json_safe_float(self.degrees_of_freedom),\n            \"t_stat\": _json_safe_float(self.t_stat),\n            \"cvrmse_auto_corr_correction\": _json_safe_float(\n                self.cvrmse_auto_corr_correction\n            ),\n            \"approx_factor_auto_corr_correction\": _json_safe_float(\n                self.approx_factor_auto_corr_correction\n            ),\n            \"fsu_base_term\": _json_safe_float(self.fsu_base_term),\n        }\n\n    @classmethod\n    def from_json(cls, data):\n        \"\"\"Loads a JSON-serializable representation into the model state.\n\n        The input of this function is a dict which can be the result\n        of :any:`json.loads`.\n        \"\"\"\n\n        c = ModelMetricsFromJson(\n            observed_length=data.get(\"observed_length\"),\n            predicted_length=data.get(\"predicted_length\"),\n            merged_length=data.get(\"merged_length\"),\n            num_parameters=data.get(\"num_parameters\"),\n            observed_mean=data.get(\"observed_mean\"),\n            predicted_mean=data.get(\"predicted_mean\"),\n            observed_variance=data.get(\"observed_variance\"),\n            predicted_variance=data.get(\"predicted_variance\"),\n            observed_skew=data.get(\"observed_skew\"),\n            predicted_skew=data.get(\"predicted_skew\"),\n            observed_kurtosis=data.get(\"observed_kurtosis\"),\n            predicted_kurtosis=data.get(\"predicted_kurtosis\"),\n            observed_cvstd=data.get(\"observed_cvstd\"),\n            predicted_cvstd=data.get(\"predicted_cvstd\"),\n            r_squared=data.get(\"r_squared\"),\n            r_squared_adj=data.get(\"r_squared_adj\"),\n            rmse=data.get(\"rmse\"),\n            rmse_adj=data.get(\"rmse_adj\"),\n            cvrmse=data.get(\"cvrmse\"),\n            cvrmse_adj=data.get(\"cvrmse_adj\"),\n            mape=data.get(\"mape\"),\n            mape_no_zeros=data.get(\"mape_no_zeros\"),\n            num_meter_zeros=data.get(\"num_meter_zeros\"),\n            nmae=data.get(\"nmae\"),\n            nmbe=data.get(\"nmbe\"),\n            autocorr_resid=data.get(\"autocorr_resid\"),\n            confidence_level=data.get(\"confidence_level\"),\n            n_prime=data.get(\"n_prime\"),\n            single_tailed_confidence_level=data.get(\"single_tailed_confidence_level\"),\n            degrees_of_freedom=data.get(\"degrees_of_freedom\"),\n            t_stat=data.get(\"t_stat\"),\n            cvrmse_auto_corr_correction=data.get(\"cvrmse_auto_corr_correction\"),\n            approx_factor_auto_corr_correction=data.get(\n                \"approx_factor_auto_corr_correction\"\n            ),\n            fsu_base_term=data.get(\"fsu_base_term\"),\n        )\n\n        return c\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom io import StringIO\nimport pandas as pd\nimport statsmodels.formula.api as smf\n\nfrom opendsm.eemeter.common.features import (\n    compute_occupancy_feature,\n    compute_temperature_bin_features,\n    compute_time_features,\n    merge_features,\n)\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\nfrom opendsm.eemeter.models.hourly_caltrack.metrics import ModelMetrics\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import (\n    CalTRACKSegmentModel,\n    SegmentedModel,\n    fit_model_segments,\n)\n\n__all__ = (\n    \"CalTRACKHourlyModelResults\",\n    \"CalTRACKHourlyModel\",\n    \"caltrack_hourly_fit_feature_processor\",\n    \"caltrack_hourly_prediction_feature_processor\",\n    \"fit_caltrack_hourly_model_segment\",\n    \"fit_caltrack_hourly_model\",\n)\n\n\nclass CalTRACKHourlyModelResults(object):\n    \"\"\"Contains information about the chosen model.\n\n    Attributes\n    ----------\n    status : :any:`str`\n        A string indicating the status of this result. Possible statuses:\n\n        - ``'NO DATA'``: No baseline data was available.\n        - ``'NO MODEL'``: A complete model could not be constructed.\n        - ``'SUCCESS'``: A model was constructed.\n    method_name : :any:`str`\n        The name of the method used to fit the baseline model.\n    model : :any:`eemeter.CalTRACKHourlyModel` or :any:`None`\n        The selected model, if any.\n    warnings : :any:`list` of :any:`eemeter.EEMeterWarning`\n        A list of any warnings reported during the model selection and fitting\n        process.\n    metadata : :any:`dict`\n        An arbitrary dictionary of metadata to be associated with this result.\n        This can be used, for example, to tag the results with attributes like\n        an ID::\n\n            {\n                'id': 'METER_12345678',\n            }\n\n    settings : :any:`dict`\n        A dictionary of settings used by the method.\n    totals_metrics : :any:`ModelMetrics`\n        A ModelMetrics object, if one is calculated and associated with this\n        model. (This initializes to None.) The ModelMetrics object contains\n        model fit information and descriptive statistics about the underlying data,\n        with that data expressed as period totals.\n    avgs_metrics : :any:`ModelMetrics`\n        A ModelMetrics object, if one is calculated and associated with this\n        model. (This initializes to None.) The ModelMetrics object contains\n        model fit information and descriptive statistics about the underlying data,\n        with that data expressed as daily averages.\n    \"\"\"\n\n    def __init__(\n        self, status, method_name, model=None, warnings=[], metadata=None, settings=None\n    ):\n        self.status = status\n        self.method_name = method_name\n\n        self.model = model\n\n        self.warnings = warnings\n\n        if metadata is None:\n            metadata = {}\n        self.metadata = metadata\n\n        if settings is None:\n            settings = {}\n        self.settings = settings\n\n        self.totals_metrics = None\n        self.avgs_metrics = None\n\n    def __repr__(self):\n        return \"CalTRACKHourlyModelResults(status='{}', method_name='{}')\".format(\n            self.status, self.method_name\n        )\n\n    def json(self, with_candidates=False):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n\n        def _json_or_none(obj):\n            return None if obj is None else obj.json()\n\n        def _json_or_none_in_dict(obj):\n            return (\n                None\n                if obj is None\n                else {key: _json_or_none(val) for key, val in obj.items()}\n            )\n\n        data = {\n            \"status\": self.status,\n            \"method_name\": self.method_name,\n            \"model\": _json_or_none(self.model),\n            \"warnings\": [w.json() for w in self.warnings],\n            \"metadata\": self.metadata,\n            \"settings\": self.settings,\n            \"totals_metrics\": _json_or_none_in_dict(self.totals_metrics),\n            \"avgs_metrics\": _json_or_none_in_dict(self.avgs_metrics),\n        }\n        return data\n\n    @classmethod\n    def from_json(cls, data):\n        \"\"\"Loads a JSON-serializable representation into the model state.\n\n        The input of this function is a dict which can be the result\n        of :any:`json.loads`.\n        \"\"\"\n\n        # \"model\" is a CalTRACKHourlyModel that was serialized\n        model = None\n        d = data.get(\"model\")\n        if d:\n            model = CalTRACKHourlyModel.from_json(d)\n\n        c = cls(\n            data.get(\"status\"),\n            data.get(\"method_name\"),\n            model=model,\n            warnings=data.get(\"warnings\"),\n            metadata=data.get(\"metadata\"),\n            settings=data.get(\"settings\"),\n        )\n\n        # Note the metrics do not contain all the data needed\n        # for reconstruction (like the input pandas) ...\n        d = data.get(\"avgs_metrics\")\n        if d:\n            c.avgs_metrics = ModelMetrics.from_json(d)  # pragma: no cover\n            c.avgs_metrics = {\n                segment_name: ModelMetrics.from_json(seg_d)\n                for segment_name, seg_d in d.items()\n            }\n        d = data.get(\"totals_metrics\")\n        if d:\n            c.totals_metrics = ModelMetrics.from_json(d)  # pragma: no cover\n            c.totals_metrics = {\n                segment_name: ModelMetrics.from_json(seg_d)\n                for segment_name, seg_d in d.items()\n            }\n        return c\n\n    def predict(self, prediction_index, temperature_data, **kwargs):\n        \"\"\"Predict over a particular index using temperature data.\n\n        Parameters\n        ----------\n        prediction_index : :any:`pandas.DatetimeIndex`\n            Time period over which to predict.\n        temperature_data : :any:`pandas.DataFrame`\n            Hourly temperature data to use for prediction. Time period should match\n            the ``prediction_index`` argument.\n        **kwargs\n            Extra keyword arguments to send to self.model.predict\n\n        Returns\n        -------\n        prediction : :any:`pandas.DataFrame`\n            The predicted usage values.\n        \"\"\"\n        return self.model.predict(prediction_index, temperature_data, **kwargs)\n\n\nclass _PredictionSegmentInfo:\n    \"\"\"\n    Class to handle the different segment_type parameters\n    that provides the correct values to the CalTrackHourlyModel initialization.\n    \"\"\"\n\n    def __init__(self, segment_type: str):\n        if segment_type not in [\"single\", \"three_month_weighted\"]:\n            raise ValueError(\"segment type must be single or three_month_weighted\")\n\n        if segment_type == \"single\":\n            self.prediction_segment_type = segment_type\n            self.prediction_segment_name_mapping = None\n            return\n\n        if segment_type == \"three_month_weighted\":\n            self.prediction_segment_name_mapping = {\n                \"jan\": \"dec-jan-feb-weighted\",\n                \"feb\": \"jan-feb-mar-weighted\",\n                \"mar\": \"feb-mar-apr-weighted\",\n                \"apr\": \"mar-apr-may-weighted\",\n                \"may\": \"apr-may-jun-weighted\",\n                \"jun\": \"may-jun-jul-weighted\",\n                \"jul\": \"jun-jul-aug-weighted\",\n                \"aug\": \"jul-aug-sep-weighted\",\n                \"sep\": \"aug-sep-oct-weighted\",\n                \"oct\": \"sep-oct-nov-weighted\",\n                \"nov\": \"oct-nov-dec-weighted\",\n                \"dec\": \"nov-dec-jan-weighted\",\n            }\n            self.prediction_segment_type = \"one_month\"\n            return\n\n\nclass CalTRACKHourlyModel(SegmentedModel):\n    \"\"\"An object which holds CalTRACK Hourly model data and metadata, and\n    which can be used for prediction.\n\n    Attributes\n    ----------\n    segment_models : :any:`dict` of `eemeter.CalTRACKSegmentModel`\n        Dictionary of models for each segment, keys are segment names.\n    occupancy_lookup : :any:`pandas.DataFrame`\n        A dataframe with occupancy flags for each hour of the week and each segment.\n        Segment names are columns, occupancy flags are 0 or 1.\n    occupied_temperature_bins : :any:`pandas.DataFrame`\n        A dataframe of bin endpoint flags for each segment. Segment names are columns.\n    unoccupied_temperature_bins : :any:`pandas.DataFrame`\n        Ditto for the unoccupied mode.\n    segment_type : :any:`str`\n        The type of segment used to fit the model\n    \"\"\"\n\n    def __init__(\n        self,\n        segment_models,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type: str,\n    ):\n        self.occupancy_lookup = occupancy_lookup\n        self.occupied_temperature_bins = occupied_temperature_bins\n        self.unoccupied_temperature_bins = unoccupied_temperature_bins\n        self.segment_type = segment_type\n\n        prediction_info = _PredictionSegmentInfo(segment_type=segment_type)\n        super(CalTRACKHourlyModel, self).__init__(\n            segment_models=segment_models,\n            prediction_segment_type=prediction_info.prediction_segment_type,\n            prediction_segment_name_mapping=prediction_info.prediction_segment_name_mapping,\n            prediction_feature_processor=caltrack_hourly_prediction_feature_processor,\n            prediction_feature_processor_kwargs={\n                \"occupancy_lookup\": self.occupancy_lookup,\n                \"occupied_temperature_bins\": self.occupied_temperature_bins,\n                \"unoccupied_temperature_bins\": self.unoccupied_temperature_bins,\n            },\n        )\n\n    def json(self):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n        data = super(CalTRACKHourlyModel, self).json()\n        data.update(\n            {\n                \"occupancy_lookup\": self.occupancy_lookup.to_json(orient=\"split\"),\n                \"occupied_temperature_bins\": self.occupied_temperature_bins.to_json(\n                    orient=\"split\"\n                ),\n                \"unoccupied_temperature_bins\": self.unoccupied_temperature_bins.to_json(\n                    orient=\"split\"\n                ),\n                \"segment_type\": self.segment_type,\n            }\n        )\n        return data\n\n    @classmethod\n    def from_json(cls, data):\n        \"\"\"Loads a JSON-serializable representation into the model state.\n\n        The input of this function is a dict which can be the result\n        of :any:`json.loads`.\n        \"\"\"\n\n        segment_models = [\n            CalTRACKSegmentModel.from_json(s) for s in data.get(\"segment_models\")\n        ]\n\n        occupancy_lookup = pd.read_json(\n            StringIO(data.get(\"occupancy_lookup\")), orient=\"split\"\n        )\n        occupancy_lookup.index = occupancy_lookup.index.astype(\"category\")\n\n        c = cls(\n            segment_models,\n            occupancy_lookup,\n            pd.read_json(\n                StringIO(data.get(\"occupied_temperature_bins\")), orient=\"split\"\n            ),\n            pd.read_json(\n                StringIO(data.get(\"unoccupied_temperature_bins\")), orient=\"split\"\n            ),\n            data.get(\"segment_type\"),\n        )\n\n        return c\n\n\ndef caltrack_hourly_fit_feature_processor(\n    segment_name,\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    \"\"\"A function that takes in temperature data and returns a dataframe of\n    features suitable for use with :any:`eemeter.fit_caltrack_hourly_model_segment`.\n    Designed for use with :any:`eemeter.iterate_segmented_dataset`.\n\n    Parameters\n    ----------\n    segment_name : :any:`str`\n        The name of the segment.\n    segmented_data : :any:`pandas.DataFrame`\n        Hourly temperature data for the segment.\n    occupancy_lookup : :any:`pandas.DataFrame`\n        A dataframe with occupancy flags for each hour of the week and each segment.\n        Segment names are columns, occupancy flags are 0 or 1.\n    occupied_temperature_bins : :any:`pandas.DataFrame`\n        A dataframe of bin endpoint flags for each segment. Segment names are columns.\n    unoccupied_temperature_bins : :any:`pandas.DataFrame`\n        Ditto for the unoccupied mode.\n\n    Returns\n    -------\n    features : :any:`pandas.DataFrame`\n        A dataframe of features with the following columns:\n\n        - 'meter_value': the observed meter value\n        - 'hour_of_week': 0-167\n        - 'bin_<0-6>_occupied': temp bin feature, or 0 if unoccupied\n        - 'bin_<0-6>_unoccupied': temp bin feature or 0 in occupied\n        - 'weight': 0.0 or 0.5 or 1.0\n    \"\"\"\n    # get occupied feature\n    hour_of_week = segmented_data.hour_of_week\n    occupancy = occupancy_lookup[segment_name]\n    occupancy_feature = compute_occupancy_feature(hour_of_week, occupancy)\n\n    # get temperature bin features\n    temperatures = segmented_data.temperature_mean\n    occupied_bin_endpoints_list = (\n        occupied_temperature_bins[segment_name]\n        .index[occupied_temperature_bins[segment_name]]\n        .tolist()\n    )\n    unoccupied_bin_endpoints_list = (\n        unoccupied_temperature_bins[segment_name]\n        .index[unoccupied_temperature_bins[segment_name]]\n        .tolist()\n    )\n    occupied_temperature_bin_features = compute_temperature_bin_features(\n        segmented_data.temperature_mean, occupied_bin_endpoints_list\n    )\n    occupied_temperature_bin_features[occupancy_feature == 0] = 0\n    occupied_temperature_bin_features.rename(\n        columns={\n            c: \"{}_occupied\".format(c)\n            for c in occupied_temperature_bin_features.columns\n        },\n        inplace=True,\n    )\n    unoccupied_temperature_bin_features = compute_temperature_bin_features(\n        segmented_data.temperature_mean, unoccupied_bin_endpoints_list\n    )\n    unoccupied_temperature_bin_features[occupancy_feature == 1] = 0\n    unoccupied_temperature_bin_features.rename(\n        columns={\n            c: \"{}_unoccupied\".format(c)\n            for c in unoccupied_temperature_bin_features.columns\n        },\n        inplace=True,\n    )\n\n    # combine features\n    return merge_features(\n        [\n            segmented_data[[\"meter_value\", \"hour_of_week\"]],\n            occupied_temperature_bin_features,\n            unoccupied_temperature_bin_features,\n            segmented_data.weight,\n        ]\n    )\n\n\ndef caltrack_hourly_prediction_feature_processor(\n    segment_name,\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    \"\"\"A function that takes in temperature data and returns a dataframe of\n    features suitable for use inside :any:`eemeter.CalTRACKHourlyModel`.\n    Designed for use with :any:`eemeter.iterate_segmented_dataset`.\n\n    Parameters\n    ----------\n    segment_name : :any:`str`\n        The name of the segment.\n    segmented_data : :any:`pandas.DataFrame`\n        Hourly temperature data for the segment.\n    occupancy_lookup : :any:`pandas.DataFrame`\n        A dataframe with occupancy flags for each hour of the week and each segment.\n        Segment names are columns, occupancy flags are 0 or 1.\n    occupied_temperature_bins : :any:`pandas.DataFrame`\n        A dataframe of bin endpoint flags for each segment. Segment names are columns.\n    unoccupied_temperature_bins : :any:`pandas.DataFrame`\n        Ditto for the unoccupied mode.\n\n    Returns\n    -------\n    features : :any:`pandas.DataFrame`\n        A dataframe of features with the following columns:\n\n        - 'hour_of_week': 0-167\n        - 'bin_<0-6>_occupied': temp bin feature, or 0 if unoccupied\n        - 'bin_<0-6>_unoccupied': temp bin feature or 0 in occupied\n        - 'weight': 1\n    \"\"\"\n    # hour of week feature\n    hour_of_week_feature = compute_time_features(\n        segmented_data.index, hour_of_week=True, day_of_week=False, hour_of_day=False\n    )\n\n    # occupancy feature\n    occupancy = occupancy_lookup[segment_name]\n    occupancy_feature = compute_occupancy_feature(\n        hour_of_week_feature.hour_of_week, occupancy\n    )\n\n    # get temperature bin features\n    temperatures = segmented_data\n    occupied_bin_endpoints_list = (\n        occupied_temperature_bins[segment_name]\n        .index[occupied_temperature_bins[segment_name]]\n        .tolist()\n    )\n    unoccupied_bin_endpoints_list = (\n        unoccupied_temperature_bins[segment_name]\n        .index[unoccupied_temperature_bins[segment_name]]\n        .tolist()\n    )\n    occupied_temperature_bin_features = compute_temperature_bin_features(\n        segmented_data.temperature_mean, occupied_bin_endpoints_list\n    )\n    occupied_temperature_bin_features[occupancy_feature == 0] = 0\n    occupied_temperature_bin_features.rename(\n        columns={\n            c: \"{}_occupied\".format(c)\n            for c in occupied_temperature_bin_features.columns\n        },\n        inplace=True,\n    )\n    unoccupied_temperature_bin_features = compute_temperature_bin_features(\n        segmented_data.temperature_mean, unoccupied_bin_endpoints_list\n    )\n    unoccupied_temperature_bin_features[occupancy_feature == 1] = 0\n    unoccupied_temperature_bin_features.rename(\n        columns={\n            c: \"{}_unoccupied\".format(c)\n            for c in unoccupied_temperature_bin_features.columns\n        },\n        inplace=True,\n    )\n\n    # combine features\n    return merge_features(\n        [\n            hour_of_week_feature,\n            occupied_temperature_bin_features,\n            unoccupied_temperature_bin_features,\n            segmented_data.weight,\n        ]\n    )\n\n\ndef fit_caltrack_hourly_model_segment(segment_name, segment_data):\n    \"\"\"Fit a model for a single segment.\n\n    Parameters\n    ----------\n    segment_name : :any:`str`\n        The name of the segment.\n    segment_data : :any:`pandas.DataFrame`\n        A design matrix for caltrack hourly, of the form returned by\n        :any:`eemeter.caltrack_hourly_prediction_feature_processor`.\n\n    Returns\n    -------\n    segment_model : :any:`CalTRACKSegmentModel`\n        A model that represents the fitted model.\n    \"\"\"\n\n    warnings = []\n    if segment_data.dropna().empty:\n        model = None\n        formula = None\n        model_params = None\n        warnings.append(\n            EEMeterWarning(\n                qualified_name=\"eemeter.fit_caltrack_hourly_model_segment.no_nonnull_data\",\n                description=\"The segment contains either an empty dataset or all NaNs.\",\n                data={\n                    \"n_rows\": segment_data.shape[0],\n                    \"n_rows_after_dropna\": segment_data.dropna().shape[0],\n                },\n            )\n        )\n\n    else:\n\n        def _get_hourly_model_formula(data):\n            return \"meter_value ~ C(hour_of_week) - 1{}\".format(\n                \"\".join(\n                    [\" + {}\".format(c) for c in data.columns if c.startswith(\"bin\")]\n                )\n            )\n\n        formula = _get_hourly_model_formula(segment_data)\n\n        # remove categories that only have null or missing entries\n        # this ensures that predictions will predict null\n        segment_data[\"hour_of_week\"] = pd.Categorical(\n            segment_data[\"hour_of_week\"],\n            categories=segment_data[\"hour_of_week\"].dropna().unique(),\n            ordered=False,\n        )\n        model = smf.wls(formula=formula, data=segment_data, weights=segment_data.weight)\n        model_params = {coeff: value for coeff, value in model.fit().params.items()}\n\n    segment_model = CalTRACKSegmentModel(\n        segment_name=segment_name,\n        model=model,\n        formula=formula,\n        model_params=model_params,\n        warnings=warnings,\n    )\n    if model:\n        this_segment_data = segment_data[segment_data.weight == 1]\n        predicted_value = pd.Series(model.fit().predict(this_segment_data))\n        segment_model.totals_metrics = ModelMetrics(\n            this_segment_data.meter_value, predicted_value, len(model_params)\n        )\n    else:\n        segment_model.totals_metrics = None\n\n    return segment_model\n\n\ndef fit_caltrack_hourly_model(\n    segmented_design_matrices,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n    segment_type: str,\n):\n    \"\"\"Fit a CalTRACK hourly model\n\n    Parameters\n    ----------\n    segmented_design_matrices : :any:`dict` of :any:`pandas.DataFrame`\n        A dictionary of dataframes of the form returned by\n        :any:`eemeter.create_caltrack_hourly_segmented_design_matrices`\n    occupancy_lookup : :any:`pandas.DataFrame`\n        A dataframe with occupancy flags for each hour of the week and each segment.\n        Segment names are columns, occupancy flags are 0 or 1.\n    occupied_temperature_bins : :any:`pandas.DataFrame`\n        A dataframe of bin endpoint flags for each segment. Segment names are columns.\n    unoccupied_temperature_bins : :any:`pandas.DataFrame`\n        Ditto for the unoccupied mode.\n\n    Returns\n    -------\n    model : :any:`CalTRACKHourlyModelResults`\n        Has a `model.predict` method which take input data and makes a prediction\n        using this model.\n    \"\"\"\n    segment_models = fit_model_segments(\n        segmented_design_matrices, fit_caltrack_hourly_model_segment\n    )\n    all_warnings = [\n        warning\n        for segment_model in segment_models\n        for warning in segment_model.warnings\n    ]\n\n    model = CalTRACKHourlyModel(\n        segment_models,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type,\n    )\n\n    model_results = CalTRACKHourlyModelResults(\n        status=\"SUCCEEDED\",\n        method_name=\"caltrack_hourly\",\n        warnings=all_warnings,\n        model=model,\n    )\n    model_results.totals_metrics = {\n        seg_model.segment_name: seg_model.totals_metrics for seg_model in segment_models\n    }\n    return model_results\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/segmentation.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom collections import namedtuple\n\nimport pandas as pd\nfrom patsy import dmatrix\n\n__all__ = (\n    \"iterate_segmented_dataset\",\n    \"segment_time_series\",\n    \"CalTRACKSegmentModel\",\n    \"SegmentedModel\",\n    \"HourlyModelPrediction\",\n)\n\n\nHourlyModelPrediction = namedtuple(\"HourlyModelPrediction\", [\"result\"])\n\n\nclass CalTRACKSegmentModel(object):\n    \"\"\"An object that captures the model fit for one segment.\n\n    Attributes\n    ----------\n    segment_name : :any:`str`\n        The name of the segment of data this model was fit to.\n    model : :any:`object`\n        The fitted model object.\n    formula : :any:`str`\n        The formula of the model regression.\n    model_param : :any:`dict`\n        A dictionary of parameters\n    warnings : :any:`list`\n        A list of eemeter warnings.\n    \"\"\"\n\n    def __init__(self, segment_name, model, formula, model_params, warnings=None):\n        self.segment_name = segment_name\n        self.model = model\n        self.formula = formula\n        self.model_params = model_params\n\n        if warnings is None:\n            warnings = []\n        self.warnings = warnings\n\n    def predict(self, data):\n        \"\"\"A function which takes input data and predicts for this segment model.\"\"\"\n        if self.formula is None:\n            var_str = \"\"\n        else:\n            var_str = self.formula.split(\"~\", 1)[1]\n\n        design_matrix_granular = dmatrix(var_str, data, return_type=\"dataframe\")\n        parameters = pd.Series(self.model_params)\n\n        # Step 1, slice\n        col_type = \"C(hour_of_week)\"\n        hour_of_week_cols = [\n            c\n            for c in design_matrix_granular.columns\n            if col_type in c and c in parameters.keys()\n        ]\n\n        # Step 2, cut out all 0s\n        design_matrix_granular = design_matrix_granular[\n            (design_matrix_granular[hour_of_week_cols] != 0).any(axis=1)\n        ]\n\n        cols_to_predict = list(\n            set(parameters.keys()).intersection(set(design_matrix_granular.keys()))\n        )\n        design_matrix_granular = design_matrix_granular[cols_to_predict]\n        parameters = parameters[cols_to_predict]\n\n        # Step 3, predict\n        prediction = design_matrix_granular.dot(parameters).rename(\"predicted_usage\")\n\n        # Step 4, put nans back in\n        prediction = prediction.reindex(data.index)\n\n        return prediction\n\n    def json(self):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n\n        data = {\n            \"segment_name\": self.segment_name,\n            \"formula\": self.formula,\n            \"warnings\": [w.json() for w in self.warnings],\n            \"model_params\": self.model_params,\n        }\n        return data\n\n    @classmethod\n    def from_json(cls, data):\n        \"\"\"Loads a JSON-serializable representation into the model state.\n\n        The input of this function is a dict which can be the result\n        of :any:`json.loads`.\n        \"\"\"\n\n        c = cls(\n            data.get(\"segment_name\"),\n            None,\n            data.get(\"formula\"),\n            data.get(\"model_params\"),\n            warnings=data.get(\"warnings\"),\n        )\n\n        return c\n\n\nclass SegmentedModel(object):\n    \"\"\"Represent a model which has been broken into multiple model segments (for\n    CalTRACK Hourly, these are month-by-month segments, each of which is associated\n    with a different model.\n\n    Parameters\n    ----------\n    segment_models : :any:`dict` of :any:`eemeter.CalTRACKSegmentModel`\n        Dictionary of segment models, keyed by segment name.\n    prediction_segment_type : :any:`str`\n        Any segment_type that can be passed to :any:`eemeter.segment_time_series`,\n        currently \"single\", \"one_month\", \"three_month\", or \"three_month_weighted\".\n    prediction_segment_name_mapping : :any:`dict` of :any:`str`\n        A dictionary mapping the segment names for the segment type used for predicting to the\n        segment names for the segment type used for fitting,\n        e.g., `{\"<predict_segment_name>\": \"<fit_segment_name>\"}`.\n    prediction_feature_processor : :any:`function`\n        A function that transforms raw inputs (temperatures) into features for each\n        segment.\n    prediction_feature_processor_kwargs : :any:`dict`\n        A dict of keyword arguments to be passed as `**kwargs` to the\n        `prediction_feature_processor` function.\n    \"\"\"\n\n    def __init__(\n        self,\n        segment_models,\n        prediction_segment_type,\n        prediction_segment_name_mapping=None,\n        prediction_feature_processor=None,\n        prediction_feature_processor_kwargs=None,\n    ):\n        self.segment_models = segment_models\n\n        fitted_model_lookup = {\n            segment_model.segment_name: segment_model\n            for segment_model in segment_models\n        }\n        if prediction_segment_name_mapping is None:\n            self.model_lookup = fitted_model_lookup\n        else:\n            self.model_lookup = {\n                pred_name: fitted_model_lookup.get(fit_name)\n                for pred_name, fit_name in prediction_segment_name_mapping.items()\n            }\n        self.prediction_segment_type = prediction_segment_type\n        self.prediction_segment_name_mapping = prediction_segment_name_mapping\n        self.prediction_feature_processor = prediction_feature_processor\n        self.prediction_feature_processor_kwargs = prediction_feature_processor_kwargs\n\n    def predict(\n        self, prediction_index, temperature, **kwargs\n    ):  # ignore extra args with kwargs\n        \"\"\"Predict over a prediction index by combining results from all models.\n\n        Parameters\n        ----------\n        prediction_index : :any:`pandas.DatetimeIndex`\n            The index over which to predict.\n        temperature : :any:`pandas.Series`\n            Hourly temperatures.\n        **kwargs\n            Extra argmuents will be ignored\n        \"\"\"\n        prediction_segmentation = segment_time_series(\n            temperature.index,\n            self.prediction_segment_type,\n            drop_zero_weight_segments=True,\n        )\n\n        iterator = iterate_segmented_dataset(\n            temperature.to_frame(\"temperature_mean\"),\n            segmentation=prediction_segmentation,\n            feature_processor=self.prediction_feature_processor,\n            feature_processor_kwargs=self.prediction_feature_processor_kwargs,\n            feature_processor_segment_name_mapping=self.prediction_segment_name_mapping,\n        )\n\n        predictions = {}\n        for segment_name, segmented_data in iterator:\n            segment_model = self.model_lookup.get(segment_name)\n            if segment_model is None:\n                continue\n            prediction = segment_model.predict(segmented_data) * segmented_data.weight\n            # NaN the zero weights and reindex\n            prediction = prediction[segmented_data.weight > 0].reindex(prediction_index)\n            predictions[segment_name] = prediction\n\n        predictions = pd.DataFrame(predictions)\n        result = pd.DataFrame({\"predicted_usage\": predictions.sum(axis=1, min_count=1)})\n        return HourlyModelPrediction(result=result)\n\n    def json(self):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n\n        def _json_or_none(obj):\n            return None if obj is None else obj.json()\n\n        data = {\n            \"segment_models\": [_json_or_none(m) for m in self.segment_models],\n            \"model_lookup\": {\n                key: _json_or_none(val) for key, val in self.model_lookup.items()\n            },\n            \"prediction_segment_type\": self.prediction_segment_type,\n            \"prediction_segment_name_mapping\": self.prediction_segment_name_mapping,\n            \"prediction_feature_processor\": self.prediction_feature_processor.__name__,\n        }\n        return data\n\n\ndef filter_zero_weights_feature_processor(segment_name, segment_data):\n    \"\"\"A default segment processor to use if none is provided.\"\"\"\n    return segment_data[segment_data.weight > 0]\n\n\ndef iterate_segmented_dataset(\n    data,\n    segmentation=None,\n    feature_processor=None,\n    feature_processor_kwargs=None,\n    feature_processor_segment_name_mapping=None,\n):\n    \"\"\"A utility for iterating over segments which allows providing a function for\n    processing outputs into features.\n\n    Parameters\n    ----------\n    data : :any:`pandas.DataFrame`, required\n        Data to segment,\n    segmentation : :any:`pandas.DataFrame`, default None\n        A segmentation of the input dataframe expressed as a dataframe which shares\n        the timeseries index of the data and has named columns of weights, which\n        are iterated over to create the outputs (or inputs to the feature processor,\n        which then creates the actual outputs).\n    feature_processor : :any:`function`, default None\n        A function that transforms raw inputs (temperatures) into features for each\n        segment.\n    feature_processor_kwargs : :any:`dict`, default None\n        A dict of keyword arguments to be passed as `**kwargs` to the\n        `feature_processor` function.\n    feature_processor_segment_name_mapping : :any:`dict`, default None\n        A mapping from the default segmentation segment names to alternate names. This\n        is useful when prediction uses a different segment type than fitting.\n    \"\"\"\n    if feature_processor is None:\n        feature_processor = filter_zero_weights_feature_processor\n\n    if feature_processor_kwargs is None:\n        feature_processor_kwargs = {}\n\n    if feature_processor_segment_name_mapping is None:\n        feature_processor_segment_name_mapping = {}\n\n    def _apply_feature_processor(segment_name, segment_data):\n        feature_processor_segment_name = feature_processor_segment_name_mapping.get(\n            segment_name, segment_name\n        )\n\n        if feature_processor is not None:\n            segment_data = feature_processor(\n                feature_processor_segment_name, segment_data, **feature_processor_kwargs\n            )\n        return segment_data\n\n    def _add_weights(data, weights):\n        return pd.merge(data, weights, left_index=True, right_index=True)\n\n    if segmentation is None:\n        # spoof segment name and weights column\n        segment_name = None\n        weights = pd.DataFrame({\"weight\": 1}, index=data.index)\n        segment_data = _add_weights(data, weights)\n\n        segment_data = _apply_feature_processor(segment_name, segment_data)\n        yield segment_name, segment_data\n    else:\n        for segment_name, segment_weights in segmentation.items():\n            weights = segment_weights.to_frame(\"weight\")\n            segment_data = _add_weights(data, weights)\n            segment_data = _apply_feature_processor(segment_name, segment_data)\n            yield segment_name, segment_data\n\n\ndef _get_calendar_year_coverage_warning(index):\n    pass\n\n\ndef _get_hourly_coverage_warning(index, min_fraction_daily_coverage=0.9):\n    pass\n\n\ndef _segment_weights_single(index):\n    return pd.DataFrame({\"all\": 1.0}, index=index)\n\n\ndef _segment_weights_one_month(index):\n    return pd.DataFrame(\n        {\n            month_name: (index.month == month_number).astype(float)\n            for month_name, month_number in [\n                (\"jan\", 1),\n                (\"feb\", 2),\n                (\"mar\", 3),\n                (\"apr\", 4),\n                (\"may\", 5),\n                (\"jun\", 6),\n                (\"jul\", 7),\n                (\"aug\", 8),\n                (\"sep\", 9),\n                (\"oct\", 10),\n                (\"nov\", 11),\n                (\"dec\", 12),\n            ]\n        },\n        index=index,\n        columns=[\n            \"jan\",\n            \"feb\",\n            \"mar\",\n            \"apr\",\n            \"may\",\n            \"jun\",\n            \"jul\",\n            \"aug\",\n            \"sep\",\n            \"oct\",\n            \"nov\",\n            \"dec\",\n        ],  # guarantee order\n    )\n\n\ndef _segment_weights_three_month(index):\n    return pd.DataFrame(\n        {\n            month_names: (index.month.map(lambda i: i in month_numbers)).astype(float)\n            for month_names, month_numbers in [\n                (\"dec-jan-feb\", (12, 1, 2)),\n                (\"jan-feb-mar\", (1, 2, 3)),\n                (\"feb-mar-apr\", (2, 3, 4)),\n                (\"mar-apr-may\", (3, 4, 5)),\n                (\"apr-may-jun\", (4, 5, 6)),\n                (\"may-jun-jul\", (5, 6, 7)),\n                (\"jun-jul-aug\", (6, 7, 8)),\n                (\"jul-aug-sep\", (7, 8, 9)),\n                (\"aug-sep-oct\", (8, 9, 10)),\n                (\"sep-oct-nov\", (9, 10, 11)),\n                (\"oct-nov-dec\", (10, 11, 12)),\n                (\"nov-dec-jan\", (11, 12, 1)),\n            ]\n        },\n        index=index,\n        columns=[\n            \"dec-jan-feb\",\n            \"jan-feb-mar\",\n            \"feb-mar-apr\",\n            \"mar-apr-may\",\n            \"apr-may-jun\",\n            \"may-jun-jul\",\n            \"jun-jul-aug\",\n            \"jul-aug-sep\",\n            \"aug-sep-oct\",\n            \"sep-oct-nov\",\n            \"oct-nov-dec\",\n            \"nov-dec-jan\",\n        ],  # guarantee order\n    )\n\n\ndef _segment_weights_three_month_weighted(index):\n    return pd.DataFrame(\n        {\n            month_names: index.month.map(\n                lambda i: month_weights.get(str(i), 0.0)\n            ).astype(float)\n            for month_names, month_weights in [\n                (\"dec-jan-feb-weighted\", {\"12\": 0.5, \"1\": 1, \"2\": 0.5}),\n                (\"jan-feb-mar-weighted\", {\"1\": 0.5, \"2\": 1, \"3\": 0.5}),\n                (\"feb-mar-apr-weighted\", {\"2\": 0.5, \"3\": 1, \"4\": 0.5}),\n                (\"mar-apr-may-weighted\", {\"3\": 0.5, \"4\": 1, \"5\": 0.5}),\n                (\"apr-may-jun-weighted\", {\"4\": 0.5, \"5\": 1, \"6\": 0.5}),\n                (\"may-jun-jul-weighted\", {\"5\": 0.5, \"6\": 1, \"7\": 0.5}),\n                (\"jun-jul-aug-weighted\", {\"6\": 0.5, \"7\": 1, \"8\": 0.5}),\n                (\"jul-aug-sep-weighted\", {\"7\": 0.5, \"8\": 1, \"9\": 0.5}),\n                (\"aug-sep-oct-weighted\", {\"8\": 0.5, \"9\": 1, \"10\": 0.5}),\n                (\"sep-oct-nov-weighted\", {\"9\": 0.5, \"10\": 1, \"11\": 0.5}),\n                (\"oct-nov-dec-weighted\", {\"10\": 0.5, \"11\": 1, \"12\": 0.5}),\n                (\"nov-dec-jan-weighted\", {\"11\": 0.5, \"12\": 1, \"1\": 0.5}),\n            ]\n        },\n        index=index,\n        columns=[\n            \"dec-jan-feb-weighted\",\n            \"jan-feb-mar-weighted\",\n            \"feb-mar-apr-weighted\",\n            \"mar-apr-may-weighted\",\n            \"apr-may-jun-weighted\",\n            \"may-jun-jul-weighted\",\n            \"jun-jul-aug-weighted\",\n            \"jul-aug-sep-weighted\",\n            \"aug-sep-oct-weighted\",\n            \"sep-oct-nov-weighted\",\n            \"oct-nov-dec-weighted\",\n            \"nov-dec-jan-weighted\",\n        ],  # guarantee order\n    )\n\n\ndef segment_time_series(index, segment_type=\"single\", drop_zero_weight_segments=False):\n    \"\"\"Split a time series index into segments by applying weights.\n\n    Parameters\n    ----------\n    index : :any:`pandas.DatetimeIndex`\n        A time series index which gets split into segments.\n    segment_type : :any:`str`\n        The method to use when creating segments.\n\n         - \"single\": creates one big segment with the name \"all\".\n         - \"one_month\": creates up to twelve segments, each of which contains a single\n           month. Segment names are \"jan\", \"feb\", ... \"dec\".\n         - \"three_month\": creates up to twelve overlapping segments, each of which\n           contains three calendar months of data. Segment names are \"dec-jan-feb\",\n           \"jan-feb-mar\", ... \"nov-dec-jan\"\n         - \"three_month_weighted\": creates up to twelve overlapping segments, each of\n           contains three calendar months of data with first and third month in each\n           segment having weights of one half. Segment names are\n           \"dec-jan-feb-weighted\", \"jan-feb-mar-weighted\", ... \"nov-dec-jan-weighted\".\n\n    Returns\n    -------\n    segmentation : `pandas.DataFrame`\n        A segmentation of the input index expressed as a dataframe which shares\n        the input index and has named columns of weights.\n    \"\"\"\n    segment_weight_func = {\n        \"single\": _segment_weights_single,\n        \"one_month\": _segment_weights_one_month,\n        \"three_month\": _segment_weights_three_month,\n        \"three_month_weighted\": _segment_weights_three_month_weighted,\n    }.get(segment_type, None)\n\n    if segment_weight_func is None:\n        raise ValueError(\"Invalid segment type: %s\" % (segment_type))\n\n    segment_weights = segment_weight_func(index)\n\n    if drop_zero_weight_segments:\n        # keep only columns with non-zero weights\n        total_weights = segment_weights.sum()\n        columns_to_keep = total_weights[total_weights > 0].index.tolist()\n        segment_weights = segment_weights[columns_to_keep]\n\n    # TODO: Do something with these\n    _get_hourly_coverage_warning(segment_weights)  # each model\n    _get_calendar_year_coverage_warning(index)  # whole index\n\n    return segment_weights\n\n\ndef fit_model_segments(segmented_dataset_dict, fit_segment):\n    \"\"\"A function which fits a model to each item in a dataset.\n\n    Parameters\n    ----------\n    segmented_dataset_dict : :any:`dict` of :any:`pandas.DataFrame`\n        A dict with keys as segment names and values as dataframes of model input.\n    fit_segment : :any:`function`\n        A function which fits a model to a dataset in the `segmented_dataset_dict`.\n\n    Returns\n    -------\n    segment_models : :any:`list` of :any:`object`\n        List of fitted model objects - the return values of the fit_segment function.\n    \"\"\"\n    segment_models = [\n        fit_segment(segment_name, segment_data)\n        for segment_name, segment_data in segmented_dataset_dict.items()\n    ]\n    return segment_models\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/usage_per_day.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytz\n\nfrom opendsm.eemeter.common.transform import day_counts\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n__all__ = (\n    \"DataSufficiency\",\n    \"caltrack_sufficiency_criteria\",\n)\n\n\nclass DataSufficiency(object):\n    \"\"\"Contains the result of a data sufficiency check.\n\n    Attributes\n    ----------\n    status : :any:`str`\n        A string indicating the status of this result. Possible statuses:\n\n        - ``'NO DATA'``: No baseline data was available.\n        - ``'FAIL'``: Data did not meet criteria.\n        - ``'PASS'``: Data met criteria.\n    criteria_name : :any:`str`\n        The name of the criteria method used to check for baseline data sufficiency.\n    warnings : :any:`list` of :any:`eemeter.EEMeterWarning`\n        A list of any warnings reported during the check for baseline data sufficiency.\n    data : :any:`dict`\n        A dictionary of data related to determining whether a warning should be generated.\n    settings : :any:`dict`\n        A dictionary of settings (keyword arguments) used.\n    \"\"\"\n\n    def __init__(self, status, criteria_name, warnings=None, data=None, settings=None):\n        self.status = status  # NO DATA | FAIL | PASS\n        self.criteria_name = criteria_name\n\n        if warnings is None:\n            warnings = []\n        self.warnings = warnings\n\n        if data is None:\n            data = {}\n        self.data = data\n\n        if settings is None:\n            settings = {}\n        self.settings = settings\n\n    def __repr__(self):\n        return (\n            \"DataSufficiency(\"\n            \"status='{status}', criteria_name='{criteria_name}')\".format(\n                status=self.status, criteria_name=self.criteria_name\n            )\n        )\n\n    def json(self):\n        \"\"\"Return a JSON-serializable representation of this result.\n\n        The output of this function can be converted to a serialized string\n        with :any:`json.dumps`.\n        \"\"\"\n        return {\n            \"status\": self.status,\n            \"criteria_name\": self.criteria_name,\n            \"warnings\": [w.json() for w in self.warnings],\n            \"data\": self.data,\n            \"settings\": self.settings,\n        }\n\n\ndef caltrack_sufficiency_criteria(\n    data_quality,\n    requested_start,\n    requested_end,\n    num_days=365,\n    min_fraction_daily_coverage=0.9,  # TODO: needs to be per year\n    min_fraction_hourly_temperature_coverage_per_period=0.9,\n):\n    \"\"\"CalTRACK daily data sufficiency criteria.\n\n    .. note::\n\n        For CalTRACK compliance, ``min_fraction_daily_coverage`` must be set\n        at ``0.9`` (section 2.2.1.2), and requested_start and requested_end must\n        not be None (section 2.2.4).\n\n\n    Parameters\n    ----------\n    data_quality : :any:`pandas.DataFrame`\n        A DataFrame containing at least the column ``meter_value`` and the two\n        columns ``temperature_null``, containing a count of null hourly\n        temperature values for each meter value, and ``temperature_not_null``,\n        containing a count of not-null hourly temperature values for each\n        meter value. Should have a :any:`pandas.DatetimeIndex`.\n    requested_start : :any:`datetime.datetime`, timezone aware (or :any:`None`)\n        The desired start of the period, if any, especially if this is\n        different from the start of the data. If given, warnings\n        are reported on the basis of this start date instead of data start\n        date. Must be explicitly set to ``None`` in order to use data start date.\n    requested_end : :any:`datetime.datetime`, timezone aware (or :any:`None`)\n        The desired end of the period, if any, especially if this is\n        different from the end of the data. If given, warnings\n        are reported on the basis of this end date instead of data end date.\n        Must be explicitly set to ``None`` in order to use data end date.\n    num_days : :any:`int`, optional\n        Exact number of days allowed in data, including extent given by\n        ``requested_start`` or ``requested_end``, if given.\n    min_fraction_daily_coverage : :any:, optional\n        Minimum fraction of days of data in total data extent for which data\n        must be available.\n    min_fraction_hourly_temperature_coverage_per_period=0.9,\n        Minimum fraction of hours of temperature data coverage in a particular\n        period. Anything below this causes the whole period to be considered\n        considered missing.\n\n    Returns\n    -------\n    data_sufficiency : :any:`eemeter.DataSufficiency`\n        The an object containing sufficiency status and warnings for this data.\n    \"\"\"\n    criteria_name = \"caltrack_sufficiency_criteria\"\n\n    if data_quality.dropna().empty:\n        return DataSufficiency(\n            status=\"NO DATA\",\n            criteria_name=criteria_name,\n            warnings=[\n                EEMeterWarning(\n                    qualified_name=\"eemeter.caltrack_sufficiency_criteria.no_data\",\n                    description=(\"No data available.\"),\n                    data={},\n                )\n            ],\n        )\n\n    data_start = data_quality.index.min().tz_convert(\"UTC\")\n    data_end = data_quality.index.max().tz_convert(\"UTC\")\n    n_days_data = (data_end - data_start).days\n\n    if requested_start is not None:\n        # check for gap at beginning\n        requested_start = requested_start.astimezone(pytz.UTC)\n        n_days_start_gap = (data_start - requested_start).days\n    else:\n        n_days_start_gap = 0\n\n    if requested_end is not None:\n        # check for gap at end\n        requested_end = requested_end.astimezone(pytz.UTC)\n        n_days_end_gap = (requested_end - data_end).days\n    else:\n        n_days_end_gap = 0\n\n    critical_warnings = []\n\n    if n_days_end_gap < 0:\n        # CalTRACK 2.2.4\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".extra_data_after_requested_end_date\"\n                ),\n                description=(\"Extra data found after requested end date.\"),\n                data={\n                    \"requested_end\": requested_end.isoformat(),\n                    \"data_end\": data_end.isoformat(),\n                },\n            )\n        )\n        n_days_end_gap = 0\n\n    if n_days_start_gap < 0:\n        # CalTRACK 2.2.4\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".extra_data_before_requested_start_date\"\n                ),\n                description=(\"Extra data found before requested start date.\"),\n                data={\n                    \"requested_start\": requested_start.isoformat(),\n                    \"data_start\": data_start.isoformat(),\n                },\n            )\n        )\n        n_days_start_gap = 0\n\n    n_days_total = n_days_data + n_days_start_gap + n_days_end_gap\n\n    n_negative_meter_values = data_quality.meter_value[\n        data_quality.meter_value < 0\n    ].shape[0]\n\n    if n_negative_meter_values > 0:\n        # CalTrack 2.3.5\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\" \".negative_meter_values\"\n                ),\n                description=(\n                    \"Found negative meter data values, which may indicate presence\"\n                    \" of solar net metering.\"\n                ),\n                data={\"n_negative_meter_values\": n_negative_meter_values},\n            )\n        )\n\n    # TODO(philngo): detect and report unsorted or repeated values.\n\n    # create masks showing which daily or billing periods meet criteria\n    valid_meter_value_rows = data_quality.meter_value.notnull()\n    valid_temperature_rows = (\n        data_quality.temperature_not_null\n        / (data_quality.temperature_not_null + data_quality.temperature_null)\n    ) > min_fraction_hourly_temperature_coverage_per_period\n    valid_rows = valid_meter_value_rows & valid_temperature_rows\n\n    # get number of days per period - for daily this should be a series of ones\n    row_day_counts = day_counts(data_quality.index)\n\n    # apply masks, giving total\n    n_valid_meter_value_days = int((valid_meter_value_rows * row_day_counts).sum())\n    n_valid_temperature_days = int((valid_temperature_rows * row_day_counts).sum())\n    n_valid_days = int((valid_rows * row_day_counts).sum())\n\n    median = data_quality.meter_value.median()\n    upper_quantile = data_quality.meter_value.quantile(0.75)\n    lower_quantile = data_quality.meter_value.quantile(0.25)\n    iqr = upper_quantile - lower_quantile\n    extreme_value_limit = median + (3 * iqr)\n    n_extreme_values = data_quality.meter_value[\n        data_quality.meter_value > extreme_value_limit\n    ].shape[0]\n    max_value = float(data_quality.meter_value.max())\n\n    if n_days_total > 0:\n        fraction_valid_meter_value_days = n_valid_meter_value_days / float(n_days_total)\n        fraction_valid_temperature_days = n_valid_temperature_days / float(n_days_total)\n        fraction_valid_days = n_valid_days / float(n_days_total)\n    else:\n        # unreachable, I think.\n        fraction_valid_meter_value_days = 0\n        fraction_valid_temperature_days = 0\n        fraction_valid_days = 0\n\n    if n_days_total != num_days:\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".incorrect_number_of_total_days\"\n                ),\n                description=(\"Total data span does not match the required value.\"),\n                data={\"num_days\": num_days, \"n_days_total\": n_days_total},\n            )\n        )\n\n    if fraction_valid_days < min_fraction_daily_coverage:\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".too_many_days_with_missing_data\"\n                ),\n                description=(\n                    \"Too many days in data have missing meter data or\"\n                    \" temperature data.\"\n                ),\n                data={\"n_valid_days\": n_valid_days, \"n_days_total\": n_days_total},\n            )\n        )\n\n    if fraction_valid_meter_value_days < min_fraction_daily_coverage:\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".too_many_days_with_missing_meter_data\"\n                ),\n                description=(\"Too many days in data have missing meter data.\"),\n                data={\n                    \"n_valid_meter_data_days\": n_valid_meter_value_days,\n                    \"n_days_total\": n_days_total,\n                },\n            )\n        )\n\n    if fraction_valid_temperature_days < min_fraction_daily_coverage:\n        critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\"\n                    \".too_many_days_with_missing_temperature_data\"\n                ),\n                description=(\"Too many days in data have missing temperature data.\"),\n                data={\n                    \"n_valid_temperature_data_days\": n_valid_temperature_days,\n                    \"n_days_total\": n_days_total,\n                },\n            )\n        )\n\n    if len(critical_warnings) > 0:\n        status = \"FAIL\"\n    else:\n        status = \"PASS\"\n\n    non_critical_warnings = []\n    if n_extreme_values > 0:\n        # CalTRACK 2.3.6\n        non_critical_warnings.append(\n            EEMeterWarning(\n                qualified_name=(\n                    \"eemeter.caltrack_sufficiency_criteria\" \".extreme_values_detected\"\n                ),\n                description=(\n                    \"Extreme values (greater than (median + (3 * IQR)),\"\n                    \" must be flagged for manual review.\"\n                ),\n                data={\n                    \"n_extreme_values\": n_extreme_values,\n                    \"median\": median,\n                    \"upper_quantile\": upper_quantile,\n                    \"lower_quantile\": lower_quantile,\n                    \"extreme_value_limit\": extreme_value_limit,\n                    \"max_value\": max_value,\n                },\n            )\n        )\n\n    warnings = critical_warnings + non_critical_warnings\n    sufficiency_data = {\n        \"extra_data_after_requested_end_date\": {\n            \"requested_end\": requested_end.isoformat() if requested_end else None,\n            \"data_end\": data_end.isoformat(),\n            \"n_days_end_gap\": n_days_end_gap,\n        },\n        \"extra_data_before_requested_start_date\": {\n            \"requested_start\": requested_start.isoformat() if requested_start else None,\n            \"data_start\": data_start.isoformat(),\n            \"n_days_start_gap\": n_days_start_gap,\n        },\n        \"negative_meter_values\": {\"n_negative_meter_values\": n_negative_meter_values},\n        \"incorrect_number_of_total_days\": {\n            \"num_days\": num_days,\n            \"n_days_total\": n_days_total,\n        },\n        \"too_many_days_with_missing_data\": {\n            \"n_valid_days\": n_valid_days,\n            \"n_days_total\": n_days_total,\n        },\n        \"too_many_days_with_missing_meter_data\": {\n            \"n_valid_meter_data_days\": n_valid_meter_value_days,\n            \"n_days_total\": n_days_total,\n        },\n        \"too_many_days_with_missing_temperature_data\": {\n            \"n_valid_temperature_data_days\": n_valid_temperature_days,\n            \"n_days_total\": n_days_total,\n        },\n        \"extreme_values_detected\": {\n            \"n_extreme_values\": n_extreme_values,\n            \"median\": median,\n            \"upper_quantile\": upper_quantile,\n            \"lower_quantile\": lower_quantile,\n            \"extreme_value_limit\": extreme_value_limit,\n            \"max_value\": max_value,\n        },\n    }\n\n    return DataSufficiency(\n        status=status,\n        criteria_name=criteria_name,\n        warnings=warnings,\n        data=sufficiency_data,\n        settings={\n            \"num_days\": num_days,\n            \"min_fraction_daily_coverage\": min_fraction_daily_coverage,\n            \"min_fraction_hourly_temperature_coverage_per_period\": min_fraction_hourly_temperature_coverage_per_period,\n        },\n    )\n"
  },
  {
    "path": "opendsm/eemeter/models/hourly_caltrack/wrapper.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.common.stats.basic import t_stat\nfrom opendsm.eemeter.common.features import (\n    estimate_hour_of_week_occupancy,\n    fit_temperature_bins,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.design_matrices import (\n    create_caltrack_hourly_preliminary_design_matrix,\n    create_caltrack_hourly_segmented_design_matrices,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.model import (\n    CalTRACKHourlyModelResults,\n    fit_caltrack_hourly_model,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import segment_time_series\n\nmonth_dict = {\n    \"jan\": 1,\n    \"feb\": 2,\n    \"mar\": 3,\n    \"apr\": 4,\n    \"may\": 5,\n    \"jun\": 6,\n    \"jul\": 7,\n    \"aug\": 8,\n    \"sep\": 9,\n    \"oct\": 10,\n    \"nov\": 11,\n    \"dec\": 12,\n}\n\n\nclass IntermediateModelVariables:\n    preliminary_design_matrix = None\n    segmentation = None\n    occupancy_lookup = None\n    occupied_temperature_bins = None\n    unoccupied_temperature_bins = None\n    segmented_design_matrices = None\n\n\nclass HourlyModel:\n    def __init__(self, settings=None):\n        self.segment_type = \"three_month_weighted\"\n        self.alpha = 0.1\n\n    def fit(self, data):\n        meter_data = data.df[\"observed\"].to_frame(\"value\")\n        temperature_data = data.df[\"temperature\"]\n\n        self.model_process_variables = IntermediateModelVariables()\n\n        # preliminary design matrix\n        preliminary_design_matrix = create_caltrack_hourly_preliminary_design_matrix(\n            meter_data, temperature_data\n        )\n        self.model_process_variables.preliminary_design_matrix = (\n            preliminary_design_matrix\n        )\n\n        # segment time series\n        segmentation = segment_time_series(\n            preliminary_design_matrix.index, self.segment_type\n        )\n        self.model_process_variables.segmentation = segmentation\n\n        # estimate occupancy\n        occupancy_lookup = estimate_hour_of_week_occupancy(\n            preliminary_design_matrix, segmentation=segmentation\n        )\n        self.model_process_variables.occupancy_lookup = occupancy_lookup\n\n        # fit temperature bins\n        (occupied_t_bins, unoccupied_t_bins) = fit_temperature_bins(\n            preliminary_design_matrix,\n            segmentation=segmentation,\n            occupancy_lookup=occupancy_lookup,\n        )\n        self.model_process_variables.occupied_temperature_bins = occupied_t_bins\n        self.model_process_variables.unoccupied_temperature_bins = unoccupied_t_bins\n\n        # create segmented design matrices\n        segmented_design_matrices = create_caltrack_hourly_segmented_design_matrices(\n            preliminary_design_matrix,\n            segmentation,\n            occupancy_lookup,\n            occupied_t_bins,\n            unoccupied_t_bins,\n        )\n        self.model_process_variables.segmented_design_matrices = (\n            segmented_design_matrices\n        )\n\n        # fit model\n        self.model = fit_caltrack_hourly_model(\n            segmented_design_matrices,\n            occupancy_lookup,\n            occupied_t_bins,\n            unoccupied_t_bins,\n            self.segment_type,\n        )\n        self.is_fit = True\n        self.model_metrics = self.model.totals_metrics\n\n        # calculate baseline residuals\n        prediction = self.model.predict(temperature_data.index, temperature_data)\n        meter_data = meter_data.merge(\n            prediction.result, left_index=True, right_index=True\n        )\n        meter_data.dropna(inplace=True)\n        meter_data[\"resid\"] = meter_data[\"value\"] - meter_data[\"predicted_usage\"]\n\n        # get uncertainty variables\n        self._autocorr_unc_vars = {}\n        if list(self.model_metrics.keys()) == [\"all\"]:\n            self._autocorr_unc_vars[\"all\"] = {\n                \"mean_baseline_usage\": np.mean(meter_data[\"value\"]),\n                \"n\": self.model_metrics[\"all\"].observed_length,\n                \"n_prime\": self.model_metrics[\"all\"].n_prime,\n                \"MSE\": np.mean(meter_data[\"resid\"] ** 2),\n            }\n        else:\n            # monthly segment model\n            model_month_dict = {\n                k.replace(\"-weighted\", \"\").split(\"-\")[1]: k\n                for k in self.model_metrics.keys()\n            }\n            meter_data[\"month\"] = meter_data.index.month\n\n            for month_abbr, model_key in model_month_dict.items():\n                month_n = month_dict[month_abbr]\n                month_data = meter_data[meter_data[\"month\"] == month_n]\n\n                self._autocorr_unc_vars[month_n] = {\n                    \"mean_baseline_usage\": np.mean(month_data[\"value\"]),\n                    \"n\": self.model_metrics[model_key].observed_length,\n                    \"n_prime\": self.model_metrics[model_key].n_prime,\n                    \"MSE\": np.mean(month_data[\"resid\"] ** 2),\n                }\n\n        return self\n\n    def predict(self, reporting_data):\n        if not self.is_fit:\n            raise RuntimeError(\"Model must be fit before predictions can be made.\")\n        prediction_index = reporting_data.df.index\n        temperature_series = reporting_data.df[\"temperature\"]\n        model_prediction = self.model.predict(prediction_index, temperature_series)\n\n        df_res = pd.concat([reporting_data.df, model_prediction.result], axis=1)\n        df_res = df_res[[\"temperature\", \"observed\", \"predicted_usage\"]]\n        df_res = df_res.rename(columns={\"predicted_usage\": \"predicted\"})\n        df_res[\"predicted_uncertainty\"] = np.nan\n\n        # if observed isn't all nan, calculate uncertainty\n        if not df_res[\"observed\"].isna().all():\n            for month_n, unc_vars in self._autocorr_unc_vars.items():\n                if month_n == \"all\":\n                    idx = df_res.index\n                else:\n                    idx = df_res.index[df_res.index.month == month_n]\n\n                mean_baseline_usage = unc_vars[\"mean_baseline_usage\"]\n                n = unc_vars[\"n\"]\n                n_prime = unc_vars[\"n_prime\"]\n                mse = unc_vars[\"MSE\"]\n\n                reporting_usage = np.sum(df_res.loc[idx][\"observed\"])\n                m = len(idx)\n                t = t_stat(self.alpha, m, tail=2)\n\n                # ASHRAE 14\n                total_unc = (\n                    1.26\n                    * t\n                    * reporting_usage\n                    / (m * mean_baseline_usage)\n                    * np.sqrt(mse * n / n_prime * (1 + 2 / n_prime) * m)\n                )\n\n                avg_unc = np.sqrt(total_unc**2 / m)\n                df_res.loc[idx, \"predicted_uncertainty\"] = avg_unc\n\n        return df_res\n\n    def to_dict(self):\n        model_dict = self.model.json()\n        model_dict[\"model\"][\"unc_vars\"] = self._autocorr_unc_vars\n        return model_dict\n\n    def to_json(self):\n        return json.dumps(self.to_dict())\n\n    @classmethod\n    def from_dict(cls, data):\n        hourly_model = cls()\n        hourly_model.model = CalTRACKHourlyModelResults.from_json(data)\n        hourly_model._autocorr_unc_vars = data[\"model\"][\"unc_vars\"]\n        hourly_model.is_fit = True\n        return hourly_model\n\n    @classmethod\n    def from_json(cls, str_data):\n        return cls.from_dict(json.loads(str_data))\n\n    @classmethod\n    def from_2_0_dict(cls, data):\n        \"\"\"fill default metrics and uncertainty variables to allow deserializing legacy models with new wrapper\"\"\"\n        monthly_unc_vars = {\"mean_baseline_usage\": 0, \"n\": 0, \"n_prime\": 1, \"MSE\": 0}\n        model_dict = dict(data)\n        model_dict[\"model\"][\"unc_vars\"] = {\n            str(month): monthly_unc_vars for month in range(1, 13)\n        }\n        return cls.from_dict(model_dict)\n\n    @classmethod\n    def from_2_0_json(cls, str_data):\n        return cls.from_2_0_dict(json.loads(str_data))\n\n    def plot(\n        self,\n        ax=None,\n        title=None,\n        figsize=None,\n        temp_range=None,\n    ):\n        raise NotImplementedError\n"
  },
  {
    "path": "opendsm/eemeter/samples/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom .load import *\n"
  },
  {
    "path": "opendsm/eemeter/samples/load.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\n\nimport pytz\nfrom dateutil.parser import parse as parse_date\nimport importlib.resources\n\nfrom ..utilities.io import meter_data_from_csv, temperature_data_from_csv\n\n__all__ = (\"samples\", \"load_sample\")\n\n\ndef _load_sample_metadata():\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        \"metadata.json\"\n    ).open(\"rb\") as f:\n        metadata = json.loads(f.read())\n    return metadata\n\n\ndef samples():\n    \"\"\"Load a list of sample data identifiers.\n\n    Returns\n    -------\n    samples : :any:`list` of :any:`str`\n        List of sample identifiers for use with :any:`opendsm.eemeter.load_sample`.\n    \"\"\"\n    sample_metadata = _load_sample_metadata()\n    return list(sorted(sample_metadata.keys()))\n\n\ndef load_sample(sample, tempF=True):\n    \"\"\"Load meter data, temperature data, and metadata for associated with a\n    particular sample identifier. Note: samples are simulated, not real, data.\n\n    Parameters\n    ----------\n    sample : :any:`str`\n        Identifier of sample. Complete list can be obtained with\n        :any:`opendsm.eemeter.samples`.\n    tempF : :any 'bool'\n        Flag regarding whether the sample temperature dataset is associated with Fahrenheit or Celsius.\n\n    Returns\n    -------\n    meter_data, temperature_data, metadata : :any:`tuple` of :any:`pandas.DataFrame`, :any:`pandas.Series`, and :any:`dict`\n        Meter data, temperature data, and metadata for this sample identifier.\n    \"\"\"\n    if tempF == True:\n        temp_units = \"tempF\"\n    else:\n        temp_units = \"tempC\"\n\n    sample_metadata = _load_sample_metadata()\n    metadata = sample_metadata.get(sample)\n    if metadata is None:\n        raise ValueError(\n            \"Sample not found: {}. Try one of these?\\n{}\".format(\n                sample,\n                \"\\n\".join(\n                    [\" - {}\".format(key) for key in sorted(sample_metadata.keys())]\n                ),\n            )\n        )\n\n    freq = metadata.get(\"freq\")\n    if freq not in (\"hourly\", \"daily\"):\n        freq = None\n\n    meter_data_filename = metadata[\"meter_data_filename\"]\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        meter_data_filename\n    ).open(\"rb\") as f:\n        meter_data = meter_data_from_csv(f, gzipped=True, freq=freq)\n\n    temperature_filename = metadata[\"temperature_filename\"]\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        temperature_filename\n    ).open(\"rb\") as f:\n        temperature_data = temperature_data_from_csv(\n            f, gzipped=True, freq=\"hourly\", temp_col=temp_units\n        )\n\n    metadata[\"blackout_start_date\"] = pytz.UTC.localize(\n        parse_date(metadata[\"blackout_start_date\"])\n    )\n    metadata[\"blackout_end_date\"] = pytz.UTC.localize(\n        parse_date(metadata[\"blackout_end_date\"])\n    )\n\n    return meter_data, temperature_data, metadata\n"
  },
  {
    "path": "opendsm/eemeter/samples/metadata.json",
    "content": "{\"il-electricity-cdd-hdd-hourly\": {\"id\": \"il-electricity-cdd-hdd-hourly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-hdd-hourly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"hourly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2000.0, \"annual_baseline_heating_load\": 4000.0, \"annual_baseline_cooling_load\": 4000.0, \"baseline_heating_balance_point\": 60, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 1800.0, \"annual_reporting_heating_load\": 3600.0, \"annual_reporting_cooling_load\": 3600.0, \"reporting_heating_balance_point\": 60, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-hdd-daily\": {\"id\": \"il-electricity-cdd-hdd-daily\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-hdd-daily.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"daily\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2000.0, \"annual_baseline_heating_load\": 4000.0, \"annual_baseline_cooling_load\": 4000.0, \"baseline_heating_balance_point\": 60, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 1800.0, \"annual_reporting_heating_load\": 3600.0, \"annual_reporting_cooling_load\": 3600.0, \"reporting_heating_balance_point\": 60, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-hdd-billing_monthly\": {\"id\": \"il-electricity-cdd-hdd-billing_monthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-hdd-billing_monthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_monthly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2000.0, \"annual_baseline_heating_load\": 4000.0, \"annual_baseline_cooling_load\": 4000.0, \"baseline_heating_balance_point\": 60, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 1800.0, \"annual_reporting_heating_load\": 3600.0, \"annual_reporting_cooling_load\": 3600.0, \"reporting_heating_balance_point\": 60, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-hdd-billing_bimonthly\": {\"id\": \"il-electricity-cdd-hdd-billing_bimonthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-hdd-billing_bimonthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_bimonthly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2000.0, \"annual_baseline_heating_load\": 4000.0, \"annual_baseline_cooling_load\": 4000.0, \"baseline_heating_balance_point\": 60, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 1800.0, \"annual_reporting_heating_load\": 3600.0, \"annual_reporting_cooling_load\": 3600.0, \"reporting_heating_balance_point\": 60, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-only-hourly\": {\"id\": \"il-electricity-cdd-only-hourly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-only-hourly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"hourly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2500.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 7500.0, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 2250.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 6750.0, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-only-daily\": {\"id\": \"il-electricity-cdd-only-daily\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-only-daily.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"daily\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2500.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 7500.0, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 2250.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 6750.0, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-only-billing_monthly\": {\"id\": \"il-electricity-cdd-only-billing_monthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-only-billing_monthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_monthly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2500.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 7500.0, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 2250.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 6750.0, \"reporting_cooling_balance_point\": 65},\n  \"il-electricity-cdd-only-billing_bimonthly\": {\"id\": \"il-electricity-cdd-only-billing_bimonthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-electricity-cdd-only-billing_bimonthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_bimonthly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 10000, \"annual_baseline_base_load\": 2500.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 7500.0, \"baseline_cooling_balance_point\": 65, \"annual_reporting_total_load\": 9000.0, \"annual_reporting_base_load\": 2250.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 6750.0, \"reporting_cooling_balance_point\": 65},\n  \"il-gas-hdd-only-hourly\": {\"id\": \"il-gas-hdd-only-hourly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-hdd-only-hourly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"hourly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 250.0, \"annual_baseline_heating_load\": 750.0, \"annual_baseline_cooling_load\": 0.0, \"baseline_heating_balance_point\": 60, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 225.0, \"annual_reporting_heating_load\": 675.0, \"annual_reporting_cooling_load\": 0.0, \"reporting_heating_balance_point\": 60},\n  \"il-gas-hdd-only-daily\": {\"id\": \"il-gas-hdd-only-daily\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-hdd-only-daily.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"daily\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 250.0, \"annual_baseline_heating_load\": 750.0, \"annual_baseline_cooling_load\": 0.0, \"baseline_heating_balance_point\": 60, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 225.0, \"annual_reporting_heating_load\": 675.0, \"annual_reporting_cooling_load\": 0.0, \"reporting_heating_balance_point\": 60},\n  \"il-gas-hdd-only-billing_monthly\": {\"id\": \"il-gas-hdd-only-billing_monthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-hdd-only-billing_monthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_monthly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 250.0, \"annual_baseline_heating_load\": 750.0, \"annual_baseline_cooling_load\": 0.0, \"baseline_heating_balance_point\": 60, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 225.0, \"annual_reporting_heating_load\": 675.0, \"annual_reporting_cooling_load\": 0.0, \"reporting_heating_balance_point\": 60},\n  \"il-gas-hdd-only-billing_bimonthly\": {\"id\": \"il-gas-hdd-only-billing_bimonthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-hdd-only-billing_bimonthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_bimonthly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 250.0, \"annual_baseline_heating_load\": 750.0, \"annual_baseline_cooling_load\": 0.0, \"baseline_heating_balance_point\": 60, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 225.0, \"annual_reporting_heating_load\": 675.0, \"annual_reporting_cooling_load\": 0.0, \"reporting_heating_balance_point\": 60},\n  \"il-gas-intercept-only-hourly\": {\"id\": \"il-gas-intercept-only-hourly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-intercept-only-hourly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"hourly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"il-gas-intercept-only-daily\": {\"id\": \"il-gas-intercept-only-daily\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-intercept-only-daily.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"daily\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"il-gas-intercept-only-billing_monthly\": {\"id\": \"il-gas-intercept-only-billing_monthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-intercept-only-billing_monthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_monthly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"il-gas-intercept-only-billing_bimonthly\": {\"id\": \"il-gas-intercept-only-billing_bimonthly\", \"temperature_filename\": \"il-tempF.csv.gz\", \"meter_data_filename\": \"il-gas-intercept-only-billing_bimonthly.csv.gz\", \"blackout_start_date\": \"2016-12-26\", \"blackout_end_date\": \"2017-01-04\", \"freq\": \"billing_bimonthly\", \"interpretation\": \"gas\", \"unit\": \"therm\", \"usaf_id\": \"724390\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"uk-electricity-hdd-only-hourly-sample-1\": {\"id\": \"uk-electricity-hdd-only-hourly-sample-1\", \"temperature_filename\": \"uk-tempC-sample-1_2.csv.gz\", \"meter_data_filename\": \"uk-electricity-hdd-only-hourly-sample-1.csv.gz\", \"blackout_start_date\": \"2013-01-11\", \"blackout_end_date\": \"2013-01-11\", \"freq\": \"hourly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"N/A\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"uk-electricity-hdd-only-hourly-sample-2\": {\"id\": \"uk-electricity-hdd-only-hourly-sample-2\", \"temperature_filename\": \"uk-tempC-sample-1_2.csv.gz\", \"meter_data_filename\": \"uk-electricity-hdd-only-hourly-sample-2.csv.gz\", \"blackout_start_date\": \"2012-12-06\", \"blackout_end_date\": \"2012-12-06\", \"freq\": \"hourly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"N/A\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"uk-electricity-hdd-only-hourly-sample-0\": {\"id\": \"uk-electricity-hdd-only-hourly-sample-0\", \"temperature_filename\": \"uk-tempC-sample-0.csv.gz\", \"meter_data_filename\": \"uk-electricity-hdd-only-hourly-sample-0.csv.gz\", \"blackout_start_date\": \"2021-04-01\", \"blackout_end_date\": \"2021-04-01\", \"freq\": \"hourly\", \"interpretation\": \"electricity\", \"unit\": \"kWh\", \"usaf_id\": \"N/A\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0},\n  \"uk-gas-hdd-only-hourly-sample-0\": {\"id\": \"uk-gas-hdd-only-hourly-sample-0\", \"temperature_filename\": \"uk-tempC-sample-0.csv.gz\", \"meter_data_filename\": \"uk-gas-hdd-only-hourly-sample-0.csv.gz\", \"blackout_start_date\": \"2021-04-01\", \"blackout_end_date\": \"2021-04-01\", \"freq\": \"hourly\", \"interpretation\": \"gas\", \"unit\": \"kWh\", \"usaf_id\": \"N/A\", \"annual_baseline_total_load\": 1000, \"annual_baseline_base_load\": 1000.0, \"annual_baseline_heating_load\": 0.0, \"annual_baseline_cooling_load\": 0.0, \"annual_reporting_total_load\": 900.0, \"annual_reporting_base_load\": 900.0, \"annual_reporting_heating_load\": 0.0, \"annual_reporting_cooling_load\": 0.0}\n}"
  },
  {
    "path": "opendsm/eemeter/utilities/__init__.py",
    "content": ""
  },
  {
    "path": "opendsm/eemeter/utilities/io.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom __future__ import annotations\n\nimport datetime\n\nimport numpy as np\nimport pandas as pd\nfrom pandas._typing import (\n    CompressionOptions,\n    CSVEngine,\n    DtypeArg,\n    DtypeBackend,\n    FilePath,\n    IndexLabel,\n    ReadCsvBuffer,\n    StorageOptions,\n    WriteBuffer,\n)\n\n__all__ = (\n    \"meter_data_from_csv\",\n    \"meter_data_from_json\",\n    \"meter_data_to_csv\",\n    \"temperature_data_from_csv\",\n    \"temperature_data_from_json\",\n    \"temperature_data_to_csv\",\n)\n\n\ndef meter_data_from_csv(\n    filepath_or_buffer: str | FilePath | ReadCsvBuffer[bytes] | ReadCsvBuffer[str],\n    tz: str | datetime.tzinfo | None = None,\n    start_col: str = \"start\",\n    value_col: str = \"value\",\n    gzipped: bool = False,\n    freq: str | None = None,\n    **kwargs,\n) -> pd.DataFrame:\n    \"\"\"Load meter data from a CSV file and convert to a dataframe.\n\n    Note: This is an example of the default csv structure assumed.\n        ```python\n        start,value\n        2017-01-01T00:00:00+00:00,0.31\n        2017-01-02T00:00:00+00:00,0.4\n        2017-01-03T00:00:00+00:00,0.58\n        ```\n\n    Args:\n        filepath_or_buffer: File path or object.\n        tz: Timezone represented in the meter data. Ex: `UTC` or `US/Pacific`\n        start_col: Date period start column.\n        value_col: Value column, can be in any unit.\n        gzipped: Whether file is gzipped.\n        freq: If given, apply frequency to data using `pandas.DataFrame.resample`. One of `['hourly', 'daily']`.\n        **kwargs: Extra keyword arguments to pass to `pandas.read_csv`, such as `sep='|'`.\n    \"\"\"\n\n    read_csv_kwargs = {\n        \"usecols\": [start_col, value_col],\n        \"dtype\": {value_col: np.float64},\n        \"parse_dates\": [start_col],\n        \"index_col\": start_col,\n    }\n\n    if gzipped:\n        read_csv_kwargs.update({\"compression\": \"gzip\"})\n\n    # allow passing extra kwargs\n    read_csv_kwargs.update(kwargs)\n\n    df = pd.read_csv(filepath_or_buffer, **read_csv_kwargs)\n    df.index = pd.to_datetime(df.index, utc=True)\n\n    # for pandas<0.24, which doesn't localize even with utc=True\n    if df.index.tz is None:\n        df.index = df.index.tz_localize(\"UTC\")  # pragma: no cover\n\n    if tz is not None:\n        df = df.tz_convert(tz)\n\n    if freq == \"hourly\":\n        df = df.resample(\"h\").sum(min_count=1)\n    elif freq == \"daily\":\n        df = df.resample(\"D\").sum(min_count=1)\n\n    return df\n\n\ndef temperature_data_from_csv(\n    filepath_or_buffer: str | FilePath | ReadCsvBuffer[bytes] | ReadCsvBuffer[str],\n    tz: str | datetime.tzinfo | None = None,\n    date_col: str = \"dt\",\n    temp_col: str = \"tempF\",\n    gzipped: bool = False,\n    freq: str | None = None,\n    **kwargs,\n):\n    \"\"\"Load meter data from a CSV file and convert to a dataframe. Farenheit is assumed for building models.\n\n    Note: This is an example of the default csv structure assumed.\n        ```python\n        dt,tempF\n        2017-01-01T00:00:00+00:00,21\n        2017-01-01T01:00:00+00:00,22.5\n        2017-01-01T02:00:00+00:00,23.5\n        ```\n\n    Args:\n        filepath_or_buffer: File path or object.\n        tz: Timezone represented in the meter data. Ex: `UTC` or `US/Pacific`\n        date_col: Date period start column.\n        temp_col: Temperature column.\n        gzipped: Whether file is gzipped.\n        freq: If given, apply frequency to data using `pandas.DataFrame.resample`. One of `['hourly', 'daily']`.\n        **kwargs: Extra keyword arguments to pass to `pandas.read_csv`, such as `sep='|'`.\n    \"\"\"\n    read_csv_kwargs = {\n        \"usecols\": [date_col, temp_col],\n        \"dtype\": {temp_col: np.float64},\n        \"parse_dates\": [date_col],\n        \"index_col\": date_col,\n    }\n\n    if gzipped:\n        read_csv_kwargs.update({\"compression\": \"gzip\"})\n\n    # allow passing extra kwargs\n    read_csv_kwargs.update(kwargs)\n\n    df = pd.read_csv(filepath_or_buffer, **read_csv_kwargs)\n    df.index = pd.to_datetime(df.index, utc=True)\n\n    # for pandas<0.24, which doesn't localize even with utc=True\n    if df.index.tz is None:\n        df.index = df.index.tz_localize(\"UTC\")  # pragma: no cover\n\n    if tz is not None:\n        df = df.tz_convert(tz)\n\n    if freq == \"hourly\":\n        df = df.resample(\"h\").sum(min_count=1)\n\n    return df[temp_col]\n\n\ndef meter_data_from_json(data: list, orient: str = \"list\") -> pd.DataFrame:\n    \"\"\"Load meter data from a list of dictionary objects or a list of lists.\n\n    Args:\n        data: A list of meter data, with each row representing a single record.\n        orient: Format of `data` parameter. Must be one of `['list', 'records']`.\n            `'list'` is a list of lists, with the first element as start date and the second element as meter usage. `'records'` is a list of dicts.\n\n    Note: This is an example of the default `list` structure.\n        ```python\n        [\n            ['2017-01-01T00:00:00+00:00', 3.5],\n            ['2017-02-01T00:00:00+00:00', 0.4],\n            ['2017-03-01T00:00:00+00:00', 0.46],\n        ]\n        ```\n\n    Note: This is an example of the `records` structure.\n        ```python\n        [\n            {'start': '2017-01-01T00:00:00+00:00', 'value': 3.5},\n            {'start': '2017-02-01T00:00:00+00:00', 'value': 0.4},\n            {'start': '2017-03-01T00:00:00+00:00', 'value': 0.46},\n        ]\n        ```\n\n    Returns:\n        DataFrame with a single column (``'value'``) and a `pandas.DatetimeIndex`. A second column (``'estimated'``) may also be included if the input data contained an estimated boolean flag.\n    \"\"\"\n\n    def _empty_meter_data_dataframe():\n        return pd.DataFrame(\n            {\"value\": []}, index=pd.DatetimeIndex([], tz=\"UTC\", name=\"start\")\n        )\n\n    if data is None:\n        return _empty_meter_data_dataframe()\n\n    if orient == \"list\":\n        df = pd.DataFrame(data, columns=[\"start\", \"value\"])\n        df[\"start\"] = pd.to_datetime(df.start, utc=True)\n        df = df.set_index(\"start\")\n        return df\n    elif orient == \"records\":\n\n        def _noneify_meter_data_row(row):\n            value = row[\"value\"]\n            if value is not None:\n                try:\n                    value = float(value)\n                except ValueError:\n                    value = None\n            out_row = {\"start\": row[\"start\"], \"value\": value}\n            if \"estimated\" in row:\n                estimated = row.get(\"estimated\")\n                out_row[\"estimated\"] = estimated in [True, \"true\", \"True\", 1, \"1\"]\n            return out_row\n\n        noneified_data = [_noneify_meter_data_row(row) for row in data]\n        df = pd.DataFrame(noneified_data)\n        if df.empty:\n            return _empty_meter_data_dataframe()\n        df[\"start\"] = pd.to_datetime(df.start, utc=True)\n        df = df.set_index(\"start\")\n        df[\"value\"] = df[\"value\"].astype(float)\n        if \"estimated\" in df.columns:\n            df[\"estimated\"] = (\n                df[\"estimated\"].where(df[\"estimated\"].notna(), False).astype(bool)\n            )\n        return df\n    else:\n        raise ValueError(\"orientation not recognized.\")\n\n\ndef temperature_data_from_json(data: list, orient: str = \"list\") -> pd.Series:\n    \"\"\"Load temperature data from json to a Series. Farenheit is assumed for building models.\n\n    Args:\n        data: A list of temperature data, with each row representing a single record.\n        orient: Format of `data` parameter. Must be `'list'`.\n            `'list'` is a list of lists, with the first element as start date and the second element as temperature.\n\n    Note: This is an example of the default `list` structure.\n        ```python\n        [\n            ['2017-01-01T00:00:00+00:00', 3.5],\n            ['2017-01-01T01:00:00+00:00', 5.4],\n            ['2017-01-01T02:00:00+00:00', 7.4],\n        ]\n        ```\n\n    Returns:\n        DataFrame with a single column (``'tempF'``) and a `pandas.DatetimeIndex`.\n\n    Raises:\n        ValueError: If `orient` is not `'list'`.\n    \"\"\"\n    if orient == \"list\":\n        df = pd.DataFrame(data, columns=[\"dt\", \"tempF\"])\n        series = df.tempF\n        series.index = pd.to_datetime(df.dt, utc=True)\n        return series\n    else:\n        raise ValueError(\"orientation not recognized.\")\n\n\ndef meter_data_to_csv(\n    meter_data: pd.DataFrame | pd.Series,\n    path_or_buf: str | FilePath | WriteBuffer[bytes] | WriteBuffer[str],\n) -> None:\n    \"\"\"Write meter data from a DataFrame or Series to a CSV. See also `pandas.DataFrame.to_csv`.\n\n    Args:\n        meter_data: DataFrame or Series with a ``'value'`` column and a `pandas.DatetimeIndex`.\n        path_or_buf: Path or file handle.\n    \"\"\"\n    if meter_data.index.name is None:\n        meter_data.index.name = \"start\"\n\n    return meter_data.to_csv(path_or_buf, index=True)\n\n\ndef temperature_data_to_csv(\n    temperature_data: pd.Series,\n    path_or_buf: str | FilePath | WriteBuffer[bytes] | WriteBuffer[str],\n) -> None:\n    \"\"\"Write temperature data to CSV. See also :any:`pandas.DataFrame.to_csv`.\n\n    Args:\n        temperature_data: Temperature data series with :any:`pandas.DatetimeIndex`.\n        path_or_buf: Path or file handle.\n    \"\"\"\n    if temperature_data.index.name is None:\n        temperature_data.index.name = \"dt\"\n    if temperature_data.name is None:\n        temperature_data.name = \"temperature\"\n\n    return temperature_data.to_frame().to_csv(path_or_buf, index=True)\n"
  },
  {
    "path": "pyproject.toml",
    "content": "[build-system]\nrequires = [\"hatchling>=1.25\"]\nbuild-backend = \"hatchling.build\"\n\n[project]\nname = \"opendsm\"\nversion = \"1.2.7\"\ndescription = \"Standard methods for predicting building energy usage\"   # set directly\n# dynamic = [\"version\"]                                  # only version is dynamic\nreadme = \"README.md\"\nrequires-python = \">=3.10\"\nlicense = { text = \"Apache-2.0\" }\nauthors = [{ name = \"opendsm\", email = \"opendsm@lists.lfenergy.org\" }]\nclassifiers = [\n  \"License :: OSI Approved :: Apache Software License\",\n  \"Programming Language :: Python :: 3.10\",\n  \"Programming Language :: Python :: 3.11\",\n  \"Programming Language :: Python :: 3.12\",\n]\n\ndependencies = [\n  \"fdasrsf>=2.4.1\",\n  # \"fdasrsf>=2.4.1,<=2.5.2\", # library broken on higher versions. Issue tracked here https://github.com/jdtuck/fdasrsf_python/issues/41\n  # \"mkl-devel\",  # needed for fdasrsf to work: https://github.com/jdtuck/fdasrsf_python/issues/41\n  \"nlopt\",\n  \"numba\",\n  \"numpy>=1.24.4\",\n  \"pandas>=1.1.0,<3.0.0\",\n  \"pyarrow>=15.0.0\",\n  \"pydantic>=2.0\",\n  \"pywavelets\",\n  \"requests\",\n  \"scikit-learn>=1.3.0\",\n  \"qpsolvers[highs]\",\n  \"scikit-fda>0.7.1\",\n  \"scikit-learn>=1.3.0\",\n  \"scipy>=1.10.1\",\n  \"statsmodels\",\n]\n\n[project.optional-dependencies]\ndev = [\n  \"black\",\n  \"coverage\",\n  \"pytest\",\n  \"pytest-cov>=7.0.0\",\n  \"pytest-profiling\",\n  \"pytest-xdist\",\n  \"snapshottest==0.6.0\",\n  \"tox\",\n  \"twine\",\n  \"typing\",\n]\n\n[tool.hatch.build.targets.wheel]\n# non-src layout: the importable package directory is opendsm/opendsm\npackages = [\"opendsm\"]\n\n[project.urls]\nHomepage = \"https://lfenergy.org/projects/opendsm\"\nDocumentation = \"https://opendsm.energy\"\nRepository = \"http://github.com/opendsm/opendsm\"\nIssues = \"http://github.com/opendsm/opendsm/issues\"\n\n[project.scripts]\nopendsm = \"opendsm.cli:main\"\n"
  },
  {
    "path": "pytest.ini",
    "content": "[pytest]\naddopts =\n    # run in parallel - requires pytest-xdist\n    -n auto\n\n    # show coverage - requires pytest-cov\n    ; --cov=./\n\n    # show lines missing coverage\n    ; --cov-report term-missing\n\n    # verbose output\n    -vv\n\nfilterwarnings =\n    error\n    # the above filter is useful to debug and fix warnings locally, but\n    # frequently causes segfaults during CI when native code is running.\n    # perhaps we can add a secondary step that fails if warnings are present\n\n    # suppressed warnings\n    default:Level value of 5 is too high\n    ignore:builtin type swigvarlink has no __module__ attribute\n    ignore:builtin type SwigPyObject has no __module__ attribute\n\n    #TODO breaks after sklearn 1.7 releases\n    default:`BaseEstimator._validate_data` is deprecated in 1.6\n"
  },
  {
    "path": "setup.cfg",
    "content": "[aliases]\ntest=pytest\n"
  },
  {
    "path": "tests/common/clustering/test_bisect_k_means.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pytest\nfrom sklearn.datasets import make_blobs\n\nfrom opendsm.common.clustering.algorithms.bisect_k_means import bisect_k_means\nfrom opendsm.common.clustering.settings import ClusteringSettings\n\n\ndef get_default_settings_dict():\n    \"\"\"Return a default settings dictionary that can be modified.\"\"\"\n    return {\n        \"algorithm_selection\": \"bisecting_kmeans\",\n        \"seed\": 42,\n    }\n\n\n@pytest.fixture\ndef simple_2d_data():\n    \"\"\"Create simple 2D synthetic data with clear clusters.\"\"\"\n    np.random.seed(42)\n    # Three distinct clusters\n    cluster1 = np.random.randn(50, 10) + np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n    cluster2 = np.random.randn(50, 10) + np.array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])\n    cluster3 = np.random.randn(50, 10) + np.array([10, 10, 10, 10, 10, 10, 10, 10, 10, 10])\n    return np.vstack([cluster1, cluster2, cluster3])\n\n\n@pytest.fixture\ndef default_settings():\n    \"\"\"Create default clustering settings.\"\"\"\n    settings_dict = get_default_settings_dict()\n    return ClusteringSettings(**settings_dict)\n\n\n@pytest.fixture\ndef custom_bisect_settings():\n    \"\"\"Create custom bisecting k-means settings.\"\"\"\n    settings_dict = get_default_settings_dict()\n    settings_dict[\"bisecting_kmeans\"] = {\n        \"recluster_count\": 2,\n        \"internal_recluster_count\": 3,\n        \"n_cluster\": {\n            \"lower\": 2,\n            \"upper\": 5\n        }\n    }\n    return ClusteringSettings(**settings_dict)\n\n\nclass TestBasicFunctionality:\n    \"\"\"Tests for basic bisect_k_means functionality.\"\"\"\n\n    def test_simple_clustering(self, simple_2d_data, default_settings):\n        \"\"\"Test basic clustering on simple synthetic data.\"\"\"\n        labels = bisect_k_means(simple_2d_data, default_settings)\n\n        # Check output format\n        assert isinstance(labels, np.ndarray)\n        assert len(labels) == len(simple_2d_data)\n        assert labels.dtype in [np.int32, np.int64]\n\n        # Check that we have valid cluster labels\n        assert len(np.unique(labels)) > 0\n        assert np.all(labels >= 0)\n\n    def test_reproducibility(self, simple_2d_data, default_settings):\n        \"\"\"Test that same seed produces same results.\"\"\"\n        labels1 = bisect_k_means(simple_2d_data, default_settings)\n        labels2 = bisect_k_means(simple_2d_data, default_settings)\n\n        assert np.array_equal(labels1, labels2)\n\n    def test_different_seeds(self, simple_2d_data):\n        \"\"\"Test that different seeds can produce different results.\"\"\"\n        settings_dict1 = get_default_settings_dict()\n        settings_dict1[\"seed\"] = 42\n        settings_dict2 = get_default_settings_dict()\n        settings_dict2[\"seed\"] = 123\n        settings1 = ClusteringSettings(**settings_dict1)\n        settings2 = ClusteringSettings(**settings_dict2)\n\n        labels1 = bisect_k_means(simple_2d_data, settings1)\n        labels2 = bisect_k_means(simple_2d_data, settings2)\n\n        # Labels might be different (permutation), but both should be valid\n        assert len(np.unique(labels1)) > 0\n        assert len(np.unique(labels2)) > 0\n\n\nclass TestClusterRangeConfiguration:\n    \"\"\"Tests for different cluster range configurations.\"\"\"\n\n    def test_single_cluster_specification(self, simple_2d_data):\n        \"\"\"Test clustering with a single specified number of clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n\n        # Should produce exactly 3 clusters\n        assert len(np.unique(labels)) == 3\n\n    def test_cluster_range(self, simple_2d_data, custom_bisect_settings):\n        \"\"\"Test clustering with a range of cluster numbers.\"\"\"\n        labels = bisect_k_means(simple_2d_data, custom_bisect_settings)\n\n        # Should produce between 2 and 5 clusters\n        n_clusters = len(np.unique(labels))\n        assert 2 <= n_clusters <= 5\n\n    def test_two_clusters(self, simple_2d_data):\n        \"\"\"Test clustering into exactly 2 clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 2\n\n    def test_many_clusters(self, simple_2d_data):\n        \"\"\"Test clustering with many clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 10, \"upper\": 10}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 10\n\n\nclass TestAlgorithmSettings:\n    \"\"\"Tests for different algorithm configuration settings.\"\"\"\n\n    def test_lloyd_inner_algorithm(self, simple_2d_data):\n        \"\"\"Test clustering with Lloyd inner algorithm.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"inner_algorithm\": \"lloyd\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_elkan_inner_algorithm(self, simple_2d_data):\n        \"\"\"Test clustering with Elkan inner algorithm.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"inner_algorithm\": \"elkan\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_largest_cluster_strategy(self, simple_2d_data):\n        \"\"\"Test clustering with largest cluster bisecting strategy.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"bisecting_strategy\": \"largest_cluster\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_biggest_inertia_strategy(self, simple_2d_data):\n        \"\"\"Test clustering with biggest inertia bisecting strategy.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"bisecting_strategy\": \"biggest_inertia\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_recluster_count(self, simple_2d_data):\n        \"\"\"Test that different recluster counts work correctly.\"\"\"\n        for recluster_count in [1, 3, 5]:\n            settings_dict = get_default_settings_dict()\n            settings_dict[\"bisecting_kmeans\"] = {\n                \"recluster_count\": recluster_count,\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n            }\n            settings = ClusteringSettings(**settings_dict)\n\n            labels = bisect_k_means(simple_2d_data, settings)\n            assert len(np.unique(labels)) == 3\n\n\nclass TestDataShapes:\n    \"\"\"Tests for different data shapes and sizes.\"\"\"\n\n    def test_small_dataset(self):\n        \"\"\"Test clustering on small dataset.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(10, 5)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 10\n        assert len(np.unique(labels)) == 2\n\n    def test_large_dataset(self):\n        \"\"\"Test clustering on larger dataset.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(1000, 20)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 5, \"upper\": 5}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 1000\n        assert len(np.unique(labels)) == 5\n\n    def test_high_dimensional_data(self):\n        \"\"\"Test clustering on high-dimensional data.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 50)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n    def test_low_dimensional_data(self):\n        \"\"\"Test clustering on low-dimensional data.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 2)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n\nclass TestEdgeCases:\n    \"\"\"Tests for edge cases and boundary conditions.\"\"\"\n\n    def test_more_clusters_than_samples(self):\n        \"\"\"Test clustering with more clusters requested than samples available.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(5, 10)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 10, \"upper\": 10}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        # Should raise ValueError due to insufficient samples\n        # min_cluster_size=2 (default), n_cluster_lower=10 requires > 20 samples\n        with pytest.raises(ValueError, match=\"Insufficient samples for clustering\"):\n            bisect_k_means(data, settings)\n\n    def test_uniform_data(self):\n        \"\"\"Test clustering on uniform data (no clear clusters).\"\"\"\n        np.random.seed(42)\n        data = np.random.uniform(-1, 1, (100, 10))\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 100\n        # Should still produce valid clusters even if not meaningful\n        assert len(np.unique(labels)) > 0\n\n    def test_identical_samples(self):\n        \"\"\"Test clustering when all samples are identical.\"\"\"\n        data = np.ones((50, 10))\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 50\n        # All samples might end up in different clusters arbitrarily\n        assert len(np.unique(labels)) > 0\n\n    def test_negative_values(self):\n        \"\"\"Test clustering with negative values.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 10) - 5  # Shift to negative\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n    def test_mixed_scale_features(self):\n        \"\"\"Test clustering with features at different scales.\"\"\"\n        np.random.seed(42)\n        # Create data with features at different scales\n        data = np.column_stack([\n            np.random.randn(100) * 0.01,  # Small scale\n            np.random.randn(100) * 1.0,   # Medium scale\n            np.random.randn(100) * 100.0  # Large scale\n        ])\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 2\n\n\nclass TestClusterQuality:\n    \"\"\"Tests to verify cluster quality and separation.\"\"\"\n\n    def test_well_separated_clusters(self):\n        \"\"\"Test that well-separated clusters are correctly identified.\"\"\"\n        np.random.seed(42)\n        # Create three very distinct clusters\n        cluster1 = np.random.randn(30, 5) + np.array([0, 0, 0, 0, 0])\n        cluster2 = np.random.randn(30, 5) + np.array([10, 10, 10, 10, 10])\n        cluster3 = np.random.randn(30, 5) + np.array([20, 20, 20, 20, 20])\n        data = np.vstack([cluster1, cluster2, cluster3])\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = bisect_k_means(data, settings)\n\n        # Should identify 3 clusters\n        assert len(np.unique(labels)) == 3\n\n        # Check that samples from same original cluster tend to be together\n        # (This is a heuristic check - labels might be permuted)\n        cluster_counts = {}\n        for i in range(3):\n            original_cluster_labels = labels[i*30:(i+1)*30]\n            most_common = np.bincount(original_cluster_labels).argmax()\n            cluster_counts[i] = np.sum(original_cluster_labels == most_common)\n\n        # At least most samples from each cluster should be together\n        for count in cluster_counts.values():\n            assert count >= 20  # At least 2/3 of samples correctly clustered\n\n\npytest.skip(reason=\"Works locally but fails in CI, needs investigation\", allow_module_level=True)\nclass TestBaselineConsistency:\n    \"\"\"Tests to ensure algorithm output doesn't change across versions.\"\"\"\n\n    def test_expected_baseline_output(self):\n        \"\"\"Test that bisecting k-means produces expected baseline output.\n\n        This test ensures the algorithm produces consistent results across\n        different versions of the code. If this test fails, it indicates\n        a breaking change in the clustering algorithm.\n        \"\"\"\n        # Create deterministic test data with well-separated clusters\n        data, _ = make_blobs(\n            n_samples=40,\n            n_features=10,\n            centers=3,\n            cluster_std=2.0,\n            random_state=42\n        )\n\n        # Configure settings for reproducible clustering\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"bisecting_kmeans\"] = {\n            \"n_cluster\": {\"lower\": 2, \"upper\": 4},\n            \"recluster_count\": 2,\n            \"internal_recluster_count\": 3,\n            \"inner_algorithm\": \"lloyd\",\n            \"bisecting_strategy\": \"largest_cluster\",\n            \"seed\": 42\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        # Run clustering\n        labels = bisect_k_means(data, settings)\n\n        # Expected baseline output - saved for version consistency\n        expected_labels = np.array([\n            0, 2, 0, 0, 0, 0, 2, 2, 1, 1,\n            2, 0, 2, 2, 1, 0, 1, 0, 0, 1,\n            2, 0, 1, 2, 1, 0, 0, 1, 1, 2,\n            1, 1, 1, 2, 1, 2, 2, 2, 0, 0\n        ])\n\n        # Verify exact match against baseline\n        np.testing.assert_array_equal(\n            labels,\n            expected_labels,\n            err_msg=\"Bisecting k-means output does not match saved baseline. \"\n                    \"This indicates a breaking change in the algorithm.\"\n        )\n\n        # Verify cluster properties\n        unique_labels, counts = np.unique(labels, return_counts=True)\n        expected_counts = {0: 14, 1: 13, 2: 13}\n\n        assert len(unique_labels) == 3, \"Expected 3 clusters\"\n        for label, count in zip(unique_labels, counts):\n            assert count == expected_counts[label], \\\n                f\"Cluster {label} has {count} samples, expected {expected_counts[label]}\"\n\n\nif __name__ == '__main__':\n    pytest.main([__file__, '-v'])"
  },
  {
    "path": "tests/common/clustering/test_cluster.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\n\"\"\"Comprehensive test suite for clustering cluster module.\"\"\"\n\nimport warnings\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.common.clustering.cluster import (\n    _cluster_merge,\n    cluster_reorder,\n    _cluster_features,\n    cluster_features,\n)\nfrom opendsm.common.clustering.settings import (\n    ClusteringSettings,\n    ClusterAlgorithms,\n)\n\n\n# =============================================================================\n# Fixtures\n# =============================================================================\n\n@pytest.fixture\ndef simple_data():\n    \"\"\"Create simple synthetic data for clustering.\"\"\"\n    np.random.seed(42)\n    # Create 3 distinct clusters\n    cluster1 = np.random.randn(20, 5) + np.array([0, 0, 0, 0, 0])\n    cluster2 = np.random.randn(20, 5) + np.array([10, 10, 10, 10, 10])\n    cluster3 = np.random.randn(20, 5) + np.array([-10, -10, -10, -10, -10])\n\n    data = np.vstack([cluster1, cluster2, cluster3])\n    return data\n\n\n@pytest.fixture\ndef simple_dataframe():\n    \"\"\"Create simple DataFrame for clustering.\"\"\"\n    np.random.seed(42)\n    data = np.random.randn(50, 24) * 10 + 50\n    df = pd.DataFrame(data)\n    return df\n\n\n@pytest.fixture\ndef time_series_dataframe():\n    \"\"\"Create time series DataFrame with distinct patterns.\"\"\"\n    np.random.seed(42)\n    n_samples = 60\n    n_timepoints = 24\n\n    data = []\n    for i in range(n_samples):\n        t = np.linspace(0, 2 * np.pi, n_timepoints)\n        if i < 20:\n            # Morning peak\n            pattern = 50 + 20 * np.sin(t - np.pi/4) + np.random.randn(n_timepoints) * 2\n        elif i < 40:\n            # Evening peak\n            pattern = 50 + 20 * np.sin(t + np.pi/4) + np.random.randn(n_timepoints) * 2\n        else:\n            # Flat with noise\n            pattern = 50 + np.random.randn(n_timepoints) * 5\n        data.append(pattern)\n\n    return pd.DataFrame(data)\n\n\n@pytest.fixture\ndef cluster_labels_simple():\n    \"\"\"Create simple cluster labels matching simple_dataframe length (50).\"\"\"\n    # 50 samples: 20 in cluster 0, 20 in cluster 1, 10 in cluster 2\n    return np.array([0] * 20 + [1] * 20 + [2] * 10)\n\n\n@pytest.fixture\ndef cluster_labels_with_outliers():\n    \"\"\"Create cluster labels with outliers (-1) matching simple_dataframe length (50).\"\"\"\n    # 50 samples with some outliers\n    labels = [0] * 15 + [1] * 15 + [2] * 10 + [-1] * 10\n    return np.array(labels)\n\n\n# =============================================================================\n# Tests for _cluster_merge\n# =============================================================================\n\n@pytest.mark.skip(reason=\"Skipping due to non-functioning function.\")\nclass TestClusterMerge:\n    \"\"\"Tests for _cluster_merge function.\"\"\"\n\n    def test_merge_two_similar_clusters(self):\n        \"\"\"Test merging two very similar clusters.\"\"\"\n        # Create two very similar clusters\n        np.random.seed(42)\n        data = np.vstack([\n            np.random.randn(10, 5),\n            np.random.randn(10, 5) + 0.1  # Very close to first cluster\n        ])\n        cluster_labels = np.array([0] * 10 + [1] * 10)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42\n        )\n\n        result = _cluster_merge(cluster_labels, data, settings, W=0.5)\n\n        # Result should be valid labels with same shape\n        assert result.shape == cluster_labels.shape\n        assert len(np.unique(result)) == 2\n\n    def test_keep_two_distinct_clusters(self):\n        \"\"\"Test keeping two distinct clusters.\"\"\"\n        # Create two well-separated clusters\n        data = np.vstack([\n            np.random.randn(10, 5),\n            np.random.randn(10, 5) + 20  # Far from first cluster\n        ])\n        cluster_labels = np.array([0] * 10 + [1] * 10)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42\n        )\n\n        result = _cluster_merge(cluster_labels, data, settings, W=0.5)\n\n        # Should keep both clusters\n        assert len(np.unique(result)) == 2\n\n    def test_merge_with_different_W_values(self):\n        \"\"\"Test merge behavior with different W threshold values.\"\"\"\n        data = np.vstack([\n            np.random.randn(10, 5),\n            np.random.randn(10, 5) + 5\n        ])\n        cluster_labels = np.array([0] * 10 + [1] * 10)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42\n        )\n\n        # With low W, more likely to merge\n        result_low = _cluster_merge(cluster_labels, data, settings, W=0.1)\n\n        # With high W, less likely to merge\n        result_high = _cluster_merge(cluster_labels, data, settings, W=0.9)\n\n        # Both should return valid labels\n        assert result_low.shape == cluster_labels.shape\n        assert result_high.shape == cluster_labels.shape\n\n    @pytest.mark.skip(reason=\"Skipping due to non-functioning function.\")\n    def test_merge_multiple_clusters(self, simple_data):\n        \"\"\"Test merging with more than two clusters.\"\"\"\n        # Create labels for 3 clusters\n        cluster_labels = np.array([0] * 20 + [1] * 20 + [2] * 20)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42\n        )\n\n        result = _cluster_merge(cluster_labels, simple_data, settings, W=0.5)\n\n        assert result.shape == cluster_labels.shape\n        # Result should have < 3 clusters (some may have merged)\n        assert len(np.unique(result)) < 3\n\n\n# =============================================================================\n# Tests for cluster_reorder\n# =============================================================================\n\nclass TestClusterReorder:\n    \"\"\"Tests for cluster_reorder function.\"\"\"\n\n    def test_reorder_by_size_ascending(self, simple_dataframe, cluster_labels_simple):\n        \"\"\"Test cluster reordering by size in ascending order.\n\n        Note: cluster_labels_simple has 20 in cluster 0, 20 in cluster 1, 10 in cluster 2.\n        The actual behavior sorts by size and assigns indices based on sorted order.\n        \"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"size\",\n                \"aggregation\": \"mean\",\n                \"reverse\": False\n            }\n        )\n\n        cluster_map = cluster_reorder(simple_dataframe, cluster_labels_simple, settings)\n\n        # Should return a dictionary mapping old labels to new labels\n        assert isinstance(cluster_map, dict)\n\n        # All unique labels should be in the map\n        unique_labels = np.unique(cluster_labels_simple[cluster_labels_simple >= 0])\n        for label in unique_labels:\n            assert label in cluster_map\n\n        # Verify reordering occurred - clusters should be remapped using np.unique on new values\n        new_labels = np.array([cluster_map[label] for label in cluster_labels_simple])\n        unique_new_labels = np.unique(new_labels[new_labels >= 0])\n\n        # Should have same number of unique clusters\n        assert len(unique_new_labels) == len(unique_labels)\n\n        # Smallest cluster (2 with size 10) maps to 0 based on actual behavior\n        assert cluster_map[2] == 0\n        assert cluster_map[0] in [1, 2]\n        assert cluster_map[1] in [1, 2]\n\n        # Test output values: verify counts after remapping\n        assert np.sum(new_labels == 0) == 10  # Smallest cluster\n        assert np.sum(new_labels == 1) == 20  # Medium clusters\n        assert np.sum(new_labels == 2) == 20\n        assert np.all((new_labels >= 0) & (new_labels <= 2))\n\n        # Test consistency: calling again with same inputs should produce same output\n        cluster_map_2 = cluster_reorder(simple_dataframe, cluster_labels_simple, settings)\n        new_labels_2 = np.array([cluster_map_2[label] for label in cluster_labels_simple])\n        assert cluster_map == cluster_map_2\n        np.testing.assert_array_equal(new_labels, new_labels_2)\n\n    def test_reorder_by_size_descending(self, simple_dataframe, cluster_labels_simple):\n        \"\"\"Test cluster reordering by size in descending order.\n\n        With reverse=True, largest clusters should get lowest indices (0, 1),\n        smallest cluster should get highest index (2).\n        cluster_labels_simple has 20 in cluster 0, 20 in cluster 1, 10 in cluster 2.\n        \"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"size\",\n                \"aggregation\": \"mean\",\n                \"reverse\": True\n            }\n        )\n\n        cluster_map = cluster_reorder(simple_dataframe, cluster_labels_simple, settings)\n\n        assert isinstance(cluster_map, dict)\n        assert len(cluster_map) > 0\n\n        # All unique labels should be in the map\n        unique_labels = np.unique(cluster_labels_simple[cluster_labels_simple >= 0])\n        for label in unique_labels:\n            assert label in cluster_map\n\n        # Verify reordering occurred using np.unique to check new label assignments\n        new_labels = np.array([cluster_map[label] for label in cluster_labels_simple])\n        unique_new_labels = np.unique(new_labels[new_labels >= 0])\n\n        # Should have same number of unique clusters\n        assert len(unique_new_labels) == len(unique_labels)\n\n        # Descending order: largest clusters get lowest indices, smallest gets highest\n        assert cluster_map[2] == 2  # Smallest cluster (size 10) maps to highest index\n        assert cluster_map[0] in [0, 1]  # Larger clusters map to lowest indices\n        assert cluster_map[1] in [0, 1]\n\n    def test_reorder_by_peak(self, time_series_dataframe):\n        \"\"\"Test cluster reordering by peak.\"\"\"\n        # Create labels with distinct patterns\n        cluster_labels = np.array([0] * 20 + [1] * 20 + [2] * 20)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"peak\",\n                \"aggregation\": \"mean\",\n                \"reverse\": False\n            }\n        )\n\n        cluster_map = cluster_reorder(time_series_dataframe, cluster_labels, settings)\n\n        assert isinstance(cluster_map, dict)\n        # Should map all non-outlier labels\n        unique_labels = np.unique(cluster_labels[cluster_labels >= 0])\n        for label in unique_labels:\n            assert label in cluster_map\n\n    def test_reorder_with_outliers(self, simple_dataframe, cluster_labels_with_outliers):\n        \"\"\"Test that outliers (-1) are excluded from reordering.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"size\",\n                \"aggregation\": \"mean\",\n                \"reverse\": False\n            }\n        )\n\n        cluster_map = cluster_reorder(simple_dataframe, cluster_labels_with_outliers, settings)\n\n        # Outlier label (-1) should still map to itself\n        assert -1 in cluster_map\n        assert cluster_map[-1] == -1\n\n    def test_reorder_different_aggregations(self, time_series_dataframe):\n        \"\"\"Test reordering with different aggregation methods.\"\"\"\n        cluster_labels = np.array([0] * 20 + [1] * 20 + [2] * 20)\n\n        for agg_method in [\"mean\", \"median\"]:\n            settings = ClusteringSettings(\n                algorithm_selection=ClusterAlgorithms.SPECTRAL,\n                seed=42,\n                cluster_sort={\n                    \"enable\": True,\n                    \"method\": \"peak\",\n                    \"aggregation\": agg_method,\n                    \"reverse\": False\n                }\n            )\n\n            cluster_map = cluster_reorder(time_series_dataframe, cluster_labels, settings)\n\n            assert isinstance(cluster_map, dict)\n            assert len(cluster_map) > 0\n\n\n# =============================================================================\n# Tests for _cluster_features\n# =============================================================================\n\nclass TestClusterFeaturesInternal:\n    \"\"\"Tests for _cluster_features internal function.\"\"\"\n\n    def test_bisecting_kmeans_clustering(self, simple_data):\n        \"\"\"Test clustering with bisecting k-means.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.BISECTING_KMEANS,\n            seed=42,\n            bisecting_kmeans={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = _cluster_features(simple_data, settings)\n\n        assert labels.shape[0] == simple_data.shape[0]\n        assert len(np.unique(labels)) >= 2\n        assert len(np.unique(labels)) <= 5\n\n    def test_spectral_clustering(self, simple_data):\n        \"\"\"Test clustering with spectral clustering.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = _cluster_features(simple_data, settings)\n\n        assert labels.shape[0] == simple_data.shape[0]\n        assert len(np.unique(labels)) >= 2\n\n    def test_adjust_cluster_count_for_small_data(self):\n        \"\"\"Test that cluster count is adjusted for small datasets.\"\"\"\n        # Small dataset with only 10 samples\n        small_data = np.random.randn(10, 5)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 20},  # Request more clusters than feasible\n                \"scoring\": {\"min_cluster_size\": 2}\n            }\n        )\n\n        labels = _cluster_features(small_data, settings)\n\n        # Should adjust to feasible number of clusters (10 // 2 = 5)\n        assert labels.shape[0] == small_data.shape[0]\n        assert len(np.unique(labels)) <= 5\n\n    def test_birch_clustering(self, simple_data):\n        \"\"\"Test clustering with Birch.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.BIRCH,\n            seed=42,\n            birch={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = _cluster_features(simple_data, settings)\n\n        assert labels.shape[0] == simple_data.shape[0]\n        assert len(np.unique(labels)) >= 2\n\n\n# =============================================================================\n# Tests for cluster_features (main entry point)\n# =============================================================================\n\nclass TestClusterFeatures:\n    \"\"\"Tests for cluster_features main function.\"\"\"\n\n    def test_basic_clustering(self, simple_dataframe):\n        \"\"\"Test basic clustering workflow.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = cluster_features(simple_dataframe, settings)\n\n        assert labels.shape[0] == simple_dataframe.shape[0]\n        assert len(np.unique(labels)) >= 2\n        assert len(np.unique(labels)) <= 5\n\n    def test_clustering_with_sorting(self, time_series_dataframe):\n        \"\"\"Test clustering with cluster sorting enabled.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            },\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"size\",\n                \"aggregation\": \"mean\",\n                \"reverse\": False\n            }\n        )\n\n        labels = cluster_features(time_series_dataframe, settings)\n\n        assert labels.shape[0] == time_series_dataframe.shape[0]\n        assert len(np.unique(labels)) >= 2\n\n    def test_clustering_bypass_for_many_clusters(self):\n        \"\"\"Test that clustering is bypassed when lower bound >= data size.\"\"\"\n        small_df = pd.DataFrame(np.random.randn(5, 10))\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            spectral={\n                \"n_cluster\": {\"lower\": 10, \"upper\": 20}  # Lower bound > data size\n            }\n        )\n\n        labels = cluster_features(small_df, settings)\n\n        # Should return unique label for each sample\n        assert labels.shape[0] == small_df.shape[0]\n        np.testing.assert_array_equal(labels, np.arange(len(small_df)))\n\n    def test_clustering_with_normalization(self, simple_dataframe):\n        \"\"\"Test clustering with pre-transform normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            normalize={\n                \"pre_transform\": True,\n                \"method\": \"standardize\",\n                \"axis\": 0\n            },\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = cluster_features(simple_dataframe, settings)\n\n        assert labels.shape[0] == simple_dataframe.shape[0]\n        assert len(np.unique(labels)) >= 2\n\n    def test_clustering_different_algorithms(self, simple_dataframe):\n        \"\"\"Test clustering with different algorithms.\"\"\"\n        algorithms = [\n            ClusterAlgorithms.BISECTING_KMEANS,\n            ClusterAlgorithms.SPECTRAL,\n        ]\n\n        for algo in algorithms:\n            settings_dict = {\n                \"algorithm_selection\": algo,\n                \"seed\": 42,\n                \"transform_selection\": \"wavelet\",\n                \"wavelet_transform\": {\n                    \"wavelet_name\": \"db1\",\n                    \"pca_n_components\": 5\n                }\n            }\n\n            # Add algorithm-specific settings\n            if algo in [ClusterAlgorithms.BISECTING_KMEANS, ClusterAlgorithms.SPECTRAL]:\n                algo_name = algo.value\n                settings_dict[algo_name] = {\n                    \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n                }\n\n            settings = ClusteringSettings(**settings_dict)\n            labels = cluster_features(simple_dataframe, settings)\n\n            assert labels.shape[0] == simple_dataframe.shape[0]\n            assert len(np.unique(labels)) >= 1\n\n    def test_clustering_with_fpca_transform(self, time_series_dataframe):\n        \"\"\"Test clustering with FPCA transformation.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"fpca\",\n            fpca_transform={\n                \"min_var_ratio\": 0.90\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        # Suppress deprecation warning for Fourier class\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            labels = cluster_features(time_series_dataframe, settings)\n\n        assert labels.shape[0] == time_series_dataframe.shape[0]\n        assert len(np.unique(labels)) >= 2\n\n    def test_reproducibility_with_seed(self, simple_dataframe):\n        \"\"\"Test that clustering is reproducible with same seed.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"seed\": 42\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels1 = cluster_features(simple_dataframe, settings)\n        labels2 = cluster_features(simple_dataframe, settings)\n\n        # Results should be identical with same seed\n        np.testing.assert_array_equal(labels1, labels2)\n\n    def test_clustering_small_dataset(self):\n        \"\"\"Test clustering with very small dataset.\"\"\"\n        small_df = pd.DataFrame(np.random.randn(10, 5))\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 2\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 3},\n                \"scoring\": {\"min_cluster_size\": 2}\n            }\n        )\n\n        labels = cluster_features(small_df, settings)\n\n        assert labels.shape[0] == small_df.shape[0]\n        # With 10 samples and min_cluster_size=2, max clusters should be 5\n        assert len(np.unique(labels)) <= 5\n\n    def test_clustering_preserves_index_order(self, simple_dataframe):\n        \"\"\"Test that clustering preserves the order of samples.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 5}\n            }\n        )\n\n        labels = cluster_features(simple_dataframe, settings)\n\n        # Labels should be in same order as input DataFrame\n        assert len(labels) == len(simple_dataframe)\n        # Each label should correspond to the same row index\n        for i in range(len(labels)):\n            assert isinstance(labels[i], (int, np.integer))\n\n\n# =============================================================================\n# Integration tests\n# =============================================================================\n\nclass TestClusteringIntegration:\n    \"\"\"Integration tests for complete clustering pipeline.\"\"\"\n\n    def test_full_pipeline_with_all_options(self, time_series_dataframe):\n        \"\"\"Test complete clustering pipeline with all options enabled.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.BISECTING_KMEANS,\n            seed=42,\n            normalize={\n                \"pre_transform\": True,\n                \"method\": \"min_max_quantile\",\n                \"quantile\": 0.05,\n                \"axis\": 1\n            },\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 8,\n                \"include_scale_feature\": True\n            },\n            bisecting_kmeans={\n                \"n_cluster\": {\"lower\": 3, \"upper\": 6},\n                \"scoring\": {\n                    \"min_cluster_size\": 5,\n                    \"distance_metric\": \"euclidean\"\n                }\n            },\n            cluster_sort={\n                \"enable\": True,\n                \"method\": \"peak\",\n                \"aggregation\": \"mean\",\n                \"reverse\": False\n            }\n        )\n\n        labels = cluster_features(time_series_dataframe, settings)\n\n        assert labels.shape[0] == time_series_dataframe.shape[0]\n        assert len(np.unique(labels)) >= 3\n        assert len(np.unique(labels)) <= 6\n\n    def test_pipeline_consistency_across_runs(self, simple_dataframe):\n        \"\"\"Test that pipeline produces consistent results across multiple runs.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=123,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"seed\": 123\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 4}\n            }\n        )\n\n        results = []\n        for _ in range(3):\n            labels = cluster_features(simple_dataframe, settings)\n            results.append(labels)\n\n        # All runs should produce identical results\n        for i in range(1, len(results)):\n            np.testing.assert_array_equal(results[0], results[i])\n\n    def test_exact_output_spectral_baseline(self):\n        \"\"\"Test exact output for spectral clustering against saved baseline.\n\n        This test compares clustering output against a saved baseline to ensure\n        consistency across code versions. Any deviation indicates a breaking change.\n        \"\"\"\n        # Create deterministic test data\n        np.random.seed(42)\n        data = np.random.randn(30, 10) * 5 + 25\n        df = pd.DataFrame(data)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.SPECTRAL,\n            seed=42,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 3,\n                \"seed\": 42\n            },\n            spectral={\n                \"n_cluster\": {\"lower\": 2, \"upper\": 4}\n            }\n        )\n\n        labels = cluster_features(df, settings)\n\n        # Expected output - saved baseline for version consistency\n        # Generated with seed=42, recorded as baseline\n        expected_labels = np.array([\n            1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1,\n            0, 1, 1, 0, 0, 0, 1, 1, 1, 0\n        ])\n\n        # Verify exact match against baseline\n        np.testing.assert_array_equal(labels, expected_labels,\n            err_msg=\"Spectral clustering output does not match saved baseline. \"\n                    \"This indicates a breaking change in the algorithm.\")\n\n        # Verify cluster properties\n        unique_labels, counts = np.unique(labels, return_counts=True)\n        assert len(unique_labels) == 2\n        expected_counts = {0: 9, 1: 21}\n        for label, count in zip(unique_labels, counts):\n            assert count == expected_counts[label], \\\n                f\"Cluster {label}: expected {expected_counts[label]} samples, got {count}\"\n\n    def test_exact_output_bisecting_kmeans_baseline(self):\n        \"\"\"Test exact output for bisecting k-means against saved baseline.\n\n        This test compares clustering output against a saved baseline to ensure\n        consistency across code versions. Any deviation indicates a breaking change.\n        \"\"\"\n        # Create deterministic test data\n        np.random.seed(123)\n        data = np.random.randn(40, 12) * 3 + 15\n        df = pd.DataFrame(data)\n\n        settings = ClusteringSettings(\n            algorithm_selection=ClusterAlgorithms.BISECTING_KMEANS,\n            seed=123,\n            transform_selection=\"wavelet\",\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 4,\n                \"seed\": 123\n            },\n            bisecting_kmeans={\n                \"n_cluster\": {\"lower\": 3, \"upper\": 5},\n                \"scoring\": {\n                    \"min_cluster_size\": 3,\n                    \"distance_metric\": \"euclidean\"\n                }\n            }\n        )\n\n        labels = cluster_features(df, settings)\n\n        # Expected output - saved baseline for version consistency\n        # Generated with seed=123, recorded as baseline\n        expected_labels = np.array([\n            0, 2, 0, 0, 0, 1, 0, 0, 1, 1, 2, 1, 2, 1, 1, 0, 0, 1, 2, 0,\n            0, 1, 2, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2, 2, 2, 1, 1, 2, 0, 2\n        ])\n\n        # Verify exact match against baseline\n        np.testing.assert_array_equal(labels, expected_labels,\n            err_msg=\"Bisecting K-means output does not match saved baseline. \"\n                    \"This indicates a breaking change in the algorithm.\")\n\n        # Verify cluster properties\n        unique_labels, counts = np.unique(labels, return_counts=True)\n        assert len(unique_labels) == 3\n        expected_counts = {0: 14, 1: 13, 2: 13}\n        for label, count in zip(unique_labels, counts):\n            assert count >= 3, f\"Cluster {label} has {count} samples, below minimum of 3\"\n            assert count == expected_counts[label], \\\n                f\"Cluster {label}: expected {expected_counts[label]} samples, got {count}\"\n\n\n# =============================================================================\n# Run tests\n# =============================================================================\n\nif __name__ == '__main__':\n    pytest.main([__file__, '-v', '--tb=short'])\n"
  },
  {
    "path": "tests/common/clustering/test_cluster_transform.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\n\"\"\"Comprehensive test suite for clustering transform module.\"\"\"\n\nimport warnings\nimport numpy as np\nimport pytest\n\nfrom opendsm.common.clustering.transform import (\n    _safe_standardize,\n    _fpca_base,\n    normalize,\n    fpca_transform,\n    wavelet_transform,\n    transform_features,\n    FpcaError,\n)\nfrom opendsm.common.clustering.settings import (\n    ClusteringSettings,\n    NormalizeSettings,\n    NormalizeChoice,\n    TransformChoice,\n)\n\n\n# =============================================================================\n# Fixtures\n# =============================================================================\n\n@pytest.fixture\ndef simple_time_series_data():\n    \"\"\"Create simple time series data for testing.\n\n    Returns 50 samples with 24 time points each, organized into three\n    distinct patterns: morning peak, evening peak, and flat.\n    \"\"\"\n    np.random.seed(42)\n    n_samples = 50\n    n_timepoints = 24\n\n    data = []\n    for i in range(n_samples):\n        t = np.linspace(0, 2 * np.pi, n_timepoints)\n        if i < 15:\n            # Morning peak pattern\n            pattern = 50 + 20 * np.sin(t - np.pi/4) + np.random.randn(n_timepoints) * 2\n        elif i < 30:\n            # Evening peak pattern\n            pattern = 50 + 20 * np.sin(t + np.pi/4) + np.random.randn(n_timepoints) * 2\n        else:\n            # Flat with noise\n            pattern = 50 + np.random.randn(n_timepoints) * 5\n        data.append(pattern)\n\n    return np.array(data)\n\n\n@pytest.fixture\ndef small_dataset():\n    \"\"\"Create a small dataset for edge case testing.\"\"\"\n    np.random.seed(123)\n    return np.random.randn(5, 10)\n\n\n@pytest.fixture\ndef large_dataset():\n    \"\"\"Create a larger dataset for performance testing.\"\"\"\n    np.random.seed(456)\n    return np.random.randn(200, 96)\n\n\n@pytest.fixture\ndef constant_data():\n    \"\"\"Create constant time series data (no variation).\"\"\"\n    return np.ones((10, 24)) * 42.0\n\n\n@pytest.fixture\ndef mixed_scale_data():\n    \"\"\"Create data with mixed scales (some constant, some varying).\"\"\"\n    np.random.seed(789)\n    data = np.random.randn(20, 24)\n    # Make some rows constant\n    data[0, :] = 10.0\n    data[5, :] = -5.0\n    data[10, :] = 0.0\n    return data\n\n\n# =============================================================================\n# Tests for _safe_standardize\n# =============================================================================\n\nclass TestSafeStandardize:\n    \"\"\"Comprehensive tests for _safe_standardize helper function.\"\"\"\n\n    def test_basic_standardization_scalar_scale(self):\n        \"\"\"Test basic standardization with scalar center and scale.\"\"\"\n        data = np.array([1.0, 2.0, 3.0, 4.0, 5.0])\n        center = 3.0\n        scale = 1.5\n\n        result = _safe_standardize(data, center, scale)\n        expected = (data - center) / scale\n\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_basic_standardization_array_scale(self):\n        \"\"\"Test basic standardization with array center and scale.\"\"\"\n        data = np.array([[1.0, 2.0, 3.0],\n                        [4.0, 5.0, 6.0],\n                        [7.0, 8.0, 9.0]])\n        center = np.array([4.0, 5.0, 6.0])\n        scale = np.array([2.0, 2.0, 2.0])\n\n        result = _safe_standardize(data, center, scale)\n        expected = (data - center) / scale\n\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_zero_scale_scalar(self):\n        \"\"\"Test standardization with near-zero scalar scale.\"\"\"\n        data = np.array([1.0, 2.0, 3.0, 4.0, 5.0])\n        center = 3.0\n        scale = 1e-15  # Near zero\n\n        result = _safe_standardize(data, center, scale)\n        expected = data - center  # Only centering, no scaling\n\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_zero_scale_array_single_column(self):\n        \"\"\"Test standardization with near-zero scale in one column.\"\"\"\n        data = np.array([[1.0, 2.0, 3.0],\n                        [4.0, 5.0, 6.0]])\n        center = np.array([2.0, 3.0, 4.0])\n        scale = np.array([1.0, 1e-15, 1.0])  # Middle element near zero\n\n        result = _safe_standardize(data, center, scale, threshold=1e-10)\n\n        # First and third columns should be scaled, middle only centered\n        assert result.shape == data.shape\n        np.testing.assert_array_almost_equal(result[:, 1], data[:, 1] - center[1])\n        # First and third should be scaled\n        np.testing.assert_array_almost_equal(result[:, 0], (data[:, 0] - center[0]) / scale[0])\n        np.testing.assert_array_almost_equal(result[:, 2], (data[:, 2] - center[2]) / scale[2])\n\n    def test_zero_scale_array_all_columns(self):\n        \"\"\"Test standardization when all scales are near zero.\"\"\"\n        data = np.array([[1.0, 2.0, 3.0],\n                        [4.0, 5.0, 6.0]])\n        center = np.array([2.0, 3.0, 4.0])\n        scale = np.array([1e-12, 1e-13, 1e-14])\n\n        result = _safe_standardize(data, center, scale, threshold=1e-10)\n\n        # All should only be centered\n        expected = data - center\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_2d_data_1d_scale_axis_0(self):\n        \"\"\"Test 2D data with 1D scale array (column standardization).\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(10, 5) * 3 + 10\n        center = np.mean(data, axis=0)\n        scale = np.std(data, axis=0)\n\n        result = _safe_standardize(data, center, scale)\n\n        assert result.shape == data.shape\n        # Each column should have approximately zero mean\n        np.testing.assert_array_almost_equal(np.mean(result, axis=0), 0, decimal=10)\n\n    def test_negative_values(self):\n        \"\"\"Test standardization with negative values.\"\"\"\n        data = np.array([-5.0, -2.0, 0.0, 2.0, 5.0])\n        center = 0.0\n        scale = 3.0\n\n        result = _safe_standardize(data, center, scale)\n        expected = data / scale\n\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_all_zeros(self):\n        \"\"\"Test standardization with all zeros.\"\"\"\n        data = np.zeros((5, 4))\n        center = np.zeros(4)\n        scale = np.array([1e-15, 1e-15, 1e-15, 1e-15])\n\n        result = _safe_standardize(data, center, scale)\n\n        # Should return zeros (centered but not scaled)\n        np.testing.assert_array_almost_equal(result, np.zeros_like(data))\n\n    def test_custom_threshold(self):\n        \"\"\"Test standardization with custom threshold.\"\"\"\n        data = np.array([1.0, 2.0, 3.0])\n        center = 2.0\n        scale = 0.001  # Between default threshold and custom threshold\n\n        # With stricter threshold, should scale\n        result1 = _safe_standardize(data, center, scale, threshold=1e-10)\n        np.testing.assert_array_almost_equal(result1, (data - center) / scale)\n\n        # With looser threshold, should only center\n        result2 = _safe_standardize(data, center, scale, threshold=0.01)\n        np.testing.assert_array_almost_equal(result2, data - center)\n\n    def test_scalar_scale_zero_ndim(self):\n        \"\"\"Test with 0-dimensional numpy array as scale.\"\"\"\n        data = np.array([1.0, 2.0, 3.0])\n        center = 2.0\n        scale = np.array(1.5)  # 0-d array\n\n        result = _safe_standardize(data, center, scale)\n        expected = (data - center) / scale\n\n        np.testing.assert_array_almost_equal(result, expected)\n\n    def test_mixed_scale_threshold_boundary(self):\n        \"\"\"Test behavior at exact threshold boundary.\"\"\"\n        data = np.array([[1.0, 2.0], [3.0, 4.0]])\n        center = np.array([2.0, 3.0])\n        threshold = 1e-10\n        scale = np.array([threshold, 2.0])  # Exactly at threshold\n\n        result = _safe_standardize(data, center, scale, threshold=threshold)\n\n        # At threshold, should only center (not scale)\n        np.testing.assert_array_almost_equal(result[:, 0], data[:, 0] - center[0])\n        np.testing.assert_array_almost_equal(result[:, 1], (data[:, 1] - center[1]) / scale[1])\n\n\n# =============================================================================\n# Tests for normalize\n# =============================================================================\n\nclass TestNormalize:\n    \"\"\"Comprehensive tests for normalize function.\"\"\"\n\n    # --- Tests for STANDARDIZE method ---\n\n    def test_standardize_axis_0(self, simple_time_series_data):\n        \"\"\"Test standardization along axis 0 (column-wise).\"\"\"\n        settings = NormalizeSettings(method=NormalizeChoice.STANDARDIZE, axis=0)\n        result = normalize(simple_time_series_data, settings)\n\n        # Each column should have approximately zero mean and unit variance\n        assert result.shape == simple_time_series_data.shape\n        col_means = np.mean(result, axis=0)\n        col_stds = np.std(result, axis=0)\n\n        np.testing.assert_array_almost_equal(col_means, 0, decimal=10)\n        assert np.min(col_stds) > 0.9\n\n    def test_standardize_axis_none(self, simple_time_series_data):\n        \"\"\"Test standardization over entire array (axis=None).\"\"\"\n        settings = NormalizeSettings(method=NormalizeChoice.STANDARDIZE, axis=None)\n        result = normalize(simple_time_series_data, settings)\n\n        # Global mean and std should be 0 and 1\n        assert result.shape == simple_time_series_data.shape\n        np.testing.assert_almost_equal(np.mean(result), 0, decimal=10)\n        np.testing.assert_almost_equal(np.std(result), 1, decimal=10)\n\n    def test_standardize_constant_data_axis_0(self):\n        \"\"\"Test standardization with constant data along axis 0.\"\"\"\n        data = np.ones((10, 24)) * 42.0\n        settings = NormalizeSettings(method=NormalizeChoice.STANDARDIZE, axis=0)\n        result = normalize(data, settings)\n\n        # Should be centered (all zeros after subtracting constant)\n        assert result.shape == data.shape\n        np.testing.assert_array_almost_equal(result, 0)\n\n    # --- Tests for MED_MAD method ---\n\n    def test_med_mad_axis_0(self, simple_time_series_data):\n        \"\"\"Test median-MAD normalization along axis 0.\"\"\"\n        settings = NormalizeSettings(method=NormalizeChoice.MED_MAD, axis=0)\n        result = normalize(simple_time_series_data, settings)\n\n        # Each column should have approximately zero median\n        assert result.shape == simple_time_series_data.shape\n        col_medians = np.median(result, axis=0)\n        np.testing.assert_array_almost_equal(col_medians, 0, decimal=10)\n\n    def test_med_mad_axis_none(self, simple_time_series_data):\n        \"\"\"Test median-MAD normalization over entire array.\"\"\"\n        settings = NormalizeSettings(method=NormalizeChoice.MED_MAD, axis=None)\n        result = normalize(simple_time_series_data, settings)\n\n        # Global median should be approximately 0\n        assert result.shape == simple_time_series_data.shape\n        np.testing.assert_almost_equal(np.median(result), 0, decimal=10)\n\n    def test_med_mad_robust_to_outliers_axis_0(self):\n        \"\"\"Test that MED_MAD is more robust to outliers than STANDARDIZE.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(10, 24)\n        # Add extreme outliers in some columns\n        data[0, 0] = 1000\n        data[1, 5] = -1000\n\n        settings = NormalizeSettings(method=NormalizeChoice.MED_MAD, axis=0)\n        result = normalize(data, settings)\n\n        # Should handle outliers reasonably\n        assert np.isfinite(result).all()\n\n    # --- Tests for MIN_MAX_QUANTILE method ---\n\n    def test_min_max_quantile_axis_1(self, simple_time_series_data):\n        \"\"\"Test min-max quantile normalization along axis 1.\"\"\"\n        settings = NormalizeSettings(\n            method=NormalizeChoice.MIN_MAX_QUANTILE,\n            quantile=0.05,\n            axis=1\n        )\n        result = normalize(simple_time_series_data, settings)\n\n        # Values should be roughly in range [-1, 1]\n        assert result.shape == simple_time_series_data.shape\n        assert np.min(result) >= -2  # Allow some tolerance for extreme quantiles\n        assert np.max(result) <= 2\n\n    def test_min_max_quantile_axis_0(self, simple_time_series_data):\n        \"\"\"Test min-max quantile normalization along axis 0 (column-wise).\n\n        Note: With quantile-based normalization, values outside the quantile\n        range will naturally fall outside [-1, 1], which is expected behavior.\n        \"\"\"\n        settings = NormalizeSettings(\n            method=NormalizeChoice.MIN_MAX_QUANTILE,\n            quantile=0.1,\n            axis=0\n        )\n        result = normalize(simple_time_series_data, settings)\n\n        assert result.shape == simple_time_series_data.shape\n        # All values should be finite\n        assert np.isfinite(result).all()\n\n        # Verify the normalization is correct by checking manually\n        # For each column, the 10th and 90th percentiles should map to -1 and 1\n        q = 0.1\n        for col_idx in range(simple_time_series_data.shape[1]):\n            col_data = simple_time_series_data[:, col_idx]\n            min_val, max_val = np.quantile(col_data, [q, 1 - q])\n\n            # Skip columns where min == max (constant)\n            if np.abs(min_val - max_val) < 1e-10:\n                continue\n\n            # For non-constant columns, check that the quantile values map correctly\n            # Values at the quantiles should be close to -1 and 1\n            result_col = result[:, col_idx]\n\n            # Find values close to the original quantiles\n            lower_mask = np.abs(col_data - min_val) < 1e-10\n            upper_mask = np.abs(col_data - max_val) < 1e-10\n\n            if np.any(lower_mask):\n                # Values at lower quantile should be close to -1\n                np.testing.assert_allclose(result_col[lower_mask], -1.0, rtol=1e-4, atol=1e-8)\n\n            if np.any(upper_mask):\n                # Values at upper quantile should be close to 1\n                np.testing.assert_allclose(result_col[upper_mask], 1.0, rtol=1e-4, atol=1e-8)\n\n        # The bulk of values (between 10th and 90th percentile) should be in [-1, 1]\n        # but outliers can be outside this range\n        percentiles = np.percentile(result, [10, 90])\n        assert percentiles[0] >= -1.5  # 10th percentile shouldn't be too extreme\n        assert percentiles[1] <= 1.5   # 90th percentile shouldn't be too extreme\n\n    def test_min_max_quantile_different_quantiles(self, simple_time_series_data):\n        \"\"\"Test different quantile values.\"\"\"\n        for q in [0.01, 0.05, 0.1, 0.25, 0.4]:\n            settings = NormalizeSettings(\n                method=NormalizeChoice.MIN_MAX_QUANTILE,\n                quantile=q,\n                axis=1\n            )\n            result = normalize(simple_time_series_data, settings)\n\n            assert result.shape == simple_time_series_data.shape\n            assert np.isfinite(result).all()\n\n    def test_min_max_quantile_constant_rows(self):\n        \"\"\"Test min-max quantile with some constant rows.\"\"\"\n        data = np.random.randn(10, 24)\n        data[3, :] = 5.0  # Constant row\n        data[7, :] = -3.0  # Another constant row\n\n        settings = NormalizeSettings(\n            method=NormalizeChoice.MIN_MAX_QUANTILE,\n            quantile=0.05,\n            axis=1\n        )\n        result = normalize(data, settings)\n\n        # Constant rows should be set to midpoint 0\n        assert result.shape == data.shape\n        np.testing.assert_almost_equal(result[3, :], 0)\n        np.testing.assert_almost_equal(result[7, :], 0)\n\n    # --- Edge cases ---\n\n    def test_normalize_single_column(self):\n        \"\"\"Test normalization with single column.\"\"\"\n        data = np.random.randn(50, 1)\n        settings = NormalizeSettings(method=NormalizeChoice.STANDARDIZE, axis=0)\n        result = normalize(data, settings)\n\n        assert result.shape == data.shape\n        np.testing.assert_almost_equal(np.mean(result), 0, decimal=10)\n\n    def test_normalize_small_values_axis_0(self):\n        \"\"\"Test normalization with very small values.\"\"\"\n        data = np.random.randn(10, 20) * 1e-8\n        settings = NormalizeSettings(method=NormalizeChoice.STANDARDIZE, axis=0)\n        result = normalize(data, settings)\n\n        assert result.shape == data.shape\n        assert np.isfinite(result).all()\n\n\n# =============================================================================\n# Tests for FPCA transform\n# =============================================================================\n\nclass TestFpcaError:\n    \"\"\"Tests for FpcaError exception.\"\"\"\n\n    def test_fpca_error_instantiation(self):\n        \"\"\"Test that FpcaError can be instantiated.\"\"\"\n        error = FpcaError(\"Test error message\")\n        assert str(error) == \"Test error message\"\n        assert isinstance(error, Exception)\n\n\nclass TestFpcaBase:\n    \"\"\"Tests for _fpca_base internal function.\"\"\"\n\n    def test_fpca_base_valid_input(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with valid input.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1])\n\n        # Suppress deprecation warnings\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = _fpca_base(x, simple_time_series_data, min_var_ratio=0.90)\n\n        # Should reduce dimensionality\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] < simple_time_series_data.shape[1]\n        assert result.shape[1] > 0\n\n    def test_fpca_base_invalid_min_var_ratio_too_low(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with min_var_ratio <= 0.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1])\n\n        with pytest.raises(FpcaError, match=\"min_var_ratio but be greater than 0\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=0.0)\n\n        with pytest.raises(FpcaError, match=\"min_var_ratio but be greater than 0\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=-0.1)\n\n    def test_fpca_base_invalid_min_var_ratio_too_high(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with min_var_ratio >= 1.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1])\n\n        with pytest.raises(FpcaError, match=\"min_var_ratio but be greater than 0\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=1.0)\n\n        with pytest.raises(FpcaError, match=\"min_var_ratio but be greater than 0\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=1.5)\n\n    def test_fpca_base_non_finite_x(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with non-finite x values.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1], dtype=float)\n        x[5] = np.nan\n\n        with pytest.raises(FpcaError, match=\"provided non finite values for fpca\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=0.90)\n\n    def test_fpca_base_non_finite_y(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with non-finite y values.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1])\n        data = simple_time_series_data.copy()\n        data[10, 15] = np.inf\n\n        with pytest.raises(FpcaError, match=\"provided non finite values for fpca\"):\n            _fpca_base(x, data, min_var_ratio=0.90)\n\n    def test_fpca_base_empty_x(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with empty x array.\"\"\"\n        x = np.array([])\n\n        with pytest.raises(FpcaError, match=\"provided empty values for fpca\"):\n            _fpca_base(x, simple_time_series_data, min_var_ratio=0.90)\n\n    def test_fpca_base_empty_y(self):\n        \"\"\"Test _fpca_base with empty y array.\"\"\"\n        x = np.arange(24)\n        y = np.array([]).reshape(0, 24)\n\n        with pytest.raises(FpcaError, match=\"provided empty values for fpca\"):\n            _fpca_base(x, y, min_var_ratio=0.90)\n\n    def test_fpca_base_different_var_ratios(self, simple_time_series_data):\n        \"\"\"Test _fpca_base with different variance ratios.\"\"\"\n        x = np.arange(simple_time_series_data.shape[1])\n\n        results = {}\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            for ratio in [0.70, 0.80, 0.90, 0.95, 0.99]:\n                result = _fpca_base(x, simple_time_series_data, min_var_ratio=ratio)\n                results[ratio] = result.shape[1]\n\n        # Higher variance ratio should require more components\n        assert results[0.95] >= results[0.90]\n        assert results[0.90] >= results[0.80]\n\n    def test_fpca_base_small_dataset(self, small_dataset):\n        \"\"\"Test _fpca_base with small dataset.\"\"\"\n        x = np.arange(small_dataset.shape[1])\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = _fpca_base(x, small_dataset, min_var_ratio=0.80)\n\n        assert result.shape[0] == small_dataset.shape[0]\n        assert result.shape[1] > 0\n\n\nclass TestFpcaTransform:\n    \"\"\"Tests for fpca_transform function.\"\"\"\n\n    def test_fpca_transform_basic(self, simple_time_series_data):\n        \"\"\"Test basic FPCA transformation.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            fpca_transform={\"min_var_ratio\": 0.90}\n        )\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = fpca_transform(simple_time_series_data, settings)\n\n        # Should reduce dimensionality\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] < simple_time_series_data.shape[1]\n        assert result.shape[1] > 0\n\n    def test_fpca_transform_different_var_ratios(self, simple_time_series_data):\n        \"\"\"Test FPCA with different variance ratios.\"\"\"\n        n_components = []\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            for min_var_ratio in [0.80, 0.90, 0.95, 0.97]:\n                settings = ClusteringSettings(\n                    algorithm_selection=\"bisecting_kmeans\",\n                    seed=42,\n                    transform_selection=TransformChoice.FPCA,\n                    fpca_transform={\"min_var_ratio\": min_var_ratio}\n                )\n                result = fpca_transform(simple_time_series_data, settings)\n                n_components.append(result.shape[1])\n\n        # Higher variance ratio typically needs more components\n        assert max(n_components) >= min(n_components)\n\n    def test_fpca_transform_small_dataset(self, small_dataset):\n        \"\"\"Test FPCA on small dataset.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            fpca_transform={\"min_var_ratio\": 0.85}\n        )\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = fpca_transform(small_dataset, settings)\n\n        assert result.shape[0] == small_dataset.shape[0]\n        assert result.shape[1] > 0\n\n    def test_fpca_transform_propagates_error(self, simple_time_series_data):\n        \"\"\"Test that FPCA transform propagates FpcaError.\"\"\"\n        # Create data with NaN\n        data = simple_time_series_data.copy()\n        data[0, 0] = np.nan\n\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            fpca_transform={\"min_var_ratio\": 0.90}\n        )\n\n        with pytest.raises(FpcaError):\n            fpca_transform(data, settings)\n\n    def test_fpca_transform_deterministic(self, simple_time_series_data):\n        \"\"\"Test that FPCA transform produces consistent results.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            fpca_transform={\"min_var_ratio\": 0.90}\n        )\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result1 = fpca_transform(simple_time_series_data, settings)\n            result2 = fpca_transform(simple_time_series_data, settings)\n\n        # Should produce same results with same input\n        np.testing.assert_array_almost_equal(result1, result2)\n\n\n# =============================================================================\n# Tests for wavelet transform\n# =============================================================================\n\nclass TestWaveletTransform:\n    \"\"\"Comprehensive tests for wavelet_transform function.\"\"\"\n\n    def test_wavelet_basic(self, simple_time_series_data):\n        \"\"\"Test basic wavelet transformation.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\"wavelet_name\": \"db1\", \"pca_n_components\": 5}\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        # Should have requested number of components plus scale feature\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] == 6  # 5 PCA components + 1 scale feature\n\n    def test_wavelet_without_scale_feature(self, simple_time_series_data):\n        \"\"\"Test wavelet transformation without scale feature.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        # Should have only PCA components (no scale feature)\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] == 5\n\n    def test_wavelet_different_wavelets(self, simple_time_series_data):\n        \"\"\"Test different wavelet types.\"\"\"\n        wavelets = [\"db1\", \"haar\", \"coif6\", \"sym11\"]\n\n        for wavelet_name in wavelets:\n            settings = ClusteringSettings(\n                algorithm_selection=\"bisecting_kmeans\",\n                seed=42,\n                transform_selection=TransformChoice.WAVELET,\n                wavelet_transform={\n                    \"wavelet_name\": wavelet_name,\n                    \"pca_n_components\": 5\n                }\n            )\n            result = wavelet_transform(simple_time_series_data, settings)\n\n            assert result.shape[0] == simple_time_series_data.shape[0]\n            assert result.shape[1] > 0\n\n    def test_wavelet_with_variance_ratio(self, simple_time_series_data):\n        \"\"\"Test wavelet with PCA variance ratio instead of n_components.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": None,\n                \"pca_min_variance_ratio_explained\": 0.90,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    def test_wavelet_with_mle(self, simple_time_series_data):\n        \"\"\"Test wavelet with MLE for PCA n_components.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": \"mle\",\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    def test_wavelet_with_post_normalization(self, simple_time_series_data):\n        \"\"\"Test wavelet with post-transform normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            normalize={\"pre_transform\": False, \"post_transform\": True, \"method\": \"standardize\"},\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        # Post-normalized features should have approximately zero mean and unit std\n        np.testing.assert_almost_equal(np.mean(result), 0, decimal=1)\n        np.testing.assert_almost_equal(np.std(result), 1, decimal=1)\n\n    def test_wavelet_different_n_levels(self, simple_time_series_data):\n        \"\"\"Test wavelet with different decomposition levels.\"\"\"\n        for n_levels in [None, 1, 2, 3]:\n            settings = ClusteringSettings(\n                algorithm_selection=\"bisecting_kmeans\",\n                seed=42,\n                transform_selection=TransformChoice.WAVELET,\n                wavelet_transform={\n                    \"wavelet_name\": \"db1\",\n                    \"wavelet_n_levels\": n_levels,\n                    \"pca_n_components\": 5,\n                    \"include_scale_feature\": False\n                }\n            )\n            result = wavelet_transform(simple_time_series_data, settings)\n\n            assert result.shape[0] == simple_time_series_data.shape[0]\n            assert result.shape[1] > 0\n\n    def test_wavelet_different_modes(self, simple_time_series_data):\n        \"\"\"Test wavelet with different extension modes.\"\"\"\n        modes = [\"smooth\", \"periodic\", \"zero\", \"symmetric\"]\n\n        for mode in modes:\n            settings = ClusteringSettings(\n                algorithm_selection=\"bisecting_kmeans\",\n                seed=42,\n                transform_selection=TransformChoice.WAVELET,\n                wavelet_transform={\n                    \"wavelet_name\": \"db1\",\n                    \"wavelet_mode\": mode,\n                    \"pca_n_components\": 5,\n                    \"include_scale_feature\": False\n                }\n            )\n            result = wavelet_transform(simple_time_series_data, settings)\n\n            assert result.shape[0] == simple_time_series_data.shape[0]\n            assert result.shape[1] > 0\n\n    def test_wavelet_small_dataset(self, small_dataset):\n        \"\"\"Test wavelet on small dataset.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 3\n            }\n        )\n\n        result = wavelet_transform(small_dataset, settings)\n\n        assert result.shape[0] == small_dataset.shape[0]\n        assert result.shape[1] > 0\n\n    def test_wavelet_large_dataset(self, large_dataset):\n        \"\"\"Test wavelet on larger dataset.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 10\n            }\n        )\n\n        result = wavelet_transform(large_dataset, settings)\n\n        assert result.shape[0] == large_dataset.shape[0]\n        assert result.shape[1] > 0\n\n    def test_wavelet_deterministic_with_seed(self, simple_time_series_data):\n        \"\"\"Test that wavelet transform is deterministic with same seed.\"\"\"\n        settings1 = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"seed\": 42\n            }\n        )\n\n        settings2 = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"seed\": 42\n            }\n        )\n\n        result1 = wavelet_transform(simple_time_series_data, settings1)\n        result2 = wavelet_transform(simple_time_series_data, settings2)\n\n        np.testing.assert_array_almost_equal(result1, result2)\n\n    def test_wavelet_scale_feature_is_median(self, simple_time_series_data):\n        \"\"\"Test that scale feature is the median of each row.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"include_scale_feature\": True\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        # Last column should be the median\n        expected_medians = np.median(simple_time_series_data, axis=1)\n        np.testing.assert_array_almost_equal(result[:, -1], expected_medians)\n\n\n# =============================================================================\n# Tests for transform_features (main entry point)\n# =============================================================================\n\nclass TestTransformFeatures:\n    \"\"\"Comprehensive tests for transform_features main function.\"\"\"\n\n    # --- Tests with FPCA ---\n\n    def test_transform_features_fpca_no_normalization(self, simple_time_series_data):\n        \"\"\"Test FPCA transform without normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            normalize={\"pre_transform\": False, \"post_transform\": False, \"method\": None},\n            fpca_transform={\"min_var_ratio\": 0.90}\n        )\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = transform_features(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    def test_transform_features_fpca_with_pre_normalization(self, simple_time_series_data):\n        \"\"\"Test FPCA transform with pre-normalization (axis=0).\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.FPCA,\n            normalize={\n                \"pre_transform\": True,\n                \"post_transform\": False,\n                \"method\": \"standardize\",\n                \"axis\": 0  # Use axis=0 for proper broadcasting\n            },\n            fpca_transform={\"min_var_ratio\": 0.90}\n        )\n\n        with warnings.catch_warnings():\n            warnings.simplefilter(\"ignore\", DeprecationWarning)\n            result = transform_features(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    # --- Tests with Wavelet ---\n\n    def test_transform_features_wavelet_no_normalization(self, simple_time_series_data):\n        \"\"\"Test wavelet transform without normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            normalize={\"pre_transform\": False, \"post_transform\": False, \"method\": None},\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            }\n        )\n\n        result = transform_features(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    def test_transform_features_wavelet_with_pre_normalization(self, simple_time_series_data):\n        \"\"\"Test wavelet transform with pre-normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            normalize={\n                \"pre_transform\": True,\n                \"post_transform\": False,\n                \"method\": \"min_max_quantile\",\n                \"quantile\": 0.05,\n                \"axis\": 1  # MIN_MAX_QUANTILE works with axis=1\n            },\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5\n            }\n        )\n\n        result = transform_features(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] > 0\n\n    def test_transform_features_wavelet_with_post_normalization(self, simple_time_series_data):\n        \"\"\"Test wavelet transform with post-normalization.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            normalize={\n                \"pre_transform\": False,\n                \"post_transform\": True,\n                \"method\": \"standardize\"\n            },\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = transform_features(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        # Should be globally normalized\n        np.testing.assert_almost_equal(np.mean(result), 0, decimal=1)\n\n    # --- Integration tests ---\n\n    def test_transform_features_reduces_dimensionality(self, large_dataset):\n        \"\"\"Test that transform reduces dimensionality appropriately.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 10,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = transform_features(large_dataset, settings)\n\n        # Should reduce from 96 to 10 dimensions\n        assert result.shape[1] == 10\n        assert result.shape[1] < large_dataset.shape[1]\n\n    def test_transform_features_reproducible(self, simple_time_series_data):\n        \"\"\"Test that results are reproducible with same settings.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            normalize={\n                \"pre_transform\": True,\n                \"method\": \"min_max_quantile\",\n                \"quantile\": 0.05,\n                \"axis\": 1\n            },\n            wavelet_transform={\n                \"wavelet_name\": \"db1\",\n                \"pca_n_components\": 5,\n                \"seed\": 42\n            }\n        )\n\n        result1 = transform_features(simple_time_series_data, settings)\n        result2 = transform_features(simple_time_series_data, settings)\n\n        np.testing.assert_array_almost_equal(result1, result2)\n\n\n# =============================================================================\n# Parametrized tests\n# =============================================================================\n\nclass TestParametrizedTransforms:\n    \"\"\"Parametrized tests across multiple configurations.\"\"\"\n\n    @pytest.mark.parametrize(\"wavelet\", [\"db1\", \"haar\", \"coif6\", \"sym11\"])\n    @pytest.mark.parametrize(\"n_components\", [3, 5, 10])\n    def test_wavelet_combinations(self, simple_time_series_data, wavelet, n_components):\n        \"\"\"Test combinations of wavelets and component counts.\"\"\"\n        settings = ClusteringSettings(\n            algorithm_selection=\"bisecting_kmeans\",\n            seed=42,\n            transform_selection=TransformChoice.WAVELET,\n            wavelet_transform={\n                \"wavelet_name\": wavelet,\n                \"pca_n_components\": n_components,\n                \"include_scale_feature\": False\n            }\n        )\n\n        result = wavelet_transform(simple_time_series_data, settings)\n\n        assert result.shape[0] == simple_time_series_data.shape[0]\n        assert result.shape[1] == n_components\n\n\n# =============================================================================\n# Run tests\n# =============================================================================\n\nif __name__ == '__main__':\n    pytest.main([__file__, '-v', '--tb=short'])\n"
  },
  {
    "path": "tests/common/clustering/test_spectral.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pytest\nfrom sklearn.datasets import make_blobs\n\nfrom opendsm.common.clustering.algorithms.spectral import spectral\nfrom opendsm.common.clustering.settings import ClusteringSettings\n\n\ndef get_default_settings_dict():\n    \"\"\"Return a default settings dictionary that can be modified.\"\"\"\n    return {\n        \"algorithm_selection\": \"spectral\",\n        \"seed\": 42,\n    }\n\n\n@pytest.fixture\ndef simple_2d_data():\n    \"\"\"Create simple 2D synthetic data with clear clusters.\"\"\"\n    np.random.seed(42)\n    # Three distinct clusters\n    cluster1 = np.random.randn(50, 10) + np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n    cluster2 = np.random.randn(50, 10) + np.array([5, 5, 5, 5, 5, 5, 5, 5, 5, 5])\n    cluster3 = np.random.randn(50, 10) + np.array([10, 10, 10, 10, 10, 10, 10, 10, 10, 10])\n    return np.vstack([cluster1, cluster2, cluster3])\n\n\n@pytest.fixture\ndef default_settings():\n    \"\"\"Create default clustering settings.\"\"\"\n    settings_dict = get_default_settings_dict()\n    return ClusteringSettings(**settings_dict)\n\n\n@pytest.fixture\ndef custom_spectral_settings():\n    \"\"\"Create custom spectral clustering settings.\"\"\"\n    settings_dict = get_default_settings_dict()\n    settings_dict[\"spectral\"] = {\n        \"recluster_count\": 2,\n        \"n_cluster\": {\n            \"lower\": 2,\n            \"upper\": 5\n        }\n    }\n    return ClusteringSettings(**settings_dict)\n\n\nclass TestBasicFunctionality:\n    \"\"\"Tests for basic spectral clustering functionality.\"\"\"\n\n    def test_simple_clustering(self, simple_2d_data, default_settings):\n        \"\"\"Test basic clustering on simple synthetic data.\"\"\"\n        labels = spectral(simple_2d_data, default_settings)\n\n        # Check output format\n        assert isinstance(labels, np.ndarray)\n        assert len(labels) == len(simple_2d_data)\n        assert labels.dtype in [np.int32, np.int64]\n\n        # Check that we have valid cluster labels\n        assert len(np.unique(labels)) > 0\n        assert np.all(labels >= 0)\n\n    def test_reproducibility(self, simple_2d_data, default_settings):\n        \"\"\"Test that same seed produces same results.\"\"\"\n        labels1 = spectral(simple_2d_data, default_settings)\n        labels2 = spectral(simple_2d_data, default_settings)\n\n        assert np.array_equal(labels1, labels2)\n\n    def test_different_seeds(self, simple_2d_data):\n        \"\"\"Test that different seeds can produce different results.\"\"\"\n        settings_dict1 = get_default_settings_dict()\n        settings_dict1[\"seed\"] = 42\n        settings_dict2 = get_default_settings_dict()\n        settings_dict2[\"seed\"] = 123\n        settings1 = ClusteringSettings(**settings_dict1)\n        settings2 = ClusteringSettings(**settings_dict2)\n\n        labels1 = spectral(simple_2d_data, settings1)\n        labels2 = spectral(simple_2d_data, settings2)\n\n        # Labels might be different (permutation), but both should be valid\n        assert len(np.unique(labels1)) > 0\n        assert len(np.unique(labels2)) > 0\n\n\nclass TestClusterRangeConfiguration:\n    \"\"\"Tests for different cluster range configurations.\"\"\"\n\n    def test_single_cluster_specification(self, simple_2d_data):\n        \"\"\"Test clustering with a single specified number of clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n\n        # Should produce exactly 3 clusters\n        assert len(np.unique(labels)) == 3\n\n    def test_cluster_range(self, simple_2d_data, custom_spectral_settings):\n        \"\"\"Test clustering with a range of cluster numbers.\"\"\"\n        labels = spectral(simple_2d_data, custom_spectral_settings)\n\n        # Should produce between 2 and 5 clusters\n        n_clusters = len(np.unique(labels))\n        assert 2 <= n_clusters <= 5\n\n    def test_two_clusters(self, simple_2d_data):\n        \"\"\"Test clustering into exactly 2 clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 2\n\n    def test_many_clusters(self, simple_2d_data):\n        \"\"\"Test clustering with many clusters.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 10, \"upper\": 10}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 10\n\n\nclass TestAlgorithmSettings:\n    \"\"\"Tests for different algorithm configuration settings.\"\"\"\n\n    def test_arpack_eigen_solver(self, simple_2d_data):\n        \"\"\"Test clustering with ARPACK eigen solver.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"eigen_solver\": \"arpack\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_lobpcg_eigen_solver(self, simple_2d_data):\n        \"\"\"Test clustering with LOBPCG eigen solver.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"eigen_solver\": \"lobpcg\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_kmeans_assign_labels(self, simple_2d_data):\n        \"\"\"Test clustering with k-means label assignment.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"assign_labels\": \"kmeans\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_discretize_assign_labels(self, simple_2d_data):\n        \"\"\"Test clustering with discretize label assignment.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"assign_labels\": \"discretize\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_cluster_qr_assign_labels(self, simple_2d_data):\n        \"\"\"Test clustering with cluster_qr label assignment.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"assign_labels\": \"cluster_qr\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_rbf_affinity(self, simple_2d_data):\n        \"\"\"Test clustering with RBF affinity matrix.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"affinity\": \"rbf\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_nearest_neighbors_affinity(self, simple_2d_data):\n        \"\"\"Test clustering with nearest neighbors affinity matrix.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"affinity\": \"nearest_neighbors\",\n                \"nearest_neighbors\": 10,\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_different_gamma_values(self, simple_2d_data):\n        \"\"\"Test clustering with different gamma values for RBF kernel.\"\"\"\n        for gamma in [0.5, 1.0, 2.0]:\n            settings_dict = get_default_settings_dict()\n            settings_dict[\"spectral\"] = {\n                \"affinity\": \"rbf\",\n                \"gamma\": gamma,\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n            }\n            settings = ClusteringSettings(**settings_dict)\n\n            labels = spectral(simple_2d_data, settings)\n            assert len(np.unique(labels)) == 3\n\n    def test_recluster_count(self, simple_2d_data):\n        \"\"\"Test that different recluster counts work correctly.\"\"\"\n        for recluster_count in [0, 1, 3]:\n            settings_dict = get_default_settings_dict()\n            settings_dict[\"spectral\"] = {\n                \"recluster_count\": recluster_count,\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n            }\n            settings = ClusteringSettings(**settings_dict)\n\n            labels = spectral(simple_2d_data, settings)\n            assert len(np.unique(labels)) == 3\n\n\nclass TestAffinityMatrixOptions:\n    \"\"\"Tests for different affinity matrix options.\"\"\"\n\n    def test_laplacian_affinity(self, simple_2d_data):\n        \"\"\"Test clustering with laplacian affinity matrix.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"affinity\": \"laplacian\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n    def test_chi2_affinity(self, simple_2d_data):\n        \"\"\"Test clustering with chi2 affinity matrix (requires non-negative data).\"\"\"\n        # Chi2 kernel requires non-negative data, so shift the data\n        shifted_data = simple_2d_data - simple_2d_data.min() + 1\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"affinity\": \"chi2\",\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(shifted_data, settings)\n        assert len(np.unique(labels)) == 3\n\n\nclass TestDataShapes:\n    \"\"\"Tests for different data shapes and sizes.\"\"\"\n\n    def test_small_dataset(self):\n        \"\"\"Test clustering on small dataset.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(10, 5)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 10\n        assert len(np.unique(labels)) == 2\n\n    def test_large_dataset(self):\n        \"\"\"Test clustering on larger dataset.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(1000, 20)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 5, \"upper\": 5}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 1000\n        assert len(np.unique(labels)) == 5\n\n    def test_high_dimensional_data(self):\n        \"\"\"Test clustering on high-dimensional data.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 50)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n    def test_low_dimensional_data(self):\n        \"\"\"Test clustering on low-dimensional data.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 2)\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n\nclass TestEdgeCases:\n    \"\"\"Tests for edge cases and boundary conditions.\"\"\"\n\n    def test_uniform_data(self):\n        \"\"\"Test clustering on uniform data (no clear clusters).\"\"\"\n        np.random.seed(42)\n        data = np.random.uniform(-1, 1, (100, 10))\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        # Should still produce valid clusters even if not meaningful\n        assert len(np.unique(labels)) > 0\n\n    def test_identical_samples(self):\n        \"\"\"Test clustering when all samples are identical.\"\"\"\n        data = np.ones((50, 10))\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 50\n        # All samples might end up in different clusters arbitrarily\n        assert len(np.unique(labels)) > 0\n\n    def test_negative_values(self):\n        \"\"\"Test clustering with negative values.\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 10) - 5  # Shift to negative\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 3\n\n    def test_mixed_scale_features(self):\n        \"\"\"Test clustering with features at different scales.\"\"\"\n        np.random.seed(42)\n        # Create data with features at different scales\n        data = np.column_stack([\n            np.random.randn(100) * 0.01,  # Small scale\n            np.random.randn(100) * 1.0,   # Medium scale\n            np.random.randn(100) * 100.0  # Large scale\n        ])\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 2, \"upper\": 2}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) == 2\n\n    def test_sparse_data(self):\n        \"\"\"Test clustering on sparse data (many zeros).\"\"\"\n        np.random.seed(42)\n        data = np.random.randn(100, 10)\n        # Make 70% of values zero\n        mask = np.random.random((100, 10)) < 0.7\n        data[mask] = 0\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n        assert len(labels) == 100\n        assert len(np.unique(labels)) > 0\n\n\nclass TestClusterQuality:\n    \"\"\"Tests to verify cluster quality and separation.\"\"\"\n\n    def test_well_separated_clusters(self):\n        \"\"\"Test that well-separated clusters are correctly identified.\"\"\"\n        np.random.seed(42)\n        # Create three very distinct clusters\n        cluster1 = np.random.randn(30, 5) + np.array([0, 0, 0, 0, 0])\n        cluster2 = np.random.randn(30, 5) + np.array([10, 10, 10, 10, 10])\n        cluster3 = np.random.randn(30, 5) + np.array([20, 20, 20, 20, 20])\n        data = np.vstack([cluster1, cluster2, cluster3])\n\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(data, settings)\n\n        # Should identify 3 clusters\n        assert len(np.unique(labels)) == 3\n\n        # Check that samples from same original cluster tend to be together\n        # (This is a heuristic check - labels might be permuted)\n        cluster_counts = {}\n        for i in range(3):\n            original_cluster_labels = labels[i*30:(i+1)*30]\n            most_common = np.bincount(original_cluster_labels).argmax()\n            cluster_counts[i] = np.sum(original_cluster_labels == most_common)\n\n        # At least most samples from each cluster should be together\n        for count in cluster_counts.values():\n            assert count >= 20  # At least 2/3 of samples correctly clustered\n\n\nclass TestComponentSettings:\n    \"\"\"Tests for n_components parameter.\"\"\"\n\n    def test_custom_n_components(self, simple_2d_data):\n        \"\"\"Test clustering with custom number of components.\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_components\": 5,\n                \"n_cluster\": {\"lower\": 5, \"upper\": 5}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 5\n\n    def test_none_n_components(self, simple_2d_data):\n        \"\"\"Test clustering with n_components=None (defaults to n_clusters).\"\"\"\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n                \"n_components\": None,\n                \"n_cluster\": {\"lower\": 3, \"upper\": 3}\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        labels = spectral(simple_2d_data, settings)\n        assert len(np.unique(labels)) == 3\n\n\npytest.skip(reason=\"Works locally but fails in CI, needs investigation\", allow_module_level=True)\nclass TestBaselineConsistency:\n    \"\"\"Tests to ensure algorithm output doesn't change across versions.\"\"\"\n\n    def test_expected_baseline_output(self):\n        \"\"\"Test that spectral clustering produces expected baseline output.\n\n        This test ensures the algorithm produces consistent results across\n        different versions of the code. If this test fails, it indicates\n        a breaking change in the clustering algorithm.\n        \"\"\"\n        # Create deterministic test data with well-separated clusters\n        data, _ = make_blobs(\n            n_samples=60,\n            n_features=10,\n            centers=3,\n            cluster_std=1.5,\n            random_state=42\n        )\n\n        # Configure settings for reproducible clustering\n        settings_dict = get_default_settings_dict()\n        settings_dict[\"spectral\"] = {\n            \"n_cluster\": {\"lower\": 3, \"upper\": 3},\n            \"seed\": 42,\n            \"assign_labels\": \"kmeans\"\n        }\n        settings = ClusteringSettings(**settings_dict)\n\n        # Run clustering\n        labels = spectral(data, settings)\n\n        # Expected baseline output - saved for version consistency\n        expected_labels = np.array([\n            1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 2, 0, 0, 0,\n            1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0,\n            1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 2, 0, 0\n        ])\n\n        # Verify exact match against baseline\n        np.testing.assert_array_equal(\n            labels,\n            expected_labels,\n            err_msg=\"Spectral clustering output does not match saved baseline. \"\n                    \"This indicates a breaking change in the algorithm.\"\n        )\n\n        # Verify cluster properties\n        unique_labels, counts = np.unique(labels, return_counts=True)\n        expected_counts = {0: 38, 1: 20, 2: 2}\n\n        assert len(unique_labels) == 3, \"Expected 3 clusters\"\n        for label, count in zip(unique_labels, counts):\n            assert count == expected_counts[label], \\\n                f\"Cluster {label} has {count} samples, expected {expected_counts[label]}\"\n\n\nif __name__ == '__main__':\n    pytest.main([__file__, '-v'])\n"
  },
  {
    "path": "tests/common/clustering/test_voting.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.common.clustering.voting import (\n    shulze_voting,\n    construct_voting_df,\n    _shulze_pairwise_preference,\n    _shulze_path_strength,\n    _shulze_rank_strength,\n)\n\n\n\nclass TestShulzePairwisePreference:\n    \"\"\"Test suite for pairwise preference calculation.\"\"\"\n\n    def test_basic_pairwise(self):\n        \"\"\"Test basic pairwise preference calculation.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n        })\n\n        Pd, pred = _shulze_pairwise_preference(df)\n\n        # Check output shapes\n        assert Pd.shape == (3, 3, 2)\n        assert pred.shape == (3, 3)\n\n        # Diagonal should be zero (no comparison with self)\n        for i in range(3):\n            assert Pd[i, i, 0] == 0\n            assert Pd[i, i, 1] == 0\n\n    def test_pairwise_with_weights(self):\n        \"\"\"Test pairwise preference with voter weights.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1],\n            'voter2': [1, 0],\n        })\n\n        weights = {'voter1': 2.0, 'voter2': 1.0}\n        Pd, pred = _shulze_pairwise_preference(df, voter_weights=weights)\n\n        # Voter1 with weight 2 should have more influence\n        assert Pd.shape == (2, 2, 2)\n\n    def test_pairwise_tie(self):\n        \"\"\"Test pairwise preference with tied rankings.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1],\n            'voter2': [1, 0],\n        })\n\n        # When candidates have same rank, both get 0.5 vote\n        Pd, pred = _shulze_pairwise_preference(df)\n        assert Pd.shape == (2, 2, 2)\n\n\nclass TestShulzePathStrength:\n    \"\"\"Test suite for path strength calculation.\"\"\"\n\n    def test_path_strength_preserves_shape(self):\n        \"\"\"Test that path strength preserves matrix shapes.\"\"\"\n        n = 5\n        Pd = np.random.rand(n, n, 2)\n        pred = np.arange(n*n).reshape(n, n)\n\n        Pd_result, pred_result = _shulze_path_strength(Pd.copy(), pred.copy())\n\n        assert Pd_result.shape == Pd.shape\n        assert pred_result.shape == pred.shape\n\n    def test_path_strength_basic(self):\n        \"\"\"Test basic path strength calculation.\"\"\"\n        # Create simple pairwise matrix\n        n = 3\n        Pd = np.zeros((n, n, 2))\n        pred = np.zeros((n, n))\n\n        # Set up simple preferences: 0>1, 1>2, 0>2\n        Pd[0, 1] = [2, 1]\n        Pd[1, 0] = [1, 2]\n        Pd[1, 2] = [2, 1]\n        Pd[2, 1] = [1, 2]\n        Pd[0, 2] = [3, 0]\n        Pd[2, 0] = [0, 3]\n\n        Pd_result, pred_result = _shulze_path_strength(Pd, pred)\n\n        # Check shapes\n        assert Pd_result.shape == (n, n, 2)\n        assert pred_result.shape == (n, n)\n\n        # Check that path strengths are computed correctly\n        # Candidate 0 should beat 1: direct path strength should be at least 2\n        assert Pd_result[0, 1, 0] >= 2\n        # Candidate 0 should beat 2: direct path strength should be 3\n        assert Pd_result[0, 2, 0] >= 3\n        # Candidate 1 should beat 2: direct path strength should be at least 2\n        assert Pd_result[1, 2, 0] >= 2\n\n        # Check reciprocal relationships (losing side)\n        assert Pd_result[1, 0, 0] <= 2  # 1 loses to 0\n        assert Pd_result[2, 0, 0] <= 1  # 2 loses to 0\n        assert Pd_result[2, 1, 0] <= 2  # 2 loses to 1\n\n\nclass TestShulzeRankStrength:\n    \"\"\"Test suite for rank strength calculation.\"\"\"\n\n    def test_rank_strength_basic(self):\n        \"\"\"Test basic rank strength calculation.\"\"\"\n        n = 3\n        Pd = np.zeros((n, n, 2))\n        pred = np.zeros((n, n))\n\n        # Set up preferences where candidate 0 beats all others\n        Pd[0, 1] = [10, 5]\n        Pd[1, 0] = [5, 10]\n        Pd[0, 2] = [10, 5]\n        Pd[2, 0] = [5, 10]\n        Pd[1, 2] = [6, 6]  # Tie\n        Pd[2, 1] = [6, 6]\n\n        wins = _shulze_rank_strength(Pd, pred)\n\n        assert wins.shape == (n,)\n        assert wins[0] == 2  # Candidate 0 beats both others\n        assert wins[1] == 0  # Candidate 1 loses to 0, ties with 2\n        assert wins[2] == 0  # Candidate 2 loses to 0, ties with 1\n\n    def test_rank_strength_all_lose(self):\n        \"\"\"Test rank strength when candidates all lose equally.\"\"\"\n        n = 3\n        Pd = np.zeros((n, n, 2))\n        pred = np.zeros((n, n))\n\n        # Everyone ties with everyone\n        for i in range(n):\n            for j in range(n):\n                if i != j:\n                    Pd[i, j] = [5, 5]\n\n        wins = _shulze_rank_strength(Pd, pred)\n\n        assert wins.shape == (n,)\n        assert np.all(wins == 0)\n\n\nclass TestConstructVotingDF:\n    \"\"\"Test suite for voting dataframe construction.\"\"\"\n\n    def test_construct_basic(self):\n        \"\"\"Test basic voting dataframe construction.\"\"\"\n        # Create mock results\n        class MockResult:\n            def __init__(self, n_clusters, scores):\n                self.n_clusters = n_clusters\n                self.score = scores\n\n        results = [\n            MockResult(2, {'algo1': 10, 'algo2': 20}),\n            MockResult(3, {'algo1': 5, 'algo2': 15}),\n            MockResult(4, {'algo1': 8, 'algo2': 12}),\n        ]\n\n        df = construct_voting_df(results)\n\n        assert isinstance(df, pd.DataFrame)\n        assert len(df.columns) > 0\n\n    def test_construct_with_nan(self):\n        \"\"\"Test voting df construction handles NaN values.\"\"\"\n        class MockResult:\n            def __init__(self, n_clusters, scores):\n                self.n_clusters = n_clusters\n                self.score = scores\n\n        results = [\n            MockResult(2, {'algo1': 10, 'algo2': np.nan}),\n            MockResult(3, {'algo1': np.nan, 'algo2': 15}),\n        ]\n\n        df = construct_voting_df(results)\n\n        # NaN should be replaced with inf and potentially dropped\n        assert isinstance(df, pd.DataFrame)\n\n    def test_construct_with_inf(self):\n        \"\"\"Test voting df construction handles inf values.\"\"\"\n        class MockResult:\n            def __init__(self, n_clusters, scores):\n                self.n_clusters = n_clusters\n                self.score = scores\n\n        results = [\n            MockResult(2, {'algo1': 10, 'algo2': -np.inf}),\n            MockResult(3, {'algo1': 5, 'algo2': 15}),\n        ]\n\n        df = construct_voting_df(results)\n\n        assert isinstance(df, pd.DataFrame)\n\n\nclass TestShulzeVoting:\n    \"\"\"Test suite for Shulze voting method.\"\"\"\n\n    def test_simple_majority(self):\n        \"\"\"Test that clear majority winner is selected.\"\"\"\n        # Create voting data where candidate 2 is clearly the best\n        # Each column is a voter, each value is a ranking\n        df = pd.DataFrame({\n            'voter1': [2, 1, 0, 3],  # prefers candidate 2\n            'voter2': [2, 1, 0, 3],  # prefers candidate 2\n            'voter3': [1, 2, 0, 3],  # prefers candidate 2\n        })\n\n        winner = shulze_voting(df)\n        assert winner == 2\n\n    def test_condorcet_winner(self):\n        \"\"\"Test that Condorcet winner (beats all others head-to-head) is selected.\"\"\"\n        # Candidate 1 should win in all pairwise comparisons\n        df = pd.DataFrame({\n            'voter1': [1, 0, 2],\n            'voter2': [1, 2, 0],\n            'voter3': [2, 1, 0],\n        })\n\n        winner = shulze_voting(df)\n        assert winner == 1\n\n    def test_unanimous_vote(self):\n        \"\"\"Test unanimous voting scenario.\"\"\"\n        # All voters prefer candidates in the same order\n        df = pd.DataFrame({\n            'voter1': [3, 1, 0, 2],\n            'voter2': [3, 1, 0, 2],\n            'voter3': [3, 1, 0, 2],\n            'voter4': [3, 1, 0, 2],\n        })\n\n        winner = shulze_voting(df)\n        assert winner == 3\n\n    def test_weighted_voting(self):\n        \"\"\"Test voting with weighted voters.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n            'voter3': [2, 1, 0],\n        })\n\n        # Give voter1 much higher weight\n        weights = {'voter1': 10.0, 'voter2': 1.0, 'voter3': 1.0}\n        winner = shulze_voting(df, voter_weights=weights)\n        assert winner == 0\n\n    def test_tie_breaking(self):\n        \"\"\"Test that ties are broken consistently (selects smallest candidate).\"\"\"\n        # Create a perfect tie scenario\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 2, 0],\n            'voter3': [2, 0, 1],\n        })\n\n        winner = shulze_voting(df)\n        # Should select smallest candidate in case of tie\n        assert isinstance(winner, (int, np.integer))\n        assert winner == 0\n\n    def test_single_candidate(self):\n        \"\"\"Test with only one candidate.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0],\n            'voter2': [0],\n            'voter3': [0],\n        })\n\n        winner = shulze_voting(df)\n        assert winner == 0\n\n    def test_two_candidates(self):\n        \"\"\"Test with two candidates.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [1, 0],\n            'voter2': [1, 0],\n            'voter3': [0, 1],\n        })\n\n        winner = shulze_voting(df)\n        # Candidate 1 wins 2-1\n        assert winner == 1\n\n    def test_two_strong_candidates(self):\n        \"\"\"Test with two candidates that dominate others.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2, 3, 4],\n            'voter2': [1, 0, 2, 3, 4],\n            'voter3': [0, 1, 2, 3, 4],\n            'voter4': [1, 0, 2, 3, 4],\n        })\n\n        winner = shulze_voting(df)\n\n        # Winner should be one of the top two\n        assert winner in [0, 1]\n\n    def test_return_preference_df(self):\n        \"\"\"Test structure of returned preference dataframe.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2, 3],\n            'voter2': [1, 0, 2, 3],\n            'voter3': [2, 1, 0, 3],\n        })\n\n        winner, pref_df = shulze_voting(df, return_preference_df=True)\n\n        # Check preference df structure\n        assert isinstance(pref_df, pd.DataFrame)\n        assert len(pref_df) == len(df)\n        assert 'wins' in pref_df.columns\n        assert pref_df['wins'].dtype in [np.int64, np.int32, int]\n        assert winner == pref_df['wins'].idxmax()\n\n    def test_window_smoothing(self):\n        \"\"\"Test that window_size parameter applies smoothing.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2, 3, 4],\n            'voter2': [1, 0, 2, 3, 4],\n            'voter3': [2, 1, 0, 3, 4],\n        })\n\n        winner_no_smooth = shulze_voting(df, window_size=0)\n        winner_smooth = shulze_voting(df, window_size=2)\n\n        # Both should return valid winners\n        assert 0 <= winner_no_smooth < len(df)\n        assert 0 <= winner_smooth < len(df)\n\n        # Smoothing should change the result\n        assert winner_no_smooth != winner_smooth  \n\n        # Smoothing winner should be 0 because it gets more weight from nearby candidates\n        assert winner_smooth == 0\n\n    def test_empty_voter_weights(self):\n        \"\"\"Test that None voter_weights defaults to equal weights.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n            'voter3': [2, 1, 0],\n        })\n\n        winner_no_weights = shulze_voting(df, voter_weights=None)\n        winner_equal_weights = shulze_voting(df, voter_weights={'voter1': 1.0, 'voter2': 1.0, 'voter3': 1.0})\n\n        # Should produce same result\n        assert winner_no_weights == winner_equal_weights\n\n    def test_weight_normalization(self):\n        \"\"\"Test that voter weights are normalized correctly.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n        })\n\n        # These should be equivalent after normalization\n        weights1 = {'voter1': 1.0, 'voter2': 1.0}\n        weights2 = {'voter1': 10.0, 'voter2': 10.0}\n\n        winner1 = shulze_voting(df, voter_weights=weights1)\n        winner2 = shulze_voting(df, voter_weights=weights2)\n\n        assert winner1 == winner2\n\n    def test_large_number_of_candidates(self):\n        \"\"\"Test with larger number of candidates.\"\"\"\n        n_candidates = 10\n        df = pd.DataFrame({\n            'voter1': np.arange(n_candidates),\n            'voter2': np.arange(n_candidates)[::-1],\n            'voter3': np.roll(np.arange(n_candidates), 3),\n            'voter4': np.roll(np.arange(n_candidates), -2),\n        })\n\n        winner = shulze_voting(df)\n        assert 0 <= winner < n_candidates\n\n\nclass TestShulzeVotingEdgeCases:\n    \"\"\"Test edge cases and error conditions.\"\"\"\n\n    def test_single_voter(self):\n        \"\"\"Test with single voter.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2, 3],\n        })\n\n        winner = shulze_voting(df)\n        # With single voter, best candidate should win\n        assert winner == 0\n\n    def test_zero_weights(self):\n        \"\"\"Test behavior with zero weights.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n        })\n\n        # One voter has zero weight\n        weights = {'voter1': 1.0, 'voter2': 0.0}\n        winner = shulze_voting(df, voter_weights=weights)\n\n        # Should be same as if voter2 doesn't exist\n        assert isinstance(winner, (int, np.integer))\n\n    def test_negative_window_size(self):\n        \"\"\"Test that negative window size is treated as zero.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n        })\n\n        # Should not raise error\n        winner = shulze_voting(df, window_size=-1)\n        assert isinstance(winner, (int, np.integer))\n\n    def test_missing_voter_weights(self):\n        \"\"\"Test behavior when weights dict doesn't include all voters.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 0, 2],\n            'voter3': [2, 1, 0],\n        })\n\n        # Only provide weights for some voters\n        weights = {'voter1': 2.0, 'voter2': 1.0}\n        winner = shulze_voting(df, voter_weights=weights)\n\n        # Should handle gracefully\n        assert isinstance(winner, (int, np.integer))\n        assert 0 <= winner < len(df)\n\n    def test_extreme_weight_differences(self):\n        \"\"\"Test with very large weight differences.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [2, 1, 0],\n            'voter3': [1, 2, 0],\n        })\n\n        # One voter with extremely high weight should dominate\n        weights = {'voter1': 1e10, 'voter2': 1.0, 'voter3': 1.0}\n        winner = shulze_voting(df, voter_weights=weights)\n\n        # Should match voter1's top choice\n        assert winner == 0\n\n    def test_cyclic_preferences(self):\n        \"\"\"Test Condorcet paradox (cyclic preferences: A>B>C>A).\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],  # A > B > C\n            'voter2': [1, 2, 0],  # B > C > A\n            'voter3': [2, 0, 1],  # C > A > B\n        })\n\n        winner = shulze_voting(df)\n\n        # Shulze method should still produce a winner\n        assert isinstance(winner, (int, np.integer))\n        assert 0 <= winner < 3\n\n    def test_large_window_size(self):\n        \"\"\"Test with window size larger than number of candidates.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2, 3],\n            'voter2': [1, 0, 2, 3],\n            'voter3': [2, 1, 0, 3],\n        })\n\n        # Window size larger than candidate count\n        winner = shulze_voting(df, window_size=100)\n\n        assert isinstance(winner, (int, np.integer))\n        assert 0 <= winner < len(df)\n\n    def test_weighted_voting_normalization(self):\n        \"\"\"Test that results are consistent regardless of weight scale.\"\"\"\n        df = pd.DataFrame({\n            'voter1': [0, 1, 2],\n            'voter2': [1, 2, 0],\n        })\n\n        # Try different weight scales\n        weights_small = {'voter1': 0.3, 'voter2': 0.7}\n        weights_large = {'voter1': 300.0, 'voter2': 700.0}\n\n        winner_small = shulze_voting(df, voter_weights=weights_small)\n        winner_large = shulze_voting(df, voter_weights=weights_large)\n\n        # Should produce same result\n        assert winner_small == winner_large\n\n    def test_very_large_number_of_candidates(self):\n        \"\"\"Test scalability with many candidates.\"\"\"\n        n_candidates = 50\n        df = pd.DataFrame({\n            'voter1': np.arange(n_candidates),\n            'voter2': np.arange(n_candidates)[::-1],\n            'voter3': np.roll(np.arange(n_candidates), 5),\n        })\n\n        winner = shulze_voting(df)\n\n        assert isinstance(winner, (int, np.integer))\n        assert 0 <= winner < n_candidates\n\n    def test_empty_dataframe_no_rows_no_cols(self):\n        \"\"\"Test with completely empty DataFrame (no rows, no columns).\"\"\"\n        df = pd.DataFrame()\n\n        with pytest.raises((IndexError, ValueError, KeyError)):\n            shulze_voting(df)\n\n    def test_empty_dataframe_no_rows(self):\n        \"\"\"Test with DataFrame that has columns but no rows.\"\"\"\n        df = pd.DataFrame(columns=['voter1', 'voter2', 'voter3'])\n\n        with pytest.raises((IndexError, ValueError, KeyError)):\n            shulze_voting(df)\n\n    def test_empty_dataframe_no_columns(self):\n        \"\"\"Test with DataFrame that has rows but no columns (no voters).\"\"\"\n        df = pd.DataFrame(index=[0, 1, 2])\n\n        with pytest.raises((IndexError, ValueError, KeyError)):\n            shulze_voting(df)\n\n\nif __name__ == '__main__':\n    pytest.main([__file__, '-v'])\n"
  },
  {
    "path": "tests/common/metrics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport pytest\nimport numpy as np\n\nfrom opendsm.common.metrics import acf\n\n\n# TODO: this is incomplete, need to add more tests\ndef test_acf():\n    # Test case 1: Test with a simple input array\n    x = np.array([1, 2, 3, 4, 5])\n    expected_output = np.array([1.0, 0.4, -0.1, -0.4])\n    assert np.allclose(acf(x), expected_output)\n\n    # Test case 3: Test with a moving mean and standard deviation\n    x = np.array([1, 2, 3, 4, 5])\n    expected_output = np.array([1.0, 1.0, 1.0, 1.0])\n    assert np.allclose(acf(x, ac_type=\"moving_stats\"), expected_output)\n\n    # Test case 4: Test with a specific lag_n\n    x = np.array([1, 2, 3, 4, 5])\n    expected_output = np.array([1.0, 0.4])\n    assert np.allclose(acf(x, lag_n=1), expected_output)"
  },
  {
    "path": "tests/common/test_basic_stats.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport numba\nimport pytest\n\nfrom opendsm.common.stats.basic import (\n    t_stat,\n    unc_factor,\n    median_absolute_deviation,\n    fast_std,\n)\n\n\ndef test_t_stat():\n    # Test case 1: Test with a two-tailed test\n    alpha = 0.05\n    n = 10\n    tail = 2\n    result = t_stat(alpha, n, tail)\n    expected = 2.262\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 2: Test with a one-tailed test\n    alpha = 0.05\n    n = 10\n    tail = 1\n    result = t_stat(alpha, n, tail)\n    expected = 1.833\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 3: Test with a larger sample size\n    alpha = 0.01\n    n = 100\n    tail = 2\n    result = t_stat(alpha, n, tail)\n    expected = 2.626\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 4: Test with a custom alpha value\n    alpha = 0.10\n    n = 10\n    tail = 2\n    result = t_stat(alpha, n, tail)\n    expected = 1.833\n    assert np.isclose(result, expected, rtol=1e-3)\n\n\ndef test_unc_factor():\n    # Test case 1: Test with a confidence interval\n    n = 10\n    interval = \"CI\"\n    alpha = 0.05\n    result = unc_factor(n, interval, alpha)\n    expected = 0.715\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 2: Test with a prediction interval\n    n = 10\n    interval = \"PI\"\n    alpha = 0.05\n    result = unc_factor(n, interval, alpha)\n    expected = 2.977\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 3: Test with a larger sample size\n    n = 100\n    interval = \"CI\"\n    alpha = 0.01\n    result = unc_factor(n, interval, alpha)\n    expected = 0.2626\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 4: Test with a custom alpha value\n    n = 10\n    interval = \"PI\"\n    alpha = 0.10\n    result = unc_factor(n, interval, alpha)\n    expected = 2.412\n    assert np.isclose(result, expected, rtol=1e-3)\n\n\ndef test_median_absolute_deviation():\n    # Test case 1: Test with a small array\n    x = np.array([1, 2, 3, 4, 5])\n    result = median_absolute_deviation(x)\n    expected = 1.4826\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 2: Test with a larger array\n    x = np.random.normal(0, 1, 1000)\n    result = median_absolute_deviation(x)\n    expected = 1\n    assert np.isclose(result, expected, rtol=1)\n\n    # Test case 3: Test with an array of zeros\n    x = np.zeros(10)\n    result = median_absolute_deviation(x)\n    expected = 0\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 4: Test with an array of ones\n    x = np.ones(10)\n    result = median_absolute_deviation(x)\n    expected = 0\n    assert np.isclose(result, expected, rtol=1e-3)\n\n\ndef test_fast_std():\n    # Test case 1: Test with no weights and no mean\n    x = np.array([1, 2, 3, 4, 5])\n    result = fast_std(x)\n    expected = 1.4142\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 2: Test with custom weights and no mean\n    x = np.array([1, 2, 3, 4, 5])\n    weights = np.array([0.1, 0.2, 0.3, 0.3, 0.1])\n    result = fast_std(x, weights)\n    expected = 1.270\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 3: Test with custom weights and custom mean\n    x = np.array([1, 2, 3, 4, 5])\n    weights = np.array([0.1, 0.2, 0.3, 0.2, 0.2])\n    mean = 3\n    result = fast_std(x, weights, mean)\n    expected = 1.414\n    assert np.isclose(result, expected, rtol=1e-3)\n\n    # Test case 4: Test with a small array\n    x = np.array([1])\n    result = fast_std(x)\n    expected = 0\n    assert np.isclose(result, expected, rtol=1e-3)\n"
  },
  {
    "path": "tests/common/test_utils.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport numba\nimport pytest\n\nfrom opendsm.common.utils import (\n    np_clip,\n    OoM,\n    RoundToSigFigs)\n\n\n\ndef test_np_clip():\n    # Test case 1: Test with a scalar input\n    a = 5\n    a_min = 0\n    a_max = 10\n    result = np_clip(a, a_min, a_max)(a, a_min, a_max)\n    expected = 5\n    assert np.allclose(result, expected)\n\n    # Test case 2: Test with an array input\n    a = np.array([1, 2, 3, 4, 5])\n    a_min = 2\n    a_max = 4\n    result = np_clip(a, a_min, a_max)(a, a_min, a_max)\n    expected = np.array([2, 2, 3, 4, 4])\n    assert np.allclose(result, expected)\n\n    # Test case 3: Test with NaN values\n    \"\"\"\n    We use the ~ operator to invert the boolean mask created by np.isnan(a), which replaces the NaN values with False. \n    We then use this mask to select the non-NaN values from the input array and the expected output array, and compare \n    them using np.allclose. This is to handle the issue where np.allclose returns False even though the result is the same as expected.\n    \"\"\"\n    a = np.array([1, 2, np.nan, 4, 5])\n    a_min = 2\n    a_max = 4\n    mask = ~np.isnan(a)\n    result = np_clip(a[mask], a_min, a_max)(a[mask], a_min, a_max)\n    expected = np.array([2, 2, np.nan, 4, 4])[mask]\n    print(result, expected)\n    assert np.allclose(result, expected)\n\n    # Test case 4: Test with a_min > a_max (should raise ValueError)\n    a = np.array([1, 2, 3, 4, 5])\n    a_min = 4\n    a_max = 2\n    try:\n        np_clip(a, a_min, a_max)(a, a_min, a_max)\n    except ValueError as e:\n        assert str(e) == \"a_min must be less than or equal to a_max\"\n\n\ndef test_OoM():\n    # Test case 1: Test with a scalar input - should give an error as the declaration must have an array input\n    x = 5000\n    with pytest.raises(Exception) as e:\n        OoM(x)\n    assert e.type in [\n        numba.core.errors.TypingError,\n        TypeError,\n    ]  # will depend whether using JIT\n\n    # Test case 2: Test with an array input\n    x = np.array([100, 1000, 10000, 100000])\n    result = OoM(x)\n    expected = np.array([2, 3, 4, 5])\n    assert np.allclose(result, expected)\n\n    # Test case 4: Test with a ceiling rounding method\n    x = np.array([99, 999, 9999, 99999])\n    result = OoM(x, method=\"ceil\")\n    expected = np.array([2, 3, 4, 5])\n    assert np.allclose(result, expected)\n\n    # Test case 4: Test with a floor rounding method\n    x = np.array([101, 1001, 10001, 100001])\n    result = OoM(x, method=\"floor\")\n    expected = np.array([2, 3, 4, 5])\n    assert np.allclose(result, expected)\n\n    # Test case 5: Test with an exact rounding method\n    x = np.array([101, 1001, 10001, 100001])\n    result = OoM(x, method=\"exact\")\n    expected = np.array([2, 3, 4, 5])\n    assert np.allclose(result, expected)\n\n    # Test case 6: Test with a non-integer input\n    x = [1234.5678]\n    result = OoM(x)\n    expected = 3\n    assert result == expected\n\n\ndef test_RoundToSigFigs():\n    # Test case 1: Test with a scalar input\n    x = [1234.5678]\n    p = 3\n    result = RoundToSigFigs(x, p)\n    expected = 1230\n    assert result == expected\n\n    # Test case 2: Test with an array input\n    x = np.array([1234.5678, 5678.1234, 0.0123456])\n    p = 4\n    result = RoundToSigFigs(x, p)\n    expected = np.array([1.235e03, 5.680e03, 1.235e-02])\n    assert np.allclose(result, expected)\n\n    # Test case 3: Test with a zero input\n    x = [0]\n    p = 3\n    result = RoundToSigFigs(x, p)\n    expected = 0\n    assert result == expected\n\n    # Test case 4: Test with a negative input\n    x = [-1234.5678]\n    p = 3\n    result = RoundToSigFigs(x, p)\n    expected = -1230\n    assert result == expected\n"
  },
  {
    "path": "tests/comparison_groups/conftest.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\n\n@pytest.fixture\ndef col_name():\n    return 'col1'\n\n\n@pytest.fixture\ndef df_treatment(col_name):\n    return pd.DataFrame(\n        [\n            {\"id\": f\"id_treatment_{x}\", col_name: x}\n            for x in (\n                list(np.arange(0, 2, 0.1))\n                + list(np.arange(2, 4, 0.5))\n                + list(np.arange(4, 6, 1))\n                + list(np.arange(6, 10, 0.2))\n            )\n        ]\n    )\n\n\n@pytest.fixture\ndef df_pool(col_name):\n    return pd.DataFrame(\n        [\n            {\"id\": f\"id_pool_{x}\", col_name: x}\n            for x in np.arange(0, 20, 0.01)\n        ]\n    )\n\n\n\n@pytest.fixture\ndef df_equiv(df_treatment, df_pool):\n    df_treatment_records = pd.DataFrame(\n        [\n            {\n                \"id\": dim_project_site_meter_id,\n                \"month\": month,\n                \"baseline_predicted_usage\": month*i,\n            }\n            for month in range(1, 13)\n            for i, dim_project_site_meter_id in enumerate(df_treatment[\"id\"].values)\n        ]\n    )\n    df_pool_records = pd.DataFrame(\n        [\n            {\n                \"id\": dim_project_site_meter_id,\n                \"month\": month,\n                \"baseline_predicted_usage\": (13 - month) * i * 0.1,\n            }\n            for month in range(1, 13)\n            for i, dim_project_site_meter_id in enumerate(df_pool[\"id\"].values)\n        ]\n    )\n    return pd.concat([df_treatment_records, df_pool_records])\n\n\n@pytest.fixture\ndef equivalence_feature_matrix(df_equiv):\n    df = df_equiv.pivot(index=\"id\", columns=[\"month\"], values=\"baseline_predicted_usage\")\n    return df.to_numpy()\n\n\n@pytest.fixture\ndef equivalence_feature_ids(df_equiv):\n    df = df_equiv.pivot(index=\"id\", columns=[\"month\"], values=\"baseline_predicted_usage\")\n    return df.index.unique()"
  },
  {
    "path": "tests/comparison_groups/imm/test_distance_calc_selection.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport random\n\nimport pandas as pd\n\nfrom opendsm.comparison_groups.individual_meter_matching.settings import Settings\nfrom opendsm.comparison_groups.individual_meter_matching.distance_calc_selection import DistanceMatching\n\n\ndef generate_group(n_entries, make_random=True, non_random_value=5, id_prefix=\"t\"):\n    return pd.DataFrame(\n        [\n            {\n                \"id\": f\"{id_prefix}_{i}\",\n                \"month_1\": random.random() if make_random else non_random_value,\n                \"month_2\": random.random() if make_random else non_random_value,\n                \"month_3\": random.random() if make_random else non_random_value,\n            }\n            for i in range(1, n_entries + 1)\n        ]\n    ).set_index(\"id\")\n\n\ndef test_distance_match():\n    random.seed(1)\n    n_treatment = 10\n    n_pool = 100\n    n_matches_per_treatment = 4\n    allow_duplicate_matches = False\n\n    comparison_pool = pd.DataFrame(\n        [\n            {\n                \"id\": f\"c_{i}\",\n                \"month_1\": random.random(),\n                \"month_2\": random.random(),\n                \"month_3\": random.random(),\n            }\n            for i in range(1, n_pool + 1)\n        ]\n    ).set_index(\"id\")\n    treatment_group = generate_group(n_treatment, make_random=True)\n    comparison_pool = generate_group(n_pool, make_random=True, id_prefix=\"c\")\n    for selection_method in [\"minimize_meter_distance\", \"minimize_loadshape_distance\"]:\n        settings = Settings(\n            selection_method=selection_method,\n            n_matches_per_treatment=n_matches_per_treatment,\n            allow_duplicate_matches=allow_duplicate_matches,\n        )\n        IMM = DistanceMatching(\n            settings=settings\n        )\n        \n        comparison_group = IMM.get_comparison_group(\n            treatment_group=treatment_group,\n            comparison_pool=comparison_pool\n        )\n        assert not comparison_group.empty\n\n\ndef test_distance_match_duplicates_allowed():\n    random.seed(1)\n    n_treatment = 10\n    n_pool = 5\n    selection_method = \"minimize_meter_distance\"\n    allow_duplicate_matches = True\n    n_matches_per_treatment = 1\n\n    # this will run out of comparison pool meters and therefore still have duplicates\n    treatment_group = generate_group(n_treatment, make_random=True)\n    comparison_pool = generate_group(n_pool, make_random=False, id_prefix=\"c\")\n    settings = Settings(\n        selection_method=selection_method,\n        n_matches_per_treatment=n_matches_per_treatment,\n        allow_duplicate_matches=allow_duplicate_matches,\n    )\n    IMM = DistanceMatching(\n        settings=settings\n    )\n    comparison_group = IMM.get_comparison_group(\n        treatment_group=treatment_group,\n        comparison_pool=comparison_pool\n    )\n    assert comparison_group[\"duplicated\"].any()\n\n\ndef test_distance_match_duplicates_forbidden():\n    random.seed(1)\n    n_treatment = 8\n    n_pool = 10\n    selection_method = \"minimize_meter_distance\"\n    allow_duplicate_matches = False\n    n_matches_per_treatment = 1\n\n    # this will run through the 'duplicates' loop several times before finding unique values\n    # however since here are more 'max runs allowed' than treatment meters, it will be\n    # able to iterate enough times to find unique matches\n\n    treatment_group = generate_group(n_treatment, make_random=True)\n    comparison_pool = generate_group(n_pool, make_random=False)\n    settings = Settings(\n        selection_method=selection_method,\n        n_matches_per_treatment=n_matches_per_treatment,\n        allow_duplicate_matches=allow_duplicate_matches,\n    )\n    IMM = DistanceMatching(\n        settings=settings\n    )\n    comparison_group = IMM.get_comparison_group(\n        treatment_group=treatment_group,\n        comparison_pool=comparison_pool\n    )\n    assert not comparison_group[\"duplicated\"].any()\n\n\ndef test_distance_match_large_treatments():\n    random.seed(1)\n\n    n_treatment = 10000\n    n_pool = 20000\n    selection_method = \"minimize_meter_distance\"\n    allow_duplicate_matches = False\n    n_matches_per_treatment = 1\n    n_treatments_per_chunk = 5000\n\n    treatment_group = generate_group(n_treatment, make_random=True)\n    comparison_pool = generate_group(n_pool, make_random=True, id_prefix=\"c\")\n    settings = Settings(\n        selection_method=selection_method,\n        n_treatments_per_chunk=n_treatments_per_chunk,\n        n_matches_per_treatment=n_matches_per_treatment,\n        allow_duplicate_matches=allow_duplicate_matches,\n    )\n    IMM = DistanceMatching(\n        settings=settings\n    )\n    comparison_group = IMM.get_comparison_group(\n        treatment_group=treatment_group,\n        comparison_pool=comparison_pool\n    )\n    assert not comparison_group.empty\n\n\ndef test_distance_duplicate_best_match():\n    n_treatment = 2\n    n_pool = 2\n    selection_method = \"minimize_meter_distance\"\n    allow_duplicate_matches = False\n    n_matches_per_treatment = 1\n\n    t_ids = [\"far\", \"close\"]\n    t_vals = [10, 1]\n    c_ids = [\"match_1\", \"match_2\"]\n    c_vals = [2, 100]\n    treatment_group = pd.DataFrame({\"id\": t_ids, \"month_1\": t_vals}).set_index(\"id\")\n    comparison_pool = pd.DataFrame({\"id\": c_ids, \"month_1\": c_vals}).set_index(\"id\")\n\n    settings = Settings(\n        selection_method=selection_method,\n        n_matches_per_treatment=n_matches_per_treatment,\n        allow_duplicate_matches=allow_duplicate_matches,\n    )\n    IMM = DistanceMatching(\n        settings=settings\n    )\n    comparison_group = IMM.get_comparison_group(\n        treatment_group=treatment_group,\n        comparison_pool=comparison_pool\n    )\n    comparison_group.set_index(\"id\", inplace=True)\n\n    assert comparison_group.loc[\"match_1\", \"treatment\"] == \"close\"\n    assert comparison_group.loc[\"match_2\", \"treatment\"] == \"far\"\n\n\ndef test_multiple_meter_matches():\n    random.seed(1)\n\n    n_treatment = 8\n    n_pool = 2000\n    selection_method = \"minimize_meter_distance\"\n    allow_duplicate_matches = False\n    n_matches_per_treatment = 5\n\n    # this will run through the 'duplicates' loop several times before finding unique values\n    # however since here are more 'max runs allowed' than treatment meters, it will be\n    # able to iterate enough times to find unique matches\n\n    treatment_group = generate_group(n_treatment, make_random=True)\n    comparison_pool = generate_group(n_pool, make_random=True)\n    settings = Settings(\n        selection_method=selection_method,\n        n_matches_per_treatment=n_matches_per_treatment,\n        allow_duplicate_matches=allow_duplicate_matches,\n    )\n    IMM = DistanceMatching(\n        settings=settings\n    )\n    comparison_group = IMM.get_comparison_group(\n        treatment_group=treatment_group,\n        comparison_pool=comparison_pool\n    )\n    assert not comparison_group[\"duplicated\"].any()\n    assert len(comparison_group) == 40\n    assert comparison_group.index.nunique() == 40\n    assert comparison_group.treatment.value_counts().nunique() == 1\n"
  },
  {
    "path": "tests/comparison_groups/stratified_sampling/test_bin.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pandas as pd\n\nfrom opendsm.comparison_groups.stratified_sampling.bins import Bin, MultiBin, BinnedData, Binning\n\n\n\ndef test_bin_filtering():\n    this_bin = Bin(\"col\", min=5, max=100, index=0)\n    filter_expr = this_bin.filter_expr()\n\n    df = pd.DataFrame({\"col\": [1, 5, 6, 100, 101]})\n    df = df[filter_expr(df)]\n\n    assert set(df[\"col\"]) == set([5, 6, 100])\n\n\ndef test_binned_data_bin_label_label_leading_zeroes():\n    col_name = 'c1'\n    b1 = Bin(col_name, min=1, max=2, index=0)\n\n    multi_bin = MultiBin(bins=[b1])\n    df = pd.DataFrame({col_name: [1.5]})\n\n    binning = Binning()\n    binning.multibins = [multi_bin]\n\n    binned_data = BinnedData(df, binning)\n    mapped_bins = binned_data._map_bins(df)\n    assert set(mapped_bins['_bin_label'].values) == set(['c1_000'])\n\n\n\n'''\n\ndef test_multi_bin_filtering():\n    b1 = Bin(\"c1\", min=5, max=100, index=0)\n    b2 = Bin(\"c2\", min=50, max=500, index=1)\n\n    mb = MultiBin(bins=[b1, b2])\n\n    df = pd.DataFrame(\n        [\n            {\"c1\": 1, \"c2\": 1, \"in\": False},\n            {\"c1\": 1, \"c2\": 100, \"in\": False},\n            {\"c1\": 10, \"c2\": 1, \"in\": False},\n            {\"c1\": 10, \"c2\": 100, \"in\": True},\n            {\"c1\": 10, \"c2\": 1000, \"in\": False},\n        ]\n    )\n    filter_expr = mb.filter_expr()\n    df = df[filter_expr(df)]\n    assert len(df) == 1\n    assert df[\"in\"].iloc[0] == True\n\n\n\ndef test_remove_bins_too_small():\n\n    bins = [\n    Bin(\"c1\", min=0, max=10, index=0),\n    Bin(\"c1\", min=10, max=20, index=1),\n    Bin(\"c1\", min=20, max=30, index=2),\n    Bin(\"c2\", min=100, max=110, index=0),\n    Bin(\"c2\", min=110, max=120, index=1),\n    Bin(\"c2\", min=120, max=130, index=2),\n    Bin(\"c2\", min=130, max=140, index=3),\n    ]\n\n    mb = MultiBin(bins=bins)\n'''\n"
  },
  {
    "path": "tests/comparison_groups/stratified_sampling/test_bin_selection.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nfrom opendsm.comparison_groups.stratified_sampling.model import StratifiedSampling, BinnedData\nfrom opendsm.comparison_groups.stratified_sampling.bin_selection import StratifiedSamplingBinSelector\nfrom opendsm.comparison_groups.stratified_sampling.bins import ModelSamplingException\n\n\n\ndef test_stratified_sampling_fit_and_sample_records_equivalence(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    stratified_sampling_obj = StratifiedSampling()\n    df_pool[\"col2\"] = df_pool[col_name]\n    df_treatment[\"col2\"] = df_treatment[col_name]\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    ## attempting to estimate both n_bins and n_samples\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        \n        min_n_bins=4,\n        max_n_bins=6,\n        random_seed=1,\n        equivalence_method='chisquare',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n\n\ndef test_stratified_sampling_fit_and_sample_records_equivalence_too_many_bins(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    stratified_sampling_obj = StratifiedSampling()\n\n    stratified_sampling_obj.add_column(col_name)\n    ## attempting to estimate both n_bins and n_samples\n    with pytest.raises(ModelSamplingException):\n        model_w_selected_bins = StratifiedSamplingBinSelector(stratified_sampling_obj,\n            df_treatment,\n            df_pool,\n\n            min_n_bins=1000,\n            max_n_bins=1002,\n            random_seed=1,\n            equivalence_method='chisquare',\n            relax_n_samples_approx_constraint=False,\n            equivalence_feature_ids = equivalence_feature_ids,\n            equivalence_feature_matrix = equivalence_feature_matrix\n        )\n\n\ndef test_stratified_sampling_fit_and_sample_records_equivalence_idempotent_check(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    df_treatment[\"col2\"] = df_treatment[col_name] * 2\n    df_treatment[\"col3\"] = df_treatment[col_name] * 3\n\n    df_pool[\"col2\"] = df_pool[col_name] * 2\n    df_pool[\"col3\"] = df_pool[col_name] * 3\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='chisquare',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    sample1 = stratified_sampling_obj.data_sample.df.index.values\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='chisquare',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    sample2 = stratified_sampling_obj.data_sample.df.index.values\n    assert set(sample1) == set(sample2)\n\n\ndef test_stratified_sampling_fit_and_sample_records_equivalence_euclidean_idempotent_check(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    df_treatment[\"col2\"] = df_treatment[col_name] * 2\n    df_treatment[\"col3\"] = df_treatment[col_name] * 3\n\n    df_pool[\"col2\"] = df_pool[col_name] * 2\n    df_pool[\"col3\"] = df_pool[col_name] * 3\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        \n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='euclidean',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    sample1 = stratified_sampling_obj.data_sample.df.index.values\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='euclidean',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    sample2 = stratified_sampling_obj.data_sample.df.index.values\n    assert set(sample1) == set(sample2)\n\n\ndef test_stratified_sampling_fit_and_sample_records_equivalence_euclidean_idempotent_check(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    df_treatment[\"col2\"] = df_treatment[col_name] * 2\n    df_treatment[\"col3\"] = df_treatment[col_name] * 3\n\n    df_pool[\"col2\"] = df_pool[col_name] * 2\n    df_pool[\"col3\"] = df_pool[col_name] * 3\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='euclidean',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    sample1 = stratified_sampling_obj.data_sample.df.index.values\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n    StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='euclidean',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix        \n    )\n    sample2 = stratified_sampling_obj.data_sample.df.index.values\n    assert set(sample1) == set(sample2)\n\n\ndef test_plot_records_based_equiv_average(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    df_treatment[\"col2\"] = df_treatment[col_name] * 2\n    df_treatment[\"col3\"] = df_treatment[col_name] * 3\n\n    df_pool[\"col2\"] = df_pool[col_name] * 2\n    df_pool[\"col3\"] = df_pool[col_name] * 3\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n\n    bin_selection = StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='euclidean',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    bin_selection.plot_records_based_equiv_average(plot=False)\n    bin_selection.results_as_json()\n\n\ndef test_plot_records_based_equiv_average_chisquare(\n    df_treatment, df_pool,  col_name, equivalence_feature_ids, equivalence_feature_matrix\n):\n    df_treatment[\"col2\"] = df_treatment[col_name] * 2\n    df_treatment[\"col3\"] = df_treatment[col_name] * 3\n\n    df_pool[\"col2\"] = df_pool[col_name] * 2\n    df_pool[\"col3\"] = df_pool[col_name] * 3\n\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n\n    bin_selection = StratifiedSamplingBinSelector(stratified_sampling_obj,\n        df_treatment,\n        df_pool,\n        min_n_bins=2,\n        max_n_bins=3,\n        random_seed=1,\n        equivalence_method='chisquare',\n        equivalence_feature_ids = equivalence_feature_ids,\n        equivalence_feature_matrix = equivalence_feature_matrix\n    )\n    bin_selection.plot_records_based_equiv_average(plot=False)\n    results = bin_selection.results_as_json()\n    assert 'bins_selected_str' in list(results['n_bin_results'][0].keys())\n"
  },
  {
    "path": "tests/comparison_groups/stratified_sampling/test_diagnostics.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nimport pandas as pd\n\nfrom opendsm.comparison_groups.stratified_sampling.model import StratifiedSampling\n\n\n@pytest.fixture\ndef diagnostics_obj(df_treatment, df_pool, col_name):\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name, n_bins=4)\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment, df_pool, n_samples_approx=len(df_treatment), random_seed=1\n    )\n    return stratified_sampling_obj.diagnostics()\n\n\ndef test_equivalence(diagnostics_obj):\n    equivalence = diagnostics_obj.equivalence()\n    assert equivalence[\"ks_ok\"].all() == True and equivalence[\"t_ok\"].all() == True\n"
  },
  {
    "path": "tests/comparison_groups/stratified_sampling/test_equivalence.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.stratified_sampling.equivalence import *\n\n\n\n@pytest.fixture\ndef equiv_X():\n    # 3 columns, 6 rows, column first \n    return np.array(\n            [[1,1,1,10,10,10],\n            [1,1,1,10,10,10],\n            [1,1,1,10,10,10]])\n\n\n@pytest.fixture\ndef equiv_Y():\n    # 3 columns, 6 rows, column first \n    return np.array(\n            [[0,1,2,3,4,5],\n            [0,0,0,0,0,0],\n            [0,0,0,0,0,0],\n            ])\n\n\n@pytest.fixture\ndef feature_matrix(equiv_X, equiv_Y):\n    x = equiv_X\n    y = equiv_Y\n    return np.concatenate([x.transpose(), y.transpose()])\n\n\n\ndef test_reshape_outputs():\n    means = [[11,12], [21,22]]\n\n    quantiles = [[11, 12, 13], \n                 [21, 22, 23]]\n\n    df = pd.DataFrame({'_bin_label': ['[11, 12]', '[12, 13]', '[21, 22]', '[22, 23]'],\n                  'value': [11, 12, 21, 22], 'feature_index': [0,0,1,1]})\n    assert df.equals(reshape_outputs(means, quantiles))\n\n\ndef test_equivalene_distance(feature_matrix):\n\n    eq = Equivalence(ix_x = [0,1,2,3,4,5], ix_y = [6,7,8,9,10,11], \n        features_matrix = feature_matrix, n_quantiles=2, how=\"euclidean\")\n    equiv_x, equiv_y, distance = eq.compute()\n    assert round(distance,2) == round(6 + 2*101**0.5,2)\n\n\n    eq = Equivalence(ix_x = [0,1,2,3,4,5], ix_y = [6,7,8,9,10,11], \n        features_matrix = feature_matrix, n_quantiles=2, how=\"chisquare\")\n    equiv_x, equiv_y, distance = eq.compute()\n    assert round(distance,2) == round(36/14 + 2*11,2)\n\n\n\n\ndef test_get_quantiles():\n    assert (get_quantile_indexes(1) == [0, 1]).all()\n    assert (get_quantile_indexes(2) == [0, 0.5, 1]).all()\n    assert (get_quantile_indexes(3) == [0, 1/3, 2/3, 1]).all()\n\n\n\n\ndef test_quantile_means_array(equiv_X, equiv_Y):\n    x = [0,1,2,3,4,5]\n    means, quantiles = quantile_means_array(x, n_quantiles=1)\n    assert (means == np.mean(x)).all()\n    means, quantiles = quantile_means_array(x, n_quantiles=len(x))\n    assert (means == x).all()\n\n    x = [1,1,1,10,10,10]\n    means, quantiles = quantile_means_array(x, n_quantiles=2)\n    assert (means == [1, 10]).all()\n\n    means_x, means_y, quantiles_x, quantiles_y = quantile_means_population(equiv_X, equiv_Y, 2)\n    assert (means_x == np.array([[1,10], [1,10], [1,10]])).all()\n    assert (means_y == np.array([[1,4], [0,0], [0,0]])).all()\n\n\ndef test_quantile_distance(equiv_X, equiv_Y):\n    means_x, means_y, quantiles_x, quantiles_y = quantile_means_population(equiv_X, equiv_Y, 2)\n    assert round(sum_column_distance(means_x, means_y, \"euclidean\"),2) == round(6 + 2*101**0.5,2)\n    assert round(sum_column_distance(means_x, means_y, \"chisquare\"),2) == round(36/14 + 2*11,2)\n\n\ndef test_equivalence_inputs(feature_matrix):\n\n    eq = Equivalence(ix_x = [1,2], ix_y = [3, 10, 11], features_matrix = feature_matrix)    \n    # X should be [1,1,1] and [1,1,1]\n    # eq.X should be sliced by column so [1,1], [1,1], [1,1]\n    # Y should be [10, 10, 10], [4, 0, 0], [5, 0, 0]\n    # eq.Y should be [10, 4, 5], [10, 0, 0], [10, 0 0]\n    assert (eq.X == np.array([[1,1], [1,1], [1,1]])).all()\n    assert (eq.Y == np.array([[10,4,5], [10,0,0], [10,0,0]])).all()\n\n\n\n\n\ndef test_index_to_ids():\n    t = np.array([11,17,15])\n    a = np.array([11,12,13,15,16,17])\n    assert (np.array(ids_to_index(t,a) == np.array([0,5,3]))).all()\n\n    t1 = np.array([\"11\",\"17\",\"15\"])\n    a1 = np.array([\"11\",\"12\",\"13\",\"15\",\"16\",\"17\"])\n    assert (np.array(ids_to_index(t1,a1) == np.array([0,5,3]))).all()\n\n    with pytest.raises(ValueError):\n        t2 = np.array([98123])\n        ids_to_index(t2,a)\n\n\n\n\n"
  },
  {
    "path": "tests/comparison_groups/stratified_sampling/test_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.comparison_groups.stratified_sampling.model import StratifiedSampling, BinnedData\nfrom opendsm.comparison_groups.stratified_sampling.bins import ModelSamplingException\n\n\ndef test_stratified_sampling_fit_and_sample():\n    stratified_sampling_obj = StratifiedSampling()\n    df_treatment = pd.DataFrame([{\"id\": f\"id_{x}\", \"col1\": x} for x in range(0, 10)])\n    df_pool = pd.DataFrame([{\"id\": f\"id_{x}\", \"col1\": x / 2.0} for x in range(0, 1000)])\n    stratified_sampling_obj.add_column(\"col1\")\n\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=10,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    sample1 = stratified_sampling_obj.data_sample.df.index.values\n\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=10,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    sample2 = stratified_sampling_obj.data_sample.df.index.values\n    assert set(sample1) == set(sample2)\n\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=10,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    sample1 = stratified_sampling_obj.data_sample.df.index.values\n\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=10,\n        random_seed=5,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    sample2 = stratified_sampling_obj.data_sample.df.index.values\n    assert set(sample1) != set(sample2)\n\n\ndef test_stratified_sampling_fit_and_sample_random_seed_check():\n    # perturb was returning different values since it was writing over the\n    # df rather than using a copy\n    df_comparison = pd.DataFrame(\n        [\n            {\n                \"id\": f\"id-{x}\",\n                \"baseline_annual_kwh\": np.random.random() * 10000,\n                \"baseline_bd_pct_heating_load\": np.random.random(),\n            }\n            for x in range(0, 200000)\n        ]\n    )\n    df_treatment = pd.DataFrame(\n        [\n            {\n                \"id\": f\"id-{x}\",\n                \"baseline_annual_kwh\": np.random.random() * 10000,\n                \"baseline_bd_pct_heating_load\": np.random.random(),\n            }\n            for x in range(0, 500)\n        ]\n    )\n    n_samples_approx = 500\n    random_seed = 1\n    stratification_params = [\"baseline_annual_kwh\", \"baseline_bd_pct_heating_load\"]\n\n    model = StratifiedSampling(\n        treatment_label=\"treatment\", pool_label=\"comparison\", output_name=\"control\"\n    )\n    [model.add_column(col) for col in stratification_params]\n\n    model.fit(df_treatment, min_n_treatment_per_bin=0)\n    model.sample(\n        df_comparison, n_samples_approx=n_samples_approx, random_seed=random_seed\n    )\n\n    for run_num in range(0, 10):\n        model_temp = StratifiedSampling(\n            treatment_label=\"treatment\", pool_label=\"comparison\", output_name=\"control\"\n        )\n        [model_temp.add_column(col) for col in stratification_params]\n        model_temp.fit(df_treatment, min_n_treatment_per_bin=0)\n        model_temp.sample(\n            df_comparison, n_samples_approx=n_samples_approx, random_seed=random_seed\n        )\n        pd.testing.assert_frame_equal(\n            model_temp.data_sample.df[stratification_params + [\"id\"]],\n            model.data_sample.df[stratification_params + [\"id\"]],\n        )\n        assert (\n            len(\n                set(model_temp.data_sample.df[\"id\"].values)\n                - set(model.data_sample.df[\"id\"].values)\n            )\n            == 0\n        )\n\n\n@pytest.fixture\ndef stratified_sampling_obj():\n    return StratifiedSampling()\n\n\ndef test_stratified_sampling_fit_and_sample_min_allowed_max_allowed(\n    stratified_sampling_obj\n):\n    col_name = \"col1\"\n    min_value_allowed = 5\n    max_value_allowed = 8\n    df_treatment = pd.DataFrame([{\"id\": f\"id_{x}\", col_name: x} for x in range(0, 10)])\n    df_pool = pd.DataFrame(\n        [{\"id\": f\"id_{x}\", col_name: x} for x in np.arange(0, 20, 0.1)]\n    )\n    stratified_sampling_obj.add_column(\n        col_name,\n        min_value_allowed=min_value_allowed,\n        max_value_allowed=max_value_allowed,\n    )\n\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=4,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    output = stratified_sampling_obj.data_sample.df[col_name].values\n    assert min(output) > min_value_allowed\n    assert max(output) < max_value_allowed\n\n\ndef test_stratified_sampling_fit_and_sample_n_samples_approx_limit(\n    df_treatment, df_pool, col_name\n):\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n\n    n_samples_approx = 40\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment, df_pool, n_samples_approx=n_samples_approx, random_seed=1\n    )\n    output = stratified_sampling_obj.data_sample.df\n    assert output[\"_bin_label\"].nunique() == 2\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert (bins_df[\"n_sampled\"] / bins_df[\"n_pct_sampled\"]).round() == n_samples_approx\n\n\ndef test_stratified_sampling_fit_and_sample_n_samples_approx_limit(\n    df_treatment, df_pool, col_name\n):\n    stratified_sampling_obj = StratifiedSampling()\n    col_name = \"col1\"\n    df_treatment = pd.DataFrame(\n        [\n            {\"id\": f\"id_{x}\", col_name: x}\n            for x in (\n                list(np.arange(0, 2, 0.1))\n                + list(np.arange(2, 4, 0.5))\n                + list(np.arange(4, 6, 1))\n                + list(np.arange(6, 10, 0.2))\n            )\n        ]\n    )\n    df_pool = pd.DataFrame(\n        [{\"id\": f\"id_{x}\", col_name: x} for x in np.arange(0, 20, 0.01)]\n    )\n    stratified_sampling_obj.add_column(col_name)\n\n    n_samples_approx = 40\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=n_samples_approx,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    output = stratified_sampling_obj.data_sample.df\n    assert output[\"_bin_label\"].nunique() == 2\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert abs(len(output) - n_samples_approx) <= 1\n\n\ndef test_stratified_sampling_fit_and_sample_n_samples_approx_variations(\n    df_treatment, df_pool, col_name\n):\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    ## attempting to estimate both n_bins and n_samples\n    stratified_sampling_obj.fit_and_sample(df_treatment, df_pool, random_seed=1)\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert len(bins_df) == 3\n\n    ## enforcing 1 bin\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name, n_bins=1)\n    stratified_sampling_obj.fit_and_sample(df_treatment, df_pool, random_seed=1)\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n\n    ## enforcing 4 bins\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name, n_bins=4)\n    stratified_sampling_obj.fit_and_sample(df_treatment, df_pool, random_seed=1)\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert len(bins_df) == 4\n\n    ## enforcing n_samples_approx=40\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        n_samples_approx=40,\n        random_seed=1,\n        min_n_sampled_to_n_treatment_ratio=None,\n    )\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    # should be within 1 of n_samples_approx\n    assert abs(len(output) - 40) <= 1\n\n\ndef test_stratified_sampling_fit_and_sample_too_many_bins(df_treatment, df_pool, col_name):\n    df_treatment[\"col2\"] = df_treatment[col_name].astype(int)\n    df_pool[\"col2\"] = df_pool[col_name].astype(int)\n    df_treatment[\"col3\"] = df_treatment[col_name].astype(int) * 2\n    df_pool[\"col3\"] = df_pool[col_name].astype(int) / 2\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\")\n    ## attempting to estimate both n_bins and n_samples\n    with pytest.raises(ValueError):\n        stratified_sampling_obj.fit_and_sample(df_treatment, df_pool, random_seed=1)\n\n\ndef test_stratified_sampling_fit_and_sample_dont_require_equivalence(\n    df_treatment, df_pool, col_name\n):\n    df_treatment[\"col2\"] = df_treatment[col_name].astype(int)\n    df_pool[\"col2\"] = df_pool[col_name].astype(int)\n    df_treatment[\"col3\"] = df_treatment[col_name].astype(int) * 2\n    df_pool[\"col3\"] = df_pool[col_name].astype(int) / 2\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    stratified_sampling_obj.add_column(\"col2\")\n    stratified_sampling_obj.add_column(\"col3\", auto_bin_require_equivalence=False)\n    ## attempting to estimate both n_bins and n_samples\n    stratified_sampling_obj.fit_and_sample(df_treatment, df_pool, random_seed=1)\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert not output.empty\n\n\ndef test_stratified_sampling_fit_and_sample_upper_limit_n_samples_approx(\n    df_treatment, df_pool, col_name\n):\n    stratified_sampling_obj = StratifiedSampling()\n    stratified_sampling_obj.add_column(col_name)\n    ## attempting to estimate both n_bins and n_samples\n    with pytest.raises(ModelSamplingException):\n        stratified_sampling_obj.fit_and_sample(\n            df_treatment, df_pool, random_seed=1, n_samples_approx=1000\n        )\n    stratified_sampling_obj.fit_and_sample(\n        df_treatment,\n        df_pool,\n        random_seed=1,\n        n_samples_approx=1000,\n        relax_n_samples_approx_constraint=True,\n    )\n    output = stratified_sampling_obj.data_sample.df\n    bins_df = stratified_sampling_obj.diagnostics().count_bins()\n    assert not output.empty\n"
  },
  {
    "path": "tests/conftest.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\nimport importlib.resources\n\nimport pytest\n\nfrom opendsm.eemeter.samples import load_sample\n\n\n@pytest.fixture\ndef sample_metadata():\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        \"metadata.json\"\n    ).open(\"rb\") as f:\n        metadata = json.loads(f.read().decode(\"utf-8\"))\n    return metadata\n\n\ndef _from_sample(sample, tempF=True):\n    meter_data, temperature_data, metadata = load_sample(sample, tempF=tempF)\n    return {\n        \"meter_data\": meter_data,\n        \"temperature_data\": temperature_data,\n        \"blackout_start_date\": metadata[\"blackout_start_date\"],\n        \"blackout_end_date\": metadata[\"blackout_end_date\"],\n    }\n\n\n@pytest.fixture\ndef il_electricity_cdd_hdd_hourly():\n    return _from_sample(\"il-electricity-cdd-hdd-hourly\")\n\n\n@pytest.fixture\ndef il_electricity_cdd_hdd_daily():\n    return _from_sample(\"il-electricity-cdd-hdd-daily\")\n\n\n@pytest.fixture\ndef il_electricity_cdd_hdd_billing_monthly():\n    return _from_sample(\"il-electricity-cdd-hdd-billing_monthly\")\n\n\n@pytest.fixture\ndef il_electricity_cdd_hdd_billing_bimonthly():\n    return _from_sample(\"il-electricity-cdd-hdd-billing_bimonthly\")\n\n\n@pytest.fixture\ndef il_gas_hdd_only_hourly():\n    return _from_sample(\"il-gas-hdd-only-hourly\")\n\n\n@pytest.fixture\ndef uk_electricity_hdd_only_hourly_sample_1():\n    return _from_sample(\"uk-electricity-hdd-only-hourly-sample-1\", tempF=False)\n\n\n@pytest.fixture\ndef uk_electricity_hdd_only_hourly_sample_2():\n    return _from_sample(\"uk-electricity-hdd-only-hourly-sample-2\", tempF=False)\n"
  },
  {
    "path": "tests/eemeter/daily_model/base_models/test_c_hdd_tidd_smooth.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients\nfrom opendsm.eemeter.models.daily.parameters import ModelType\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings as Settings\nfrom opendsm.eemeter.models.daily.base_models.c_hdd_tidd import fit_c_hdd_tidd\nfrom opendsm.eemeter.models.daily.fit_base_models import _get_opt_settings\n\n\ndef test_fit_c_hdd_tidd_smooth():\n    # Test case 1: Test with initial_fit=True\n    T = np.array([10, 20, 30, 40, 50, 60, 70, 80, 90, 100]).astype(float)\n    obs = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).astype(float)\n    settings = Settings(\n        developer_mode=True,\n        alpha_selection=0.1,\n        alpha_final=0.2,\n        segment_minimum_count=5,\n        maximum_slope_OoM_scaler=1,\n    )\n    opt_options = _get_opt_settings(settings)\n    x0 = ModelCoefficients(\n        model_type=ModelType.HDD_TIDD_SMOOTH,\n        intercept=0.0,\n        hdd_bp=0.0,\n        hdd_beta=0.0,\n        hdd_k=0.0,\n        cdd_bp=None,\n        cdd_beta=None,\n        cdd_k=None,\n    )\n    bnds = None\n    weights = None\n    initial_fit = True\n    smooth = True\n    res = fit_c_hdd_tidd(T, obs, weights, settings, opt_options, smooth, x0, bnds, initial_fit)\n    assert res.success == True\n\n    # Test case 2: Test with initial_fit=False\n    T = np.array([10, 20, 30, 40, 50, 60, 70, 80, 90, 100]).astype(float)\n    obs = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).astype(float)\n    settings = Settings(\n        developer_mode=True,\n        alpha_selection=0.1,\n        alpha_final=0.2,\n        segment_minimum_count=5,\n        maximum_slope_OoM_scaler=1,\n    )\n    opt_options = _get_opt_settings(settings)\n    x0 = ModelCoefficients(\n        model_type=ModelType.HDD_TIDD_SMOOTH,\n        intercept=0.0,\n        hdd_bp=0.0,\n        hdd_beta=0.0,\n        hdd_k=0.0,\n        cdd_bp=None,\n        cdd_beta=None,\n        cdd_k=None,\n    )\n    bnds = None\n    initial_fit = False\n    smooth = True\n    res = fit_c_hdd_tidd(T, obs, weights, settings, opt_options, smooth, x0, bnds, initial_fit)\n    assert res.success == True\n\n    # Test case 3: Test with x0=None\n    T = np.array([10, 20, 30, 40, 50, 60, 70, 80, 90, 100]).astype(float)\n    obs = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).astype(float)\n    settings = Settings(\n        developer_mode=True,\n        alpha_selection=0.1,\n        alpha_final=0.2,\n        segment_minimum_count=5,\n        maximum_slope_OoM_scaler=1,\n    )\n    opt_options = _get_opt_settings(settings)\n    x0 = None\n    bnds = None\n    initial_fit = True\n    smooth = True\n    res = fit_c_hdd_tidd(T, obs, weights, settings, opt_options, smooth, x0, bnds, initial_fit)\n    assert res.success == True"
  },
  {
    "path": "tests/eemeter/daily_model/base_models/test_full_model_finder.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nfrom opendsm.eemeter.models.daily.base_models.full_model import full_model\n\n\ndef test_full_model_import():\n    hdd_bp = 50\n    hdd_beta = 0.01\n    hdd_k = 0.001\n    cdd_bp = 80\n    cdd_beta = 0.02\n    cdd_k = 0.002\n    intercept = 100\n    T_fit_bnds = np.array([10, 100]).astype(np.double)\n    T = np.linspace(10, 100, 130).astype(np.double)\n\n    res = full_model(\n        hdd_bp, hdd_beta, hdd_k, cdd_bp, cdd_beta, cdd_k, intercept, T_fit_bnds, T\n    )\n    assert res.size == T.size\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_billing_data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nfrom opendsm.eemeter.models.billing.data import (\n    BillingBaselineData,\n    BillingReportingData,\n)\nfrom opendsm.eemeter.samples import load_sample\nimport numpy as np\nimport pandas as pd\nfrom pandas import Timestamp, DatetimeIndex, DataFrame\nimport pytest\n\nTEMPERATURE_SEED = 29\nMETER_SEED = 41\nNUM_DAYS_IN_YEAR = 365\n\n\n@pytest.fixture\ndef get_datetime_index(request):\n    # Request = [frequency , is_timezone_aware]\n\n    # Create a DateTimeIndex at given frequency and timezone if requested\n    inclusive = \"both\" if request.param[0] in [\"MS\", \"2MS\"] else \"left\"\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=inclusive,\n        freq=request.param[0],\n        tz=\"US/Eastern\" if request.param[1] else None,\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_half_hourly_with_timezone():\n    # Create a DateTimeIndex at 30-minute intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"30min\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_hourly_with_timezone():\n    # Create a DateTimeIndex at 30-minute intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"h\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_daily_with_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"D\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_monthly_with_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"both\",\n        freq=\"MS\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_bimonthly_with_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"both\",\n        freq=\"2MS\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_daily_without_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\", end=\"2024-01-01\", inclusive=\"left\", freq=\"D\"\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_temperature_data_half_hourly(get_datetime_index_half_hourly_with_timezone):\n    datetime_index = get_datetime_index_half_hourly_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_temperature_data_hourly(get_datetime_index_hourly_with_timezone):\n    datetime_index = get_datetime_index_hourly_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_temperature_data_daily(get_datetime_index_daily_with_timezone):\n    datetime_index = get_datetime_index_daily_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_meter_data_daily(get_datetime_index_daily_with_timezone):\n    datetime_index = get_datetime_index_daily_with_timezone\n\n    np.random.seed(METER_SEED)\n    # Create a 'meter_value' column with random data\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"observed\": meter_value}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_meter_data_monthly(get_datetime_index_monthly_with_timezone):\n    datetime_index = get_datetime_index_monthly_with_timezone\n\n    np.random.seed(METER_SEED)\n    # Create a 'meter_value' column with random data\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"observed\": meter_value}, index=datetime_index)\n    df.iloc[-1, df.columns.get_loc(\"observed\")] = np.nan\n\n    return df\n\n\n@pytest.fixture\ndef get_meter_data_bimonthly(get_datetime_index_bimonthly_with_timezone):\n    datetime_index = get_datetime_index_bimonthly_with_timezone\n\n    np.random.seed(METER_SEED)\n    # Create a 'meter_value' column with random data\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"observed\": meter_value}, index=datetime_index)\n    df.iloc[-1, df.columns.get_loc(\"observed\")] = np.nan\n\n    return df\n\n\n# Check that a missing timezone raises a Value Error\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", False]], indirect=True)\ndef test_billing_baseline_data_with_missing_timezone(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"meter\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n\n    with pytest.raises(ValueError):\n        cls = BillingBaselineData(df, is_electricity_data=True)\n\n\n# Check that a missing datetime index and column raises a Value Error\ndef test_billing_baseline_data_with_missing_datetime_index_and_column():\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(NUM_DAYS_IN_YEAR)\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(NUM_DAYS_IN_YEAR)\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"meter\": meter_value, \"temperature\": temperature_mean})\n\n    with pytest.raises(ValueError):\n        cls = BillingBaselineData(df, is_electricity_data=True)\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"MS\", True]], indirect=True)\ndef test_billing_baseline_data_with_monthly_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n    meter_value[-1] = np.nan\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"observed\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n    df.index = df.index[:-1].union([df.index[-1] - pd.Timedelta(days=1)])\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    # DQ because only 12 days worth of temperature data is available\n    assert len(cls.disqualification) == 2\n    assert set([dq.qualified_name for dq in cls.disqualification]) == set([\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n    ])\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"2MS\", True]], indirect=True)\ndef test_billing_baseline_data_with_bimonthly_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"observed\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n    df.index = df.index[:-1].union([df.index[-1] - pd.Timedelta(days=1)])\n    df.iloc[-1, df.columns.get_loc(\"observed\")] = np.nan\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    # Because two months are missing\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    # DQ because only 6 days worth of temperature data is available\n    assert len(cls.disqualification) == 2\n    assert set([dq.qualified_name for dq in cls.disqualification]) == set(\n        [\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        ]\n    )\n\n\ndef test_billing_baseline_data_with_monthly_hourly_frequencies(\n    get_meter_data_monthly, get_temperature_data_hourly\n):\n    # Create a DataFrame with uneven frequency\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_monthly\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n    df = df[:-1]  # when using dataframe input, rows are exact length\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\ndef test_billing_baseline_data_with_bimonthly_hourly_frequencies(\n    get_meter_data_bimonthly, get_temperature_data_hourly\n):\n    # Create a DataFrame with uneven frequency\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_bimonthly\n\n    # Merge 'df' and 'df_meter' in a left join, as df input should not have trailing nan\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"left\")\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\ndef test_billing_baseline_data_with_monthly_daily_frequencies(\n    get_meter_data_monthly, get_temperature_data_daily\n):\n    # Create a DataFrame with uneven frequency\n    df = get_temperature_data_daily\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_monthly\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n    df = df[:-1]  # when using dataframe input, rows are exact length\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 0\n\n\ndef test_billing_baseline_data_with_bimonthly_daily_frequencies(\n    get_meter_data_bimonthly, get_temperature_data_daily\n):\n    # Create a DataFrame with uneven frequency\n    df = get_temperature_data_daily\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_bimonthly\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n    df = df[:-1]  # when using dataframe input, rows are exact length\n\n    cls = BillingBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    # assert round(cls.df.observed.sum(), 2) == round(df.observed[:-1].sum(), 2)\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 0\n\n\ndef test_billing_baseline_data_with_specific_hourly_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-hourly\")\n    # Take the extra month for billing data\n    meter = meter[\n        (meter.index.year == 2017)\n        | ((meter.index.year == 2018) & (meter.index.month == 1))\n    ]\n    temperature = temperature[\n        (temperature.index.year == 2017)\n        | ((temperature.index.year == 2018) & (temperature.index.month == 1))\n    ]\n\n    cls = BillingBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert (\n        len(cls.df) == (meter.index[-1] - meter.index[0]).days + 1\n    )  # hourly series does not have trailing nan\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.inferior_model_usage\",\n    ]\n    assert len(cls.disqualification) == 1\n    assert (\n        cls.disqualification[0].qualified_name\n        == \"eemeter.sufficiency_criteria.incorrect_number_of_total_days\"\n    )\n\n\ndef test_billing_baseline_data_with_specific_daily_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-daily\")\n    # Take the extra month for billing data\n    meter = meter[\n        (meter.index.year == 2017)\n        | ((meter.index.year == 2018) & (meter.index.month == 1))\n    ]\n    temperature = temperature[\n        (temperature.index.year == 2017)\n        | ((temperature.index.year == 2018) & (temperature.index.month == 1))\n    ]\n    cls = BillingBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert (\n        len(cls.df) == (meter.index[-1] - meter.index[0]).days + 1\n    )  # daily series does not have trailing nan\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.inferior_model_usage\",\n    ]\n    assert len(cls.disqualification) == 1\n    assert (\n        cls.disqualification[0].qualified_name\n        == \"eemeter.sufficiency_criteria.incorrect_number_of_total_days\"\n    )\n\n\ndef test_billing_baseline_data_with_specific_missing_daily_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-daily\")\n    # Take the extra month for billing data\n    meter = meter[\n        (meter.index.year == 2017)\n        | ((meter.index.year == 2018) & (meter.index.month == 1))\n    ]\n    temperature = temperature[\n        (temperature.index.year == 2017)\n        | ((temperature.index.year == 2018) & (temperature.index.month == 1))\n    ]\n    # Set 1 month meter data to NaN\n    meter.loc[meter.index.month == 4] = np.nan\n\n    cls = BillingBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert (\n        len(cls.df) == (meter.index[-1] - meter.index[0]).days + 1\n    )  # daily series does not have trailing nan\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.inferior_model_usage\",\n    ]\n    assert len(cls.disqualification) == 1\n    assert (\n        cls.disqualification[0].qualified_name\n        == \"eemeter.sufficiency_criteria.incorrect_number_of_total_days\"\n    )\n\n\ndef test_billing_baseline_data_with_specific_monthly_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-billing_monthly\")\n    # Take the extra month for billing data\n    meter = meter[\n        (meter.index.year == 2017)\n        | ((meter.index.year == 2018) & (meter.index.month == 1))\n    ]\n    temperature = temperature[\n        (temperature.index.year == 2017)\n        | ((temperature.index.year == 2018) & (temperature.index.month == 1))\n    ]\n    cls = BillingBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == (meter.index[-1] - meter.index[0]).days\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert set([warning.qualified_name for warning in cls.warnings]) == set(\n        [\"eemeter.data_quality.utc_index\"]\n    )\n    assert len(cls.disqualification) == 0\n\n\n@pytest.mark.parametrize(\n    \"get_datetime_index\", [[\"30min\", True], [\"h\", True]], indirect=True\n)\ndef test_billing_reporting_data_with_missing_half_hourly_frequencies(\n    get_datetime_index,\n):\n    datetime_index = get_datetime_index\n    datetime_index = datetime_index[datetime_index.year == 2023]\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df[mask].sample(frac=0.6, random_state=42).index, \"temperature\"] = np.nan\n\n    cls = BillingReportingData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n\n    if datetime_index.freq == \"30min\":\n        assert len(cls.df.temperature.dropna()) == 268\n    elif datetime_index.freq == \"h\":\n        assert len(cls.df.temperature.dropna()) == 270\n\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\"\n    )\n    assert len(cls.disqualification) == 3\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", True]], indirect=True)\ndef test_billing_reporting_data_with_missing_daily_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n    datetime_index = datetime_index[datetime_index.year == 2023]\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df[mask].sample(frac=0.6, random_state=42).index, \"temperature\"] = np.nan\n\n    cls = BillingReportingData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert len(cls.df.temperature.dropna()) == len(df.temperature.dropna())\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 3\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_dst_handling():\n    # 2020-03-08 02:00 is nonexistent, should push to 03:00\n    tz = \"America/New_York\"\n    idx = DatetimeIndex(\n        [\n            Timestamp(\"2020-03-07 02\", tz=tz),\n            Timestamp(\"2020-04-06 02\", tz=tz),\n            Timestamp(\"2020-05-06 02\", tz=tz),\n        ]\n    )\n    df = DataFrame({\"observed\": [1] * 3, \"temperature\": [50] * 3}, index=idx)\n    baseline = BillingBaselineData(df, is_electricity_data=True)\n    assert len(baseline.df) == 61\n    hours = np.unique(baseline.df.index.hour)\n    assert (hours == [2, 3]).all()\n\n    # 2020-11-01 01:00 is ambiguous, single index should be chosen\n    tz = \"America/New_York\"\n    idx = DatetimeIndex(\n        [\n            Timestamp(\"2020-10-31 01\", tz=tz),\n            Timestamp(\"2020-11-28 01\", tz=tz),\n            Timestamp(\"2020-12-28 01\", tz=tz),\n        ]\n    )\n    df = DataFrame({\"observed\": [1] * 3, \"temperature\": [50] * 3}, index=idx)\n    baseline = BillingBaselineData(df, is_electricity_data=True)\n    assert (baseline.df.index.hour == 1).all()\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_daily_data.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nfrom datetime import datetime\n\nfrom opendsm.eemeter.common.transform import get_baseline_data\nfrom opendsm.eemeter.models.daily.data import DailyBaselineData, DailyReportingData\nfrom opendsm.eemeter.samples import load_sample\nimport numpy as np\nimport pandas as pd\nfrom pandas import Timestamp, DatetimeIndex, DataFrame\nimport pytest\n\nTEMPERATURE_SEED = 29\nMETER_SEED = 41\nNUM_DAYS_IN_YEAR = 365\n\n\n@pytest.fixture\ndef get_datetime_index(request):\n    # Request = [frequency , is_timezone_aware]\n\n    # Create a DateTimeIndex at 30-minute intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=request.param[0],\n        tz=\"US/Eastern\" if request.param[1] else None,\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_half_hourly_with_timezone():\n    # Create a DateTimeIndex at 30-minute intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"30min\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_hourly_with_timezone():\n    # Create a DateTimeIndex at 30-minute intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"h\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_daily_with_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\",\n        end=\"2024-01-01\",\n        inclusive=\"left\",\n        freq=\"D\",\n        tz=\"US/Eastern\",\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_datetime_index_daily_without_timezone():\n    # Create a DateTimeIndex at daily intervals\n    datetime_index = pd.date_range(\n        start=\"2023-01-01\", end=\"2024-01-01\", inclusive=\"left\", freq=\"D\"\n    )\n\n    return datetime_index\n\n\n@pytest.fixture\ndef get_temperature_data_half_hourly(get_datetime_index_half_hourly_with_timezone):\n    datetime_index = get_datetime_index_half_hourly_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_temperature_data_hourly(get_datetime_index_hourly_with_timezone):\n    datetime_index = get_datetime_index_hourly_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_meter_data_daily(get_datetime_index_daily_with_timezone):\n    datetime_index = get_datetime_index_daily_with_timezone\n\n    np.random.seed(METER_SEED)\n    # Create a 'meter_value' column with random data\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"observed\": meter_value}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_meter_data_daily_with_extreme_values_and_negative_values(\n    get_datetime_index_daily_with_timezone,\n):\n    datetime_index = get_datetime_index_daily_with_timezone\n\n    np.random.seed(METER_SEED)\n    # Create a 'meter_value' column with random data\n    # Last 60 will be for extreme values\n    meter_value = np.random.normal(loc=0.0, scale=100.0, size=len(datetime_index) - 60)\n    median = np.median(meter_value)\n    q75, q25 = np.percentile(meter_value, [75, 25])\n    iqr = q75 - q25\n\n    # Generate some extreme values more than thrice the interquartile range from the median\n    extreme_values_right = (\n        median + (3 * iqr) + np.random.normal(loc=0.0, scale=100.0, size=30)\n    )\n    extreme_values_left = median - (\n        (3 * iqr) + np.random.normal(loc=0.0, scale=100.0, size=30)\n    )\n\n    meter_value = np.concatenate(\n        (extreme_values_right, meter_value, extreme_values_left)\n    )\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"observed\": meter_value}, index=datetime_index)\n\n    return df\n\n\n@pytest.fixture\ndef get_temperature_data_daily(get_datetime_index_daily_with_timezone):\n    datetime_index = get_datetime_index_daily_with_timezone\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' column with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    return df\n\n\n# Check that a missing timezone raises a Value Error\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", False]], indirect=True)\ndef test_daily_baseline_data_with_missing_timezone(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"meter\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n\n    with pytest.raises(ValueError):\n        cls = DailyBaselineData(df, is_electricity_data=True)\n\n\n# Check that a missing datetime index and column raises a Value Error\ndef test_daily_baseline_data_with_missing_datetime_index_and_column():\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(NUM_DAYS_IN_YEAR)\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(NUM_DAYS_IN_YEAR)\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"meter\": meter_value, \"temperature\": temperature_mean})\n\n    with pytest.raises(ValueError):\n        cls = DailyBaselineData(df, is_electricity_data=True)\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", True]], indirect=True)\ndef test_daily_baseline_data_with_datetime_column(get_datetime_index):\n    df = pd.DataFrame()\n    df[\"datetime\"] = get_datetime_index\n    np.random.seed(TEMPERATURE_SEED)\n    df[\"temperature\"] = np.random.rand(len(get_datetime_index))\n    np.random.seed(METER_SEED)\n    df[\"observed\"] = np.random.rand(len(get_datetime_index))\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 0\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", True]], indirect=True)\ndef test_daily_baseline_data_with_same_daily_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"observed\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 0\n\n\n@pytest.mark.parametrize(\n    \"get_datetime_index\", [[\"30min\", True], [\"h\", True]], indirect=True\n)\ndef test_daily_baseline_data_with_same_hourly_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    np.random.seed(METER_SEED)\n    meter_value = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"observed\": meter_value, \"temperature\": temperature_mean},\n        index=datetime_index,\n    )\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    # TODO: Because of the weird behaviour of as_freq() on the last hour for downsampling, so we can't add it\n    assert round(cls.df.observed.sum(), 2) == round(df.observed[:-1].sum(), 2)\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_daily_and_half_hourly_frequencies(\n    get_temperature_data_half_hourly, get_meter_data_daily\n):\n    # Create a DataFrame with uneven frequency\n    df = get_temperature_data_half_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_daily_and_hourly_frequencies(\n    get_meter_data_daily, get_temperature_data_hourly\n):\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_extreme_values_in_daily_and_hourly_frequencies(\n    get_meter_data_daily_with_extreme_values_and_negative_values,\n    get_temperature_data_hourly,\n):\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily_with_extreme_values_and_negative_values\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.extreme_values_detected\"\n    )\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_extreme_and_negative_values_in_daily_and_hourly_frequencies(\n    get_meter_data_daily_with_extreme_values_and_negative_values,\n    get_temperature_data_hourly,\n):\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily_with_extreme_values_and_negative_values\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=False)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.extreme_values_detected\"\n    )\n    assert len(cls.disqualification) == 1\n    assert (\n        cls.disqualification[0].qualified_name\n        == \"eemeter.sufficiency_criteria.negative_observed_values\"\n    )\n\n\ndef test_daily_baseline_data_with_specific_hourly_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-hourly\")\n    meter = meter[meter.index.year == 2017]\n    temperature = temperature[temperature.index.year == 2017]\n    cls = DailyBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    # TODO: Because of the weird behaviour of as_freq() on the last hour for downsampling, so we can't add it\n    assert round(cls.df.observed.sum(), 2) == round(meter.value[:-1].sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.extreme_values_detected\",\n    ]\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_specific_daily_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-daily\")\n    meter = meter[meter.index.year == 2017]\n    temperature = temperature[temperature.index.year == 2017]\n    cls = DailyBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.extreme_values_detected\",\n    ]\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_missing_specific_daily_input():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-daily\")\n    meter = meter[meter.index.year == 2017]\n    # Set 1 month meter data to NaN\n    meter.loc[meter.index.month == 4] = np.nan\n    temperature = temperature[temperature.index.year == 2017]\n    cls = DailyBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(meter.value.sum(), 2)\n    assert len(cls.warnings) == 2\n    assert [warning.qualified_name for warning in cls.warnings] == [\n        \"eemeter.data_quality.utc_index\",\n        \"eemeter.sufficiency_criteria.extreme_values_detected\",\n    ]\n    assert len(cls.disqualification) == 0\n\n\ndef test_daily_baseline_data_with_missing_hourly_temperature_data(\n    get_meter_data_daily, get_temperature_data_hourly\n):\n    df = get_temperature_data_hourly\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    df.loc[df[mask].sample(frac=0.6).index, \"temperature\"] = np.nan\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\"\n    )\n    assert len(cls.disqualification) == 3\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_daily_baseline_data_with_missing_half_hourly_temperature_data(\n    get_meter_data_daily, get_temperature_data_half_hourly\n):\n    df = get_temperature_data_half_hourly\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df[mask].sample(frac=0.6).index, \"temperature\"] = np.nan\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\"\n    )\n    assert len(cls.disqualification) == 3\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_daily_baseline_data_with_missing_daily_temperature_data(\n    get_meter_data_daily, get_temperature_data_daily\n):\n    df = get_temperature_data_daily\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df.index.dayofweek.isin([1, 3]), \"temperature\"] = np.nan\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    assert len(cls.disqualification) == 3\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_daily_baseline_data_with_missing_meter_data(\n    get_meter_data_daily, get_temperature_data_hourly\n):\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Set Tuesdays & Thursdays data as missing\n    df_meter.loc[df_meter.index.dayofweek.isin([1, 3]), \"observed\"] = np.nan\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    # assert all(warning.qualified_name in expected_warnings for warning in cls.warnings)\n    assert len(cls.disqualification) == 2\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_observed_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_daily_baseline_data_with_missing_meter_data_37_days(\n    get_meter_data_daily, get_temperature_data_hourly\n):\n    df = get_temperature_data_hourly\n\n    # Create a DataFrame with daily frequency\n    df_meter = get_meter_data_daily\n\n    # Set Tuesdays & Thursdays data as missing\n    df_meter.loc[df_meter.index[1:38], \"observed\"] = np.nan\n\n    # Merge 'df' and 'df_meter' in an outer join\n    df = df.merge(df_meter, left_index=True, right_index=True, how=\"outer\")\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert round(cls.df.observed.sum(), 2) == round(df.observed.sum(), 2)\n    assert len(cls.warnings) == 0\n    # assert all(warning.qualified_name in expected_warnings for warning in cls.warnings)\n    assert len(cls.disqualification) == 2\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_observed_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_duplicate_datetime_index_values():\n    # Create a Timestamp with a specific date\n    timestamp = pd.Timestamp(\"2023-01-01\")\n\n    # Create an Index with 365 identical timestamps\n    datetime_index = pd.DatetimeIndex([timestamp] * 365, tz=\"US/Eastern\")\n\n    # Create random values for 'observed' and 'temperature'\n    observed = np.random.rand(len(datetime_index))\n    temperature = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(\n        data={\"observed\": observed, \"temperature\": temperature}, index=datetime_index\n    )\n\n    cls = DailyBaselineData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == 1\n\n\n@pytest.mark.parametrize(\n    \"get_datetime_index\", [[\"30min\", True], [\"h\", True]], indirect=True\n)\ndef test_daily_reporting_data_with_half_hourly_and_hourly_frequencies(\n    get_datetime_index,\n):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    cls = DailyReportingData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert len(cls.warnings) == 0\n    assert len(cls.disqualification) == 0\n\n\n@pytest.mark.parametrize(\n    \"get_datetime_index\", [[\"30min\", True], [\"h\", True]], indirect=True\n)\ndef test_daily_reporting_data_with_missing_half_hourly_and_hourly_frequencies(\n    get_datetime_index,\n):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df[mask].sample(frac=0.6, random_state=42).index, \"temperature\"] = np.nan\n\n    cls = DailyReportingData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n\n    if datetime_index.freq == \"30min\":\n        assert len(cls.df.temperature.dropna()) == 268\n    elif datetime_index.freq == \"h\":\n        assert len(cls.df.temperature.dropna()) == 270\n\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\"\n    )\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\ndef test_daily_reporting_data_high_frequency_temperature_warning_gives_proper_results():\n\n    datetime_index = pd.date_range(\n        \"2023-01-01\", \"2023-01-08\", freq=\"h\", tz=\"US/Eastern\"\n    )\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    # Nan all of 2023-01-01\n    df.loc[\"2023-01-01 06:00\":\"2023-01-01 18:00\", \"temperature\"] = np.nan\n\n    cls = DailyReportingData(df, is_electricity_data=True)\n\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.missing_high_frequency_temperature_data\"\n    )\n    # Should return just the day that has too many nulls.\n    assert len(cls.warnings[0].data) == 1\n\n\n@pytest.mark.parametrize(\"get_datetime_index\", [[\"D\", True]], indirect=True)\ndef test_daily_reporting_data_with_missing_daily_frequencies(get_datetime_index):\n    datetime_index = get_datetime_index\n\n    np.random.seed(TEMPERATURE_SEED)\n    # Create a 'temperature_mean' and meter_value columns with random data\n    temperature_mean = np.random.rand(len(datetime_index))\n\n    # Create the DataFrame\n    df = pd.DataFrame(data={\"temperature\": temperature_mean}, index=datetime_index)\n\n    # Create a mask for Tuesdays and Thursdays\n    mask = df.index.dayofweek.isin([1, 3])\n\n    # Set 60% of the temperature data as missing on Tuesdays and Thursdays\n    # This should cause the high frequency temperature check to fail on these days\n    df.loc[df[mask].sample(frac=0.6, random_state=42).index, \"temperature\"] = np.nan\n\n    cls = DailyReportingData(df, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n    assert len(cls.df.temperature.dropna()) == len(df.temperature.dropna())\n    assert len(cls.warnings) == 1\n    assert (\n        cls.warnings[0].qualified_name\n        == \"eemeter.sufficiency_criteria.unable_to_confirm_daily_temperature_sufficiency\"\n    )\n    expected_disqualifications = [\n        \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n        \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n    ]\n    assert all(\n        disqualification.qualified_name in expected_disqualifications\n        for disqualification in cls.disqualification\n    )\n\n\n@pytest.fixture\ndef baseline_data_daily_params(il_electricity_cdd_hdd_daily):\n    def _baseline(tz=\"UTC\", hour=0):\n        meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n        temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n        blackout_start_date = il_electricity_cdd_hdd_daily[\"blackout_start_date\"]\n        baseline_meter_data, warnings = get_baseline_data(\n            meter_data, end=blackout_start_date\n        )\n        baseline_meter_data.index = map(\n            lambda x: x.replace(hour=x.hour + hour), baseline_meter_data.index\n        )\n        baseline_meter_data.index = baseline_meter_data.index.tz_convert(tz)\n        return baseline_meter_data, temperature_data\n\n    yield _baseline\n\n\n@pytest.mark.parametrize(\n    [\"tz\", \"hour\"], [[\"US/Pacific\", 3], [\"US/Eastern\", 8], [\"Europe/London\", 13]]\n)\ndef test_offset_temperature_aggregations(baseline_data_daily_params, tz, hour):\n    baseline_meter_data, temp_series = baseline_data_daily_params(tz=tz, hour=hour)\n    baseline = DailyBaselineData.from_series(\n        baseline_meter_data, temp_series, is_electricity_data=True\n    )\n    tz = baseline_meter_data.index.tz\n\n    abs_diff = 0\n    for day in baseline.df.index:\n        abs_diff += abs(\n            temp_series[day : day + pd.Timedelta(hours=23)].mean()\n            - baseline.df.temperature.loc[day].squeeze()\n        )\n    assert abs_diff < 0.000001\n\n\ndef test_non_ns_datetime_index():\n    meter, temperature, _ = load_sample(\"il-electricity-cdd-hdd-hourly\")\n    meter = meter[meter.index.year == 2017]\n    temperature = temperature[temperature.index.year == 2017]\n\n    # convert to microseconds\n    meter.index = meter.index.astype(\"datetime64[us, UTC]\")\n    temperature.index = temperature.index.astype(\"datetime64[us, UTC]\")\n    cls = DailyBaselineData.from_series(meter, temperature, is_electricity_data=True)\n\n    assert cls.df is not None\n    assert len(cls.df) == NUM_DAYS_IN_YEAR\n\n\ndef test_offset_aggregations_hourly(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    baseline_meter_data = baseline_meter_data.iloc[3:]  # begin from 3AM UTC\n    baseline = DailyBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n    assert baseline is not None\n    assert len(baseline.df) == NUM_DAYS_IN_YEAR\n\n\ndef test_dst_handling():\n    # 2020-03-08 02:00 is nonexistent, should push to 03:00\n    tz = \"America/New_York\"\n    idx = DatetimeIndex(\n        [Timestamp(\"2020-03-07 02\", tz=tz), Timestamp(\"2021-03-06 02\", tz=tz)]\n    )\n    df = DataFrame({\"observed\": [1] * 2, \"temperature\": [50] * 2}, index=idx)\n    baseline = DailyBaselineData(df, is_electricity_data=True)\n    assert len(baseline.df) == 365\n    hours, counts = np.unique(baseline.df.index.hour, return_counts=True)\n    assert (hours == [2, 3]).all()\n    assert (counts == [364, 1]).all()\n\n    # 2020-11-01 01:00 is ambiguous, single index should be chosen\n    tz = \"America/New_York\"\n    idx = DatetimeIndex(\n        [Timestamp(\"2020-03-07 01\", tz=tz), Timestamp(\"2021-03-06 01\", tz=tz)]\n    )\n    df = DataFrame({\"observed\": [1] * 2, \"temperature\": [50] * 2}, index=idx)\n    baseline = DailyBaselineData(df, is_electricity_data=True)\n    assert len(baseline.df) == 365\n    assert (baseline.df.index.hour == 1).all()\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_daily_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport pytest\n\nimport numpy as np\n\nfrom opendsm.eemeter import DailyModel, DailyBaselineData, DailyReportingData\nfrom opendsm.eemeter.samples import load_sample\nfrom opendsm.eemeter.common.transform import get_baseline_data\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\n\n\n@pytest.fixture\ndef daily_series():\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-daily\"\n    )\n    blackout_start_date = sample_metadata[\"blackout_start_date\"]\n    meter_data.index = meter_data.index.tz_convert(\"US/Pacific\")\n\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date, max_days=365\n    )\n    baseline_meter_data = baseline_meter_data[:-1]  # drop nan\n    return baseline_meter_data, temperature_data\n\n\n@pytest.fixture\ndef bad_daily_series(daily_series):\n    meter, temp = daily_series\n    meter[:50] += 3000\n    return meter, temp\n\n\n@pytest.fixture\ndef missing_daily_data(bad_daily_series) -> DailyBaselineData:\n    meter, temp = bad_daily_series\n    meter = meter[:-90]\n    baseline_data = DailyBaselineData.from_series(meter, temp, is_electricity_data=True)\n    return baseline_data\n\n\n@pytest.fixture\ndef bad_daily_data(bad_daily_series) -> DailyBaselineData:\n    meter, temp = bad_daily_series\n    baseline_data = DailyBaselineData.from_series(meter, temp, is_electricity_data=True)\n    return baseline_data\n\n\ndef test_disqualified_data_error(missing_daily_data):\n    with pytest.raises(DataSufficiencyError):\n        model = DailyModel().fit(missing_daily_data)\n    model = DailyModel().fit(missing_daily_data, ignore_disqualification=True)\n    with pytest.raises(DisqualifiedModelError):\n        model.predict(bad_daily_data)\n    model.predict(missing_daily_data, ignore_disqualification=True)\n\n\ndef test_model_cvrmse_error(bad_daily_data):\n    model = DailyModel().fit(bad_daily_data)\n    with pytest.raises(DisqualifiedModelError):\n        model.predict(bad_daily_data)\n    model.predict(bad_daily_data, ignore_disqualification=True)\n\n\ndef test_timezone_behavior(daily_series):\n    # TODO probably move some of this to dataclass tests\n    meter, temp = daily_series\n    # ensure that meter is using local tz\n    assert str(meter.index.tz) == \"US/Pacific\"\n    assert str(temp.index.tz) == \"UTC\"\n\n    baseline_data = DailyBaselineData.from_series(meter, temp, is_electricity_data=True)\n\n    # require is_electricity_data flag when passing meter data\n    with pytest.raises(ValueError):\n        DailyReportingData.from_series(meter, temp)\n\n    # fail when passing timezone both through index as well as param\n    with pytest.raises(ValueError):\n        DailyReportingData.from_series(meter, temp, tzinfo=meter.index.tz)\n\n    model = DailyModel().fit(baseline_data)\n\n    # fail when attempting to predict on data with different timezone from baseline\n    reporting_data_no_meter_utc = DailyReportingData.from_series(None, temp)\n    assert model.baseline_timezone != reporting_data_no_meter_utc.tz\n    with pytest.raises(ValueError):\n        model.predict(reporting_data_no_meter_utc)\n\n    reporting_data = DailyReportingData.from_series(\n        meter, temp, is_electricity_data=True\n    )\n    res1 = model.predict(reporting_data)\n    reporting_data_no_meter = DailyReportingData.from_series(\n        None, temp, tzinfo=meter.index.tz\n    )\n    res2 = model.predict(reporting_data_no_meter)\n    assert round((res1[\"temperature\"] - res2[\"temperature\"]).sum(), 2) == 0\n    assert round((res1[\"predicted\"] - res2[\"predicted\"]).sum(), 2) == 0\n\n\ndef test_predict_df_matches_input_index(daily_series):\n    meter, temp = daily_series\n    baseline_data = DailyBaselineData.from_series(meter, temp, is_electricity_data=True)\n    baseline_model = DailyModel().fit(baseline_data)\n\n    temp[temp.index.day > 20] = np.nan\n    reporting_data_missing_temp = DailyBaselineData.from_series(\n        meter, temp, is_electricity_data=True\n    )\n    res = baseline_model.predict(reporting_data_missing_temp)\n    assert len(res) == len(reporting_data_missing_temp.df)"
  },
  {
    "path": "tests/eemeter/daily_model/test_data.csv",
    "content": "datetime,temperature,observed,season,day_of_week\n2019-03-02 00:00:00+00,53.137118613144565,73.24072778136863,shoulder,6\n2019-03-03 00:00:00+00,53.73536557792833,20.576371394610867,shoulder,7\n2019-03-04 00:00:00+00,52.27461468937442,107.59656751108996,shoulder,1\n2019-03-05 00:00:00+00,54.192802615441245,96.09131731105559,shoulder,2\n2019-03-06 00:00:00+00,56.78148607080606,96.21321208467792,shoulder,3\n2019-03-07 00:00:00+00,52.818154162476425,77.11286303349503,shoulder,4\n2019-03-08 00:00:00+00,48.88319890004381,92.20143556211055,shoulder,5\n2019-03-09 00:00:00+00,49.19597710197547,79.16295922176249,shoulder,6\n2019-03-10 00:00:00+00,50.911395249015264,31.206426011962783,shoulder,7\n2019-03-11 00:00:00+00,51.434189116903624,73.7982216076401,shoulder,1\n2019-03-12 00:00:00+00,55.322761999187044,82.96067185711587,shoulder,2\n2019-03-13 00:00:00+00,53.876121687218756,61.696324594845464,shoulder,3\n2019-03-14 00:00:00+00,50.706722489272586,85.45856233005058,shoulder,4\n2019-03-15 00:00:00+00,55.21367377374803,78.95046580907251,shoulder,5\n2019-03-16 00:00:00+00,59.25438171426027,59.17765382637946,shoulder,6\n2019-03-17 00:00:00+00,57.56811181859746,21.48744296287084,shoulder,7\n2019-03-18 00:00:00+00,60.20243047146752,63.95756941987107,shoulder,1\n2019-03-19 00:00:00+00,61.601793383630564,49.37610986598179,shoulder,2\n2019-03-20 00:00:00+00,54.577296949926335,50.751423947338495,shoulder,3\n2019-03-21 00:00:00+00,54.21057244847071,74.67964115698608,shoulder,4\n2019-03-22 00:00:00+00,52.790671676989746,75.35829793626799,shoulder,5\n2019-03-23 00:00:00+00,56.76628803680522,47.198095413425705,shoulder,6\n2019-03-24 00:00:00+00,52.6545244793039,23.416806919029653,shoulder,7\n2019-03-25 00:00:00+00,54.72267090074537,70.00685395242245,shoulder,1\n2019-03-26 00:00:00+00,56.17962664174209,69.85704061330627,shoulder,2\n2019-03-27 00:00:00+00,57.65821386146988,76.00249307430846,shoulder,3\n2019-03-28 00:00:00+00,54.16222751485439,88.79705869658989,shoulder,4\n2019-03-29 00:00:00+00,53.68995119072392,73.90991523206974,shoulder,5\n2019-03-30 00:00:00+00,56.88719487231945,50.732280494152235,shoulder,6\n2019-03-31 00:00:00+00,61.421812272282935,21.03153990808867,shoulder,7\n2019-04-01 00:00:00+00,59.9158574870139,62.72629514962518,shoulder,1\n2019-04-02 00:00:00+00,57.409722546789986,70.57930483141617,shoulder,2\n2019-04-03 00:00:00+00,58.98758783745752,75.27537659043706,shoulder,3\n2019-04-04 00:00:00+00,60.32769588807567,87.5679817967266,shoulder,4\n2019-04-05 00:00:00+00,57.3180445322531,81.34859771377022,shoulder,5\n2019-04-06 00:00:00+00,61.492211790844635,59.32218908895996,shoulder,6\n2019-04-07 00:00:00+00,63.72071045095698,33.12337336894876,shoulder,7\n2019-04-08 00:00:00+00,64.9640452022286,52.98814354404445,shoulder,1\n2019-04-09 00:00:00+00,60.28003406581222,59.64007449019475,shoulder,2\n2019-04-10 00:00:00+00,59.37017427274727,50.72654082222284,shoulder,3\n2019-04-11 00:00:00+00,57.92055267966963,61.037323365908634,shoulder,4\n2019-04-12 00:00:00+00,61.52881175033112,44.88417595277447,shoulder,5\n2019-04-13 00:00:00+00,65.42977907309661,57.16846502139367,shoulder,6\n2019-04-14 00:00:00+00,60.06854021569319,23.39249851264629,shoulder,7\n2019-04-15 00:00:00+00,54.68151750292829,64.9552145846722,shoulder,1\n2019-04-16 00:00:00+00,57.350288583819015,49.90133460329755,shoulder,2\n2019-04-17 00:00:00+00,61.99518232890943,37.282712768312656,shoulder,3\n2019-04-18 00:00:00+00,66.9076515958632,48.87279080518876,shoulder,4\n2019-04-19 00:00:00+00,66.23609377931407,54.631722733039204,shoulder,5\n2019-04-20 00:00:00+00,59.76829571840837,39.07124118925527,shoulder,6\n2019-04-21 00:00:00+00,59.50850430305928,20.2666420163597,shoulder,7\n2019-04-22 00:00:00+00,66.89572671041299,36.46552902352498,shoulder,1\n2019-04-23 00:00:00+00,72.15128191994911,54.82757528041897,shoulder,2\n2019-04-24 00:00:00+00,74.68105229723209,64.21832659852436,shoulder,3\n2019-04-25 00:00:00+00,68.42296514757524,63.44219365272238,shoulder,4\n2019-04-26 00:00:00+00,67.51927056317977,62.04735651772149,shoulder,5\n2019-04-27 00:00:00+00,62.83978457161031,48.28645288156518,shoulder,6\n2019-04-28 00:00:00+00,60.73622427102306,39.497643607023065,shoulder,7\n2019-04-29 00:00:00+00,59.66899094789422,48.153307110827306,shoulder,1\n2019-04-30 00:00:00+00,60.96911329037422,52.08454531373839,shoulder,2\n2019-05-01 00:00:00+00,63.74647715333248,45.86440019897403,shoulder,3\n2019-05-02 00:00:00+00,66.93251551117243,55.656328429107354,shoulder,4\n2019-05-03 00:00:00+00,64.2305348192942,47.78735946581706,shoulder,5\n2019-05-04 00:00:00+00,61.18838251610232,50.80007277689701,shoulder,6\n2019-05-05 00:00:00+00,56.976857614849486,19.255823296670584,shoulder,7\n2019-05-06 00:00:00+00,58.81318526793773,57.62919745649882,shoulder,1\n2019-05-07 00:00:00+00,61.39605177900328,61.17995077213011,shoulder,2\n2019-05-08 00:00:00+00,61.70259800489901,51.55768679418883,shoulder,3\n2019-05-09 00:00:00+00,59.2626448207345,59.8867578799644,shoulder,4\n2019-05-10 00:00:00+00,63.734929267018614,47.51305119348468,shoulder,5\n2019-05-11 00:00:00+00,63.703761664332625,50.68158875803705,shoulder,6\n2019-05-12 00:00:00+00,66.42233666838789,26.61124590313798,shoulder,7\n2019-05-13 00:00:00+00,62.35971150989514,54.071092028591615,shoulder,1\n2019-05-14 00:00:00+00,60.706999449454685,60.72680185621305,shoulder,2\n2019-05-15 00:00:00+00,63.94642070843988,50.15781465347994,shoulder,3\n2019-05-16 00:00:00+00,56.357076906700854,54.553317936545554,shoulder,4\n2019-05-17 00:00:00+00,59.03478747860212,56.525814713438244,shoulder,5\n2019-05-18 00:00:00+00,56.30579331556001,57.196512427983876,shoulder,6\n2019-05-19 00:00:00+00,56.474969854008926,30.7850537259825,shoulder,7\n2019-05-20 00:00:00+00,57.080395576672736,55.926574502380504,shoulder,1\n2019-05-21 00:00:00+00,59.64301311995156,65.03093967960959,shoulder,2\n2019-05-22 00:00:00+00,63.445164044962254,66.89027425065113,shoulder,3\n2019-05-23 00:00:00+00,65.54563521764962,69.63083716677207,shoulder,4\n2019-05-24 00:00:00+00,65.45026386385462,64.0748705017955,shoulder,5\n2019-05-25 00:00:00+00,63.06734130480435,38.8458758363542,shoulder,6\n2019-05-26 00:00:00+00,56.49243123573132,32.98615206883415,shoulder,7\n2019-05-27 00:00:00+00,60.588647416757446,21.33708534467387,shoulder,1\n2019-05-28 00:00:00+00,67.36379286531535,55.17112403367709,shoulder,2\n2019-05-29 00:00:00+00,67.35219856943151,60.66781936353276,shoulder,3\n2019-05-30 00:00:00+00,61.39633509813009,56.79049173645935,shoulder,4\n2019-05-31 00:00:00+00,67.95968206878484,62.22038521167098,shoulder,5\n2019-06-01 00:00:00+00,68.34850678398392,49.64457106737547,summer,6\n2019-06-02 00:00:00+00,63.078733819364054,26.466267091078812,summer,7\n2019-06-03 00:00:00+00,67.54545299138591,68.24108606683022,summer,1\n2019-06-04 00:00:00+00,75.06773008504824,74.56370869115767,summer,2\n2019-06-05 00:00:00+00,75.67679748920239,55.33611586203097,summer,3\n2019-06-06 00:00:00+00,66.55439175516642,51.5163910717773,summer,4\n2019-06-07 00:00:00+00,65.24455902042341,62.18573287937046,summer,5\n2019-06-08 00:00:00+00,71.27953477538222,75.94979748353374,summer,6\n2019-06-09 00:00:00+00,78.43577714782236,31.336805021484984,summer,7\n2019-06-10 00:00:00+00,86.2071663291415,69.6000569550156,summer,1\n2019-06-11 00:00:00+00,86.91811908220063,81.47403122756285,summer,2\n2019-06-12 00:00:00+00,82.46289725540773,79.37086077829086,summer,3\n2019-06-13 00:00:00+00,69.37021906843245,64.6248140556269,summer,4\n2019-06-14 00:00:00+00,65.82358402295168,51.97378336287416,summer,5\n2019-06-15 00:00:00+00,63.044787695672056,57.78188380422936,summer,6\n2019-06-16 00:00:00+00,64.62996482057434,28.18135174606497,summer,7\n2019-06-17 00:00:00+00,71.24610043385442,61.6173694156463,summer,1\n2019-06-18 00:00:00+00,75.02405529938166,61.21026983837603,summer,2\n2019-06-19 00:00:00+00,68.27080420739186,56.03275536295207,summer,3\n2019-06-20 00:00:00+00,67.75004238409616,58.7339959986228,summer,4\n2019-06-21 00:00:00+00,69.88363861410387,53.28982892570211,summer,5\n2019-06-22 00:00:00+00,75.62647063539471,60.805327357301806,summer,6\n2019-06-23 00:00:00+00,78.63878226997569,30.002492133395428,summer,7\n2019-06-24 00:00:00+00,72.0607651444057,55.2571711813241,summer,1\n2019-06-25 00:00:00+00,73.9596837376786,59.11674922969432,summer,2\n2019-06-26 00:00:00+00,65.63995678847465,54.41014814035278,summer,3\n2019-06-27 00:00:00+00,66.43089974532354,49.4437058989904,summer,4\n2019-06-28 00:00:00+00,69.62884765123434,48.1051801417369,summer,5\n2019-06-29 00:00:00+00,71.20858207549176,76.2806627285206,summer,6\n2019-06-30 00:00:00+00,69.6486764121407,35.32169187834266,summer,7\n2019-07-01 00:00:00+00,69.01460035307792,56.01218744553439,summer,1\n2019-07-02 00:00:00+00,68.94362993759611,63.73250437116537,summer,2\n2019-07-03 00:00:00+00,68.69365890296282,45.78531338013363,summer,3\n2019-07-04 00:00:00+00,72.52917174967259,27.842850375047863,summer,4\n2019-07-05 00:00:00+00,73.96131999581613,26.76046638336144,summer,5\n2019-07-06 00:00:00+00,70.90186504370509,53.445738416646165,summer,6\n2019-07-07 00:00:00+00,64.80566040964877,36.52039431103506,summer,7\n2019-07-08 00:00:00+00,66.81764490397484,59.88005582081406,summer,1\n2019-07-09 00:00:00+00,65.44866688758032,51.28144970834331,summer,2\n2019-07-10 00:00:00+00,73.8282852344148,50.55913560732101,summer,3\n2019-07-11 00:00:00+00,74.39414492604355,42.58727989658026,summer,4\n2019-07-12 00:00:00+00,72.87929748493141,58.762072019612006,summer,5\n2019-07-13 00:00:00+00,75.3267453259947,55.26002796021136,summer,6\n2019-07-14 00:00:00+00,74.06283068534967,21.281365133922463,summer,7\n2019-07-15 00:00:00+00,78.82328633187909,69.84284318464628,summer,1\n2019-07-16 00:00:00+00,74.35155623996995,61.026718668295906,summer,2\n2019-07-17 00:00:00+00,74.67942541491331,49.21097581681647,summer,3\n2019-07-18 00:00:00+00,73.4977774095084,45.78382283167484,summer,4\n2019-07-19 00:00:00+00,70.87554367997066,42.63503225967031,summer,5\n2019-07-20 00:00:00+00,68.39277864148266,46.83298570752552,summer,6\n2019-07-21 00:00:00+00,71.47291279815413,38.75263672857915,summer,7\n2019-07-22 00:00:00+00,76.95796899786833,44.40547774021117,summer,1\n2019-07-23 00:00:00+00,75.46225482734296,42.10926964439052,summer,2\n2019-07-24 00:00:00+00,80.03209292937065,58.814654727091394,summer,3\n2019-07-25 00:00:00+00,77.07129784421464,57.61578290645527,summer,4\n2019-07-26 00:00:00+00,74.28253547538647,38.92668437906225,summer,5\n2019-07-27 00:00:00+00,76.79937468277544,47.62046146210038,summer,6\n2019-07-28 00:00:00+00,79.8640266022431,28.26731659906121,summer,7\n2019-07-29 00:00:00+00,67.7923543309771,48.59637922930998,summer,1\n2019-07-30 00:00:00+00,68.2605794376176,55.487236013611664,summer,2\n2019-07-31 00:00:00+00,72.29282352723882,57.92029491618369,summer,3\n2019-08-01 00:00:00+00,67.9226064475243,51.2292841188063,summer,4\n2019-08-02 00:00:00+00,74.86188118124049,56.647762419374494,summer,5\n2019-08-03 00:00:00+00,77.70004597497588,63.87611698122371,summer,6\n2019-08-04 00:00:00+00,75.82396026171737,27.96977141178209,summer,7\n2019-08-05 00:00:00+00,73.00259761424053,66.88456561568074,summer,1\n2019-08-06 00:00:00+00,73.95469822411799,67.52903425576153,summer,2\n2019-08-07 00:00:00+00,69.84759174013622,58.4634571496794,summer,3\n2019-08-08 00:00:00+00,66.60074095994835,64.70990751866535,summer,4\n2019-08-09 00:00:00+00,72.58496517213516,62.62294334171344,summer,5\n2019-08-10 00:00:00+00,71.3928486512502,46.67050777626701,summer,6\n2019-08-11 00:00:00+00,74.09942315138585,37.04792966543948,summer,7\n2019-08-12 00:00:00+00,80.0336098316482,67.26913854556555,summer,1\n2019-08-13 00:00:00+00,81.19668906056131,66.75989916616527,summer,2\n2019-08-14 00:00:00+00,83.44386097177167,79.19947524645968,summer,3\n2019-08-15 00:00:00+00,85.19043957881546,70.84826934384851,summer,4\n2019-08-16 00:00:00+00,80.5940173090673,54.884161966536695,summer,5\n2019-08-17 00:00:00+00,71.2214704197305,52.49763885652746,summer,6\n2019-08-18 00:00:00+00,69.77502754665211,36.51702413653174,summer,7\n2019-08-19 00:00:00+00,68.20834324126972,50.87372703430165,summer,1\n2019-08-20 00:00:00+00,69.18034522887818,62.417765320494944,summer,2\n2019-08-21 00:00:00+00,75.31905185963791,62.280849736664734,summer,3\n2019-08-22 00:00:00+00,79.5276597147532,65.87129263991694,summer,4\n2019-08-23 00:00:00+00,72.91249253403666,68.25799312159799,summer,5\n2019-08-24 00:00:00+00,75.29310030233283,48.20915650097428,summer,6\n2019-08-25 00:00:00+00,77.61288383279704,44.377363307790176,summer,7\n2019-08-26 00:00:00+00,77.93968711541537,64.09496477893339,summer,1\n2019-08-27 00:00:00+00,76.60838173582529,67.09222617893592,summer,2\n2019-08-28 00:00:00+00,69.60379485369128,50.471342930379336,summer,3\n2019-08-29 00:00:00+00,74.03151108603846,53.812426528539405,summer,4\n2019-08-30 00:00:00+00,73.44832960843189,62.02791273959827,summer,5\n2019-08-31 00:00:00+00,79.01116103356198,52.82892947539808,summer,6\n2019-09-01 00:00:00+00,81.66810728281695,19.82892551360618,summer,7\n2019-09-02 00:00:00+00,77.5876801683695,22.97137346043969,summer,1\n2019-09-03 00:00:00+00,72.89078993763778,60.24035151754082,summer,2\n2019-09-04 00:00:00+00,75.8890310219594,48.836583061076155,summer,3\n2019-09-05 00:00:00+00,69.89871491998373,47.86994910765702,summer,4\n2019-09-06 00:00:00+00,70.49334415519309,50.192557090728236,summer,5\n2019-09-07 00:00:00+00,67.319505962889,53.26277960437409,summer,6\n2019-09-08 00:00:00+00,71.09050434859257,23.236372232718605,summer,7\n2019-09-09 00:00:00+00,70.25428560411913,45.82593148309299,summer,1\n2019-09-10 00:00:00+00,70.06464547343438,39.14526520000033,summer,2\n2019-09-11 00:00:00+00,73.15740745103979,39.489787251504154,summer,3\n2019-09-12 00:00:00+00,78.9861183637697,56.537825581623245,summer,4\n2019-09-13 00:00:00+00,79.37533927894798,58.55336413199284,summer,5\n2019-09-14 00:00:00+00,78.70676117179568,68.5194725335404,summer,6\n2019-09-15 00:00:00+00,69.99005738855075,25.783251837145972,summer,7\n2019-09-16 00:00:00+00,68.27556515434613,41.73064873456788,summer,1\n2019-09-17 00:00:00+00,66.32186255588499,40.541571854853395,summer,2\n2019-09-18 00:00:00+00,68.37909468457875,48.442877478650246,summer,3\n2019-09-19 00:00:00+00,69.34339336870303,53.03055308788898,summer,4\n2019-09-20 00:00:00+00,72.35838524162914,55.979668455374544,summer,5\n2019-09-21 00:00:00+00,72.24620636785096,48.59572532040283,summer,6\n2019-09-22 00:00:00+00,70.94571810612271,47.11399015286055,summer,7\n2019-09-23 00:00:00+00,75.09521002975683,40.39343754291672,summer,1\n2019-09-24 00:00:00+00,80.84413945588504,40.14575990508442,summer,2\n2019-09-25 00:00:00+00,83.27811255724183,33.129513709374145,summer,3\n2019-09-26 00:00:00+00,72.94960331732007,26.123436190056363,summer,4\n2019-09-27 00:00:00+00,63.23827243973101,27.818132707570697,summer,5\n2019-09-28 00:00:00+00,62.53492609284291,27.925459444484893,summer,6\n2019-09-29 00:00:00+00,60.60666366783909,13.414009277650582,summer,7\n2019-09-30 00:00:00+00,59.790575806430994,26.6700220541917,summer,1\n2019-10-01 00:00:00+00,58.52184540669101,35.33445742529893,shoulder,2\n2019-10-02 00:00:00+00,64.73270992142496,28.703187152673475,shoulder,3\n2019-10-03 00:00:00+00,62.682338762010104,36.33374722280265,shoulder,4\n2019-10-04 00:00:00+00,63.340257093929935,32.49916045608942,shoulder,5\n2019-10-05 00:00:00+00,65.78225614430846,27.14102165795462,shoulder,6\n2019-10-06 00:00:00+00,68.24037979969167,15.923655253741822,shoulder,7\n2019-10-07 00:00:00+00,72.98929724841025,35.33757847504879,shoulder,1\n2019-10-08 00:00:00+00,69.65378902902297,27.500399390805455,shoulder,2\n2019-10-09 00:00:00+00,64.17079273649955,23.813228882670394,shoulder,3\n2019-10-10 00:00:00+00,63.348027878599495,23.493598829621764,shoulder,4\n2019-10-11 00:00:00+00,64.543268565973,28.96788800700003,shoulder,5\n2019-10-12 00:00:00+00,63.70737322397421,26.927165870289127,shoulder,6\n2019-10-13 00:00:00+00,61.09162255996184,20.39739760809155,shoulder,7\n2019-10-14 00:00:00+00,58.76245147923707,35.35024037417232,shoulder,1\n2019-10-15 00:00:00+00,59.107874753931924,38.225546398300736,shoulder,2\n2019-10-16 00:00:00+00,59.184051482127764,61.156866730526374,shoulder,3\n2019-10-17 00:00:00+00,64.13165000212723,36.42331847404418,shoulder,4\n2019-10-18 00:00:00+00,58.33533462459896,26.26685926432833,shoulder,5\n2019-10-19 00:00:00+00,61.19439492883712,36.68358834112852,shoulder,6\n2019-10-20 00:00:00+00,63.01754502048632,22.787138480796447,shoulder,7\n2019-10-21 00:00:00+00,67.02595645935587,27.046608511045022,shoulder,1\n2019-10-22 00:00:00+00,69.08001663956308,28.08256916867112,shoulder,2\n2019-10-23 00:00:00+00,70.7708378392122,16.290706993648982,shoulder,3\n2019-10-24 00:00:00+00,75.15115212269926,34.08721962297556,shoulder,4\n2019-10-25 00:00:00+00,69.32846649823053,26.75918989695484,shoulder,5\n2019-10-26 00:00:00+00,66.34263862849042,29.220440817981952,shoulder,6\n2019-10-27 00:00:00+00,61.88270731323973,9.634858179989612,shoulder,7\n2019-10-28 00:00:00+00,57.574576893441815,34.09118885539274,shoulder,1\n2019-10-29 00:00:00+00,55.19063223255341,38.022099783934124,shoulder,2\n2019-10-30 00:00:00+00,56.859185157201935,32.56898878991925,shoulder,3\n2019-10-31 00:00:00+00,53.23455173551785,49.919612083048115,shoulder,4\n2019-11-01 00:00:00+00,54.568354991301405,55.89221979000794,winter,5\n2019-11-02 00:00:00+00,55.79003222219632,55.6120248607042,winter,6\n2019-11-03 00:00:00+00,58.29861013403849,21.101914077375493,winter,7\n2019-11-04 00:00:00+00,59.87569609967629,48.27599296606751,winter,1\n2019-11-05 00:00:00+00,60.321479073877065,41.64827580758599,winter,2\n2019-11-06 00:00:00+00,57.292968656186126,49.70497165239743,winter,3\n2019-11-07 00:00:00+00,55.59502176447923,33.470648732150785,winter,4\n2019-11-08 00:00:00+00,56.05826438837185,47.872245285837735,winter,5\n2019-11-09 00:00:00+00,57.024427339315885,43.53022801062667,winter,6\n2019-11-10 00:00:00+00,57.05959067516701,26.422668770803497,winter,7\n2019-11-11 00:00:00+00,61.618094912034955,64.30108171426097,winter,1\n2019-11-12 00:00:00+00,60.29453851571979,41.21558507486215,winter,2\n2019-11-13 00:00:00+00,57.456431444968565,49.982081304161156,winter,3\n2019-11-14 00:00:00+00,56.17898969896696,47.13379516122574,winter,4\n2019-11-15 00:00:00+00,60.84767002578624,50.86148613457842,winter,5\n2019-11-16 00:00:00+00,60.39473695647918,43.785392963964824,winter,6\n2019-11-17 00:00:00+00,56.306325891549626,16.505761886631106,winter,7\n2019-11-18 00:00:00+00,59.80206351113627,56.64216528681977,winter,1\n2019-11-19 00:00:00+00,58.73893888354822,57.27638158860387,winter,2\n2019-11-20 00:00:00+00,56.555538417765185,32.80137533117325,winter,3\n2019-11-21 00:00:00+00,53.75884931198261,46.78477409696745,winter,4\n2019-11-22 00:00:00+00,52.316918852237194,49.941065351120486,winter,5\n2019-11-23 00:00:00+00,54.095373592951375,59.008843320983786,winter,6\n2019-11-24 00:00:00+00,53.96136919928893,26.170648076593857,winter,7\n2019-11-25 00:00:00+00,50.62180366378275,61.43250138547132,winter,1\n2019-11-26 00:00:00+00,45.375711168975116,51.776230550567554,winter,2\n2019-11-27 00:00:00+00,45.75394466260938,54.47113633832736,winter,3\n2019-11-28 00:00:00+00,43.61376124279938,15.842122924655385,winter,4\n2019-11-29 00:00:00+00,44.50675238396271,63.140030144173224,winter,5\n2019-11-30 00:00:00+00,40.82836063140818,75.67217139429783,winter,6\n2019-12-01 00:00:00+00,51.56514809413482,25.12659699953764,winter,7\n2019-12-02 00:00:00+00,53.27661491025686,70.64066697081084,winter,1\n2019-12-03 00:00:00+00,54.85201607378994,67.69440437473412,winter,2\n2019-12-04 00:00:00+00,52.96446712533269,75.64553988219753,winter,3\n2019-12-05 00:00:00+00,53.74901996091119,81.95970508143486,winter,4\n2019-12-06 00:00:00+00,56.907780070199514,60.247235387208626,winter,5\n2019-12-07 00:00:00+00,58.856434999839465,63.38135045289999,winter,6\n2019-12-08 00:00:00+00,56.123522137104366,24.963483394077542,winter,7\n2019-12-09 00:00:00+00,52.48230930949152,64.21405633949757,winter,1\n2019-12-10 00:00:00+00,54.48261999125536,61.356079727312355,winter,2\n2019-12-11 00:00:00+00,54.97032831463157,24.393852716325213,winter,3\n2019-12-12 00:00:00+00,58.57111749741894,68.62790486149407,winter,4\n2019-12-13 00:00:00+00,57.26636464119373,75.38274271536724,winter,5\n2019-12-14 00:00:00+00,53.3083944185529,58.431954865646134,winter,6\n2019-12-15 00:00:00+00,48.83970871025506,36.428474552935256,winter,7\n2019-12-16 00:00:00+00,48.574670706228325,79.27788464938769,winter,1\n2019-12-17 00:00:00+00,45.32622235990349,99.12992231278221,winter,2\n2019-12-18 00:00:00+00,50.80341958877182,82.33792576293904,winter,3\n2019-12-19 00:00:00+00,52.783744083992296,78.20096439284458,winter,4\n2019-12-20 00:00:00+00,47.822793638536,99.29426728590605,winter,5\n2019-12-21 00:00:00+00,49.00553318913267,91.9506643765134,winter,6\n2019-12-22 00:00:00+00,48.82502808435691,40.985755703395945,winter,7\n2019-12-23 00:00:00+00,44.03812472981052,53.713874533431266,winter,1\n2019-12-24 00:00:00+00,48.79872415163294,38.1108079630476,winter,2\n2019-12-25 00:00:00+00,49.3232383132318,36.01347776610808,winter,3\n2019-12-26 00:00:00+00,51.77208368228864,95.25655815001174,winter,4\n2019-12-27 00:00:00+00,45.34209432596163,93.74548610766941,winter,5\n2019-12-28 00:00:00+00,47.106488077599714,64.19483775941112,winter,6\n2019-12-29 00:00:00+00,47.790951773560614,26.906355894296986,winter,7\n2019-12-30 00:00:00+00,53.581811399864414,48.837465319612555,winter,1\n2019-12-31 00:00:00+00,48.833966913788444,35.85289877460335,winter,2\n2020-01-01 00:00:00+00,53.68818137849708,-8.949622275030016,winter,3\n2020-01-02 00:00:00+00,52.632377682508334,-0.7946768912093022,winter,4\n2020-01-03 00:00:00+00,48.192799739862586,24.0256080236098,winter,5\n2020-01-04 00:00:00+00,50.36340063474181,70.93628525779403,winter,6\n2020-01-05 00:00:00+00,51.00752359358262,29.56889103105214,winter,7\n2020-01-06 00:00:00+00,47.43727062688896,79.661231698029,winter,1\n2020-01-07 00:00:00+00,44.820785325726305,82.09707696183041,winter,2\n2020-01-08 00:00:00+00,50.01523139182983,86.63266732807747,winter,3\n2020-01-09 00:00:00+00,50.08118300429677,88.27105778062221,winter,4\n2020-01-10 00:00:00+00,47.34992119524401,81.36044966525102,winter,5\n2020-01-11 00:00:00+00,51.41521026435299,73.35413219022368,winter,6\n2020-01-12 00:00:00+00,45.35434231814758,23.741660738406654,winter,7\n2020-01-13 00:00:00+00,50.65757304424703,71.54069395966269,winter,1\n2020-01-14 00:00:00+00,50.050806841635,90.866449869571,winter,2\n2020-01-15 00:00:00+00,43.30165769367253,97.73216159711114,winter,3\n2020-01-16 00:00:00+00,48.93041920467607,87.09670428640662,winter,4\n2020-01-17 00:00:00+00,44.14965647025005,83.95097462845436,winter,5\n2020-01-18 00:00:00+00,48.533606073772,83.49873875061449,winter,6\n2020-01-19 00:00:00+00,45.84359598326136,41.39709387640873,winter,7\n2020-01-20 00:00:00+00,47.209736291948516,61.38906421553486,winter,1\n2020-01-21 00:00:00+00,53.476651745118026,68.25357870800845,winter,2\n2020-01-22 00:00:00+00,54.05805426777448,92.44089963364891,winter,3\n2020-01-23 00:00:00+00,50.92994106041078,88.67958119602659,winter,4\n2020-01-24 00:00:00+00,54.491320185312226,60.418759332602484,winter,5\n2020-01-25 00:00:00+00,57.73805766338176,23.554693531912392,winter,6\n2020-01-26 00:00:00+00,57.30881816473342,30.22651296790617,winter,7\n2020-01-27 00:00:00+00,50.44771019942605,56.75934497550486,winter,1\n2020-01-28 00:00:00+00,54.11790780836803,73.3767445166123,winter,2\n2020-01-29 00:00:00+00,52.15515059374105,58.06410668574979,winter,3\n2020-01-30 00:00:00+00,55.67746310710462,83.32269199962236,winter,4\n2020-01-31 00:00:00+00,56.27866394530186,72.34850438718759,winter,5\n2020-02-01 00:00:00+00,51.20260707944656,51.34470175016527,winter,6\n2020-02-02 00:00:00+00,52.429369479376156,40.92857485482883,winter,7\n2020-02-03 00:00:00+00,45.031919433380814,25.495783413683114,winter,1\n2020-02-04 00:00:00+00,46.521142187421006,68.18923581592101,winter,2\n2020-02-05 00:00:00+00,49.72557550177604,79.18498520902662,winter,3\n2020-02-06 00:00:00+00,50.33126273544205,66.96157238629866,winter,4\n2020-02-07 00:00:00+00,51.16954930087094,68.98340286288293,winter,5\n2020-02-08 00:00:00+00,49.32849570430554,69.33025424224415,winter,6\n2020-02-09 00:00:00+00,53.26165265287247,29.717901636215288,winter,7\n2020-02-10 00:00:00+00,55.258757418008884,61.911859925598755,winter,1\n2020-02-11 00:00:00+00,61.84578247351189,56.11444216095445,winter,2\n2020-02-12 00:00:00+00,55.15724054411032,48.49283907298848,winter,3\n2020-02-13 00:00:00+00,49.500758034984614,69.5463066429468,winter,4\n2020-02-14 00:00:00+00,51.70369392924528,74.79977417528724,winter,5\n2020-02-15 00:00:00+00,54.103412053226386,70.16221039633545,winter,6\n2020-02-16 00:00:00+00,55.093138138481656,53.111725786323625,winter,7\n2020-02-17 00:00:00+00,56.194080772571795,62.269864316263295,winter,1\n2020-02-18 00:00:00+00,55.818265838700384,72.90018884010671,winter,2\n2020-02-19 00:00:00+00,54.2009657758995,77.22801604346152,winter,3\n2020-02-20 00:00:00+00,54.88514871870705,66.66981622280281,winter,4\n2020-02-21 00:00:00+00,57.281077167676095,78.66583755220967,winter,5\n2020-02-22 00:00:00+00,54.77944866731585,71.04918843528911,winter,6\n2020-02-23 00:00:00+00,55.95897473566892,22.487865434756703,winter,7\n2020-02-24 00:00:00+00,54.99123375091521,65.25520777511282,winter,1\n2020-02-25 00:00:00+00,59.38642655126644,56.389535397618324,winter,2\n2020-02-26 00:00:00+00,58.43486564664218,65.15903041277639,winter,3\n2020-02-27 00:00:00+00,60.5344610285767,59.51806610072933,winter,4\n2020-02-28 00:00:00+00,60.61752631874749,50.39931375823029,winter,5\n2020-02-29 00:00:00+00,56.433490715689345,55.136581244844336,winter,6"
  },
  {
    "path": "tests/eemeter/daily_model/test_fit_base_models.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients\n\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings as Settings\nfrom opendsm.eemeter.models.daily.parameters import ModelType\nfrom opendsm.eemeter.models.daily.fit_base_models import (\n    fit_initial_models_from_full_model,\n    fit_model,\n    fit_final_model,\n    _get_opt_settings,\n)\n\nfrom opendsm.eemeter.models.daily.optimize_results import OptimizedResult\n\n\n@pytest.fixture\ndef meter_data():\n    df_meter = pd.DataFrame(\n        {\n            \"temperature\": [\n                10.0,\n                15.0,\n                20.0,\n                25.0,\n                30.0,\n                35.0,\n                40.0,\n                45.0,\n                50.0,\n                55.0,\n                60.0,\n                65.0,\n                70.0,\n                75.0,\n                80.0,\n                85.0,\n                90.0,\n                95.0,\n                100.0,\n            ],\n            \"observed\": [\n                100.0,\n                150.0,\n                200.0,\n                250.0,\n                300.0,\n                350.0,\n                400.0,\n                450.0,\n                500.0,\n                550.0,\n                600.0,\n                650.0,\n                700.0,\n                750.0,\n                800.0,\n                850.0,\n                900.0,\n                950.0,\n                1000.0,\n            ],\n            \"datetime\": [\n                \"2022-01-01\",\n                \"2022-01-02\",\n                \"2022-01-03\",\n                \"2022-01-04\",\n                \"2022-01-05\",\n                \"2022-01-06\",\n                \"2022-01-07\",\n                \"2022-01-08\",\n                \"2022-01-09\",\n                \"2022-01-10\",\n                \"2022-01-11\",\n                \"2022-01-12\",\n                \"2022-01-13\",\n                \"2022-01-14\",\n                \"2022-01-15\",\n                \"2022-01-16\",\n                \"2022-01-17\",\n                \"2022-01-18\",\n                \"2022-01-19\",\n            ],\n        }\n    )\n    df_meter[\"datetime\"] = pd.to_datetime(df_meter[\"datetime\"])\n    df_meter.set_index(\"datetime\", inplace=True)\n    return df_meter\n\n\n@pytest.fixture\ndef get_settings():\n    return Settings()\n\n\n@pytest.fixture\ndef get_optimized_result(get_settings):\n    return OptimizedResult(\n        x=np.array([1, 2, 3, 4]),\n        bnds=[\n            [0, 1],\n            [0, 2],\n            [0, 3],\n            [0, 4],\n            [0, 5],\n            [0, 6],\n            [0, 7],\n            [0, 8],\n            [0, 9],\n            [0, 10],\n        ],\n        coef_id=[\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"],\n        loss_alpha=0.1,\n        C=0.5,\n        T=np.array([10, 20, 30, 40, 50, 60, 70, 80, 90, 100]),\n        model=np.array([2, 4, 6, 8, 10, 12, 14, 16, 18, 20]),\n        weight=np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]),\n        resid=np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]),\n        jac=None,\n        mean_loss=0.2,\n        TSS=0.3,\n        success=True,\n        message=\"Optimization successful.\",\n        nfev=10,\n        time_elapsed=0.5,\n        settings=get_settings,\n    )\n\n\ndef test_fit_initial_models_from_full_model(meter_data, get_settings):\n    # Test case 1: Test the function with a sample dataset\n    model_res = fit_initial_models_from_full_model(meter_data, get_settings)\n    assert isinstance(model_res, OptimizedResult)\n\n    # Test case 3: Test the function with a dataset that has missing values\n    T = np.array([10, 20, 30, 40, 50])\n    obs = np.array([1, 2, np.nan, 4, 5])\n    model_res = fit_initial_models_from_full_model(meter_data, get_settings)\n    assert isinstance(model_res, OptimizedResult)\n\n    # Test case 4: Test the function with a dataset that has negative values\n    T = np.array([10, 20, 30, 40, 50])\n    obs = np.array([-1, 2, 3, 4, 5])\n    model_res = fit_initial_models_from_full_model(meter_data, get_settings)\n    assert isinstance(model_res, OptimizedResult)\n\n\ndef test_fit_model(meter_data, get_settings):\n    # Test case 1: Test for model_key = \"hdd_tidd_cdd_smooth\"\n    T = np.array(meter_data[\"temperature\"])\n    obs = np.array(meter_data[\"observed\"])\n    weights = None\n\n    fit_input = [T, obs, weights, get_settings, _get_opt_settings(get_settings)]\n    res = fit_model(\"hdd_tidd_cdd_smooth\", fit_input, None, None)\n    assert isinstance(res, OptimizedResult)\n\n    # Test case 2: Test for model_key = \"hdd_tidd_cdd\"\n    res = fit_model(\"hdd_tidd_cdd\", fit_input, None, None)\n    assert isinstance(res, OptimizedResult)\n\n    # Test case 3: Test for model_key = \"c_hdd_tidd_smooth\"\n    res = fit_model(\"c_hdd_tidd_smooth\", fit_input, None, None)\n    assert isinstance(res, OptimizedResult)\n\n    # Test case 4: Test for model_key = \"c_hdd_tidd\"\n    x0 = ModelCoefficients(\n        model_type=ModelType.HDD_TIDD,\n        cdd_beta=1.747475624458497,\n        cdd_bp=74.69216148926878,\n        cdd_k=0.2548934690459498,\n        hdd_beta=1.308196391571347,\n        hdd_bp=63.332029669746596,\n        hdd_k=0.0,\n        intercept=49.97929032502437,\n    )\n    res = fit_model(\"c_hdd_tidd\", fit_input, x0, None)\n    assert isinstance(res, OptimizedResult)\n\n    # Test case 5: Test for model_key = \"tidd\"\n    res = fit_model(\"tidd\", fit_input, None, None)\n    assert isinstance(res, OptimizedResult)\n\n    # TODO: add more data specific cases\n\n\ndef test_fit_final_model(meter_data, get_settings, get_optimized_result):\n    # Test case 1: Test if the function returns an instance of OptimizedResult\n    HoF = get_optimized_result\n\n    res = fit_final_model(meter_data, HoF, get_settings)\n    assert isinstance(res, OptimizedResult)\n\n    # Test case 2: Test if the function raises a TypeError when the input arguments are of the wrong type\n    with pytest.raises(TypeError):\n        fit_final_model(\"not a dataframe\", \"not an OptimizedResult\", \"not a dictionary\")\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_fit_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nfrom opendsm.eemeter.models.daily.model import DailyModel\nfrom opendsm.eemeter.models.daily.data import DailyBaselineData\nfrom opendsm.eemeter.models.daily.optimize_results import OptimizedResult\n\n\n# Define the current directory\ncurrent_dir = Path(__file__).resolve().parent\n\n\nclass TestFitModel:\n    @classmethod\n    def setup_class(cls):\n        # Create a sample meter data DataFrame from the test data\n        df = pd.read_csv(current_dir / \"test_data.csv\")\n        df.index = pd.to_datetime(df[\"datetime\"])\n        df = df[[\"temperature\", \"observed\"]]\n        cls.meter_data = DailyBaselineData(df, is_electricity_data=True)\n\n    def test_fit_model(self):\n        # Create a DailyModel instance\n        fm = DailyModel().fit(self.meter_data, ignore_disqualification=True)\n\n        # Test that the combinations attribute is a list\n        assert isinstance(fm.combinations, list)\n\n        # Test that the combinations attribute is as expected\n        expected_combinations = [\n            \"fw-su_sh_wi\",\n            \"fw-sh_wi__fw-su\",\n            \"fw-sh_wi__wd-su__we-su\",\n            \"wd-su_sh_wi__we-su_sh_wi\",\n            \"fw-su__wd-sh_wi__we-sh_wi\",\n            \"wd-su__wd-sh_wi__we-su_sh_wi\",\n            \"wd-su_sh_wi__we-su__we-sh_wi\",\n            \"wd-su__wd-sh_wi__we-su__we-sh_wi\",\n        ]\n        assert fm.combinations == expected_combinations\n\n        # Test that the components attribute is a list\n        assert isinstance(fm.components, list)\n\n        # Test that the components attribute is as expected\n        expected_components = [\n            \"fw-su\",\n            \"wd-su\",\n            \"we-su\",\n            \"fw-sh_wi\",\n            \"wd-sh_wi\",\n            \"we-sh_wi\",\n            \"fw-su_sh_wi\",\n            \"wd-su_sh_wi\",\n            \"we-su_sh_wi\",\n        ]\n        assert fm.components == expected_components\n\n        # Test that the fit_components attribute is a dictionary\n        assert isinstance(fm.fit_components, dict)\n\n        # Test that the wRMSE_base attribute is a float\n        assert isinstance(fm.wRMSE_base, float)\n        assert np.isclose(fm.wRMSE_base, 18.39, rtol=1e-2)\n\n        # Test that the best combination is as expected\n        expected_best_combination = \"wd-su_sh_wi__we-su_sh_wi\"\n        assert fm.best_combination == expected_best_combination\n\n        # Test that the final model is as expected\n        combinations = expected_best_combination.split(\"__\")\n        for combination in combinations:\n            assert isinstance(fm.model[combination], OptimizedResult)\n\n        # Test that the error attribute values are as expected\n        expected_model_error = {\n            \"wrmse\": 16.96,\n            \"rmse\": 16.96,\n            \"mae\": 13.40,\n            \"cvrmse\": 0.3207,\n            \"pnrmse\": 0.6326,\n        }\n        for k in expected_model_error:\n            assert np.isclose(getattr(fm.baseline_metrics, k), expected_model_error[k], rtol=1e-2)\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_objective_function.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport pytest\n\nfrom opendsm.eemeter.models.daily.objective_function import (\n    get_idx,\n    no_weights_obj_fcn,\n    obj_fcn_decorator,\n)\n\nfrom opendsm.eemeter.models.daily.base_models.hdd_tidd_cdd import (\n    evaluate_hdd_tidd_cdd_smooth,\n    _hdd_tidd_cdd_smooth_weight,\n)\n\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings as Settings\n\n\ndef test_get_idx():\n    # Test case 1: Test with empty lists\n    A = []\n    B = []\n    assert get_idx(A, B) == []\n\n    # Test case 2: Test with one empty list\n    A = [\"a\", \"b\", \"c\"]\n    B = []\n    assert get_idx(A, B) == []\n\n    # Test case 3: Test with one non-empty list\n    A = [\"a\", \"b\", \"c\"]\n    B = [\"a\", \"b\", \"c\", \"d\", \"e\"]\n    assert get_idx(A, B) == [0, 1, 2]\n\n    # Test case 4: Test with two non-empty lists\n    A = [\"a\", \"b\", \"c\"]\n    B = [\"a1\", \"b2\", \"c3\", \"d4\", \"e5\"]\n    assert get_idx(A, B) == [0, 1, 2]\n\n    # Test case 5: Test with two non-empty lists with duplicates\n    A = [\"a\", \"b\", \"c\"]\n    B = [\"a1\", \"b2\", \"c3\", \"a4\", \"e5\"]\n    assert get_idx(A, B) == [0, 1, 2, 3]\n\n\ndef test_no_weights_obj_fcn():\n    # Test case 1: Test with X, obs and idx_bp as None, should raise an error\n    X = None\n    obs = None\n    idx_bp = None\n    model_fcn = lambda x: x\n    aux_inputs = (model_fcn, obs, idx_bp)\n    with pytest.raises(TypeError):\n        no_weights_obj_fcn(X, aux_inputs)\n\n    # Test case 2: Test with X, obs and idx_bp as empty arrays\n    X = np.array([])\n    obs = np.array([])\n    idx_bp = np.array([])\n    model_fcn = lambda x: x\n    aux_inputs = (model_fcn, obs, idx_bp)\n    assert no_weights_obj_fcn(X, aux_inputs) == 0\n\n    # Test case 3: Test with X, obs and idx_bp as non-empty arrays\n    X = np.array([1, 2, 3])\n    obs = np.array([2, 4, 6])\n    idx_bp = np.array([0, 2])\n    model_fcn = lambda x: x * 2\n    aux_inputs = (model_fcn, obs, idx_bp)\n    assert no_weights_obj_fcn(X, aux_inputs) == 0\n\n    # Test case 4: Test with X, obs and idx_bp as non-empty arrays with negative values\n    X = np.array([-1, -2, -3])\n    obs = np.array([-2, -4, -6])\n    idx_bp = np.array([0, 2])\n    model_fcn = lambda x: x * -2\n    aux_inputs = (model_fcn, obs, idx_bp)\n    assert no_weights_obj_fcn(X, aux_inputs) == 192\n\n\n@pytest.fixture\ndef obj_fcn_test(self):\n    # Create an instance of obj_fcn_decorator for testing\n\n    # Define inputs\n    T = np.array([1, 2, 3, 4, 5])\n    obs = np.array([1, 2, 3, 4, 5])\n    settings = Settings()\n    coef_id = [\"dd_k\", \"dd_beta\", \"dd_bp\"]\n    return obj_fcn_decorator(\n        lambda x: x,\n        lambda x: x,\n        lambda x: x,\n        T,\n        obs,\n        settings,\n        alpha=2.0,\n        coef_id=coef_id,\n        initial_fit=True,\n    )\n\n\ndef test_obj_fcn_decorator():\n    # Test case 1: Test with minimum input values\n    model_fcn_full = evaluate_hdd_tidd_cdd_smooth\n    weight_fcn = _hdd_tidd_cdd_smooth_weight\n    TSS_fcn = None\n    T = np.array([1, 2, 3]).astype(float)\n    obs = np.array([4, 5, 6]).astype(float)\n    weights = None\n    settings = type(\n        \"Settings\",\n        (object,),\n        {\n            \"segment_minimum_count\": 1,\n            \"regularization_percent_lasso\": 0.5,\n            \"regularization_alpha\": 0.1,\n        },\n    )()\n    alpha = 2.0\n    coef_id = [\"dd_k\", \"dd_beta\", \"dd_bp\"]\n    initial_fit = True\n    obj_fcn = obj_fcn_decorator(\n        model_fcn_full,\n        weight_fcn,\n        TSS_fcn,\n        T,\n        obs,\n        weights,\n        settings,\n        alpha,\n        coef_id,\n        initial_fit,\n    )\n    assert (\n        obj_fcn(\n            [73.48349431, 0.0, 0.0, 81.66823342, 5.34878023, 0.0, 50.85274713],\n            [model_fcn_full, obs, [0, 1]],\n        )\n        == 2105.4340868444824\n    )\n\n    # Test case 2: Test with initial fit set as False\n    model_fcn_full = evaluate_hdd_tidd_cdd_smooth\n    weight_fcn = _hdd_tidd_cdd_smooth_weight\n    TSS_fcn = None\n    T = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).astype(float)\n    obs = np.array([11, 12, 13, 14, 15, 16, 17, 18, 19, 20]).astype(float)\n    settings = type(\n        \"Settings\",\n        (object,),\n        {\n            \"segment_minimum_count\": 2,\n            \"regularization_percent_lasso\": 0.2,\n            \"regularization_alpha\": 0.5,\n        },\n    )()\n    alpha = 3.0\n    coef_id = [\"dd_k\", \"dd_beta\"]\n    initial_fit = False\n    obj_fcn = obj_fcn_decorator(\n        model_fcn_full,\n        weight_fcn,\n        TSS_fcn,\n        T,\n        obs,\n        weights,\n        settings,\n        alpha,\n        coef_id,\n        initial_fit,\n    )\n    assert (\n        obj_fcn(\n            [1.5, 0.0, 0.0, 85.5, 10.254, 0.0, 50.85], [model_fcn_full, obs, [0, 1]]\n        )\n        == 1295.1226641177577\n    )\n"
  },
  {
    "path": "tests/eemeter/daily_model/test_optimize.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport pytest\nfrom opendsm.eemeter.models.daily.optimize import obj_fcn_dec, Optimizer\n\nfrom opendsm.eemeter.models.daily.objective_function import obj_fcn_decorator\n\nfrom opendsm.eemeter.models.daily.base_models.hdd_tidd_cdd import (\n    evaluate_hdd_tidd_cdd_smooth,\n    _hdd_tidd_cdd_smooth_weight,\n)\n\nfrom opendsm.eemeter.models.daily.fit_base_models import _get_opt_settings\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings as Settings\n\n\ndef test_obj_fcn_dec():\n    # Test case 1: Check if the function returns the expected output for valid input\n    def obj_fcn(x):\n        return np.sum(x**2)\n\n    x0 = np.array([1, 2, 3])\n    bnds = np.array([[0, 1], [1, 2], [2, 3]])\n    obj_fcn_eval, idx_opt = obj_fcn_dec(obj_fcn, x0, bnds)\n\n    x = np.array([0.5, 1.5, 2.5])\n    expected_output = 5\n    assert obj_fcn_eval(x) == expected_output\n    assert idx_opt == [0, 1, 2]\n\n\ndef get_obj_fcn(settings):\n    # Test case 1: Test with minimum input values\n    model_fcn_full = evaluate_hdd_tidd_cdd_smooth\n    weight_fcn = _hdd_tidd_cdd_smooth_weight\n    TSS_fcn = None\n    T = np.array([1, 2, 3, 4, 5, 6, 7]).astype(float)\n    obs = np.array([2, 4, 6, 8, 10, 12, 14]).astype(float)\n    base_weights = None\n    alpha = 2.0\n    coef_id = [\n        \"hdd_bp\",\n        \"hdd_beta\",\n        \"hdd_k\",\n        \"cdd_bp\",\n        \"cdd_beta\",\n        \"cdd_k\",\n        \"intercept\",\n    ]\n    initial_fit = True\n    return obj_fcn_decorator(\n        model_fcn_full,\n        weight_fcn,\n        TSS_fcn,\n        T,\n        obs,\n        base_weights,\n        settings,\n        alpha,\n        coef_id,\n        initial_fit,\n    )\n\n\n@pytest.fixture\ndef get_x0():\n    return np.array([73.48349431, 0.0, 0.0, 81.66823342, 5.34878023, 0.0, 50.85274713])\n\n\n@pytest.fixture\ndef get_bnds():\n    return np.array(\n        [\n            [59.79057581, 86.91811908],\n            [0.0, 16.0463407],\n            [0.0, 1.0],\n            [59.79057581, 86.91811908],\n            [0.0, 16.0463407],\n            [0.0, 1.0],\n            [20.13393783, 79.33486982],\n        ]\n    )\n\n\ndef test_optimizer_run(get_x0, get_bnds):\n    x0 = get_x0\n    bnds = get_bnds\n    coef_id = [\n        \"hdd_bp\",\n        \"hdd_beta\",\n        \"hdd_k\",\n        \"cdd_bp\",\n        \"cdd_beta\",\n        \"cdd_k\",\n        \"intercept\",\n    ]\n\n    # Test case 1: Test with empty options\n    settings = Settings()\n    opt_settings = _get_opt_settings(settings)\n    obj_fcn = get_obj_fcn(settings)\n    optimizer = Optimizer(obj_fcn, x0, bnds, coef_id, settings, opt_settings)\n    res = optimizer.run()\n    assert np.allclose(res.x, np.array([20.13393]), rtol=1e-5, atol=1e-5)\n\n    # Test case 2: Test with scipy algorithm\n    settings = Settings(developer_mode=True, algorithm_choice=\"scipy_Nelder-Mead\")\n    opt_settings = _get_opt_settings(settings)\n    obj_fcn = get_obj_fcn(settings)\n    optimizer = Optimizer(obj_fcn, x0, bnds, coef_id, settings, opt_settings)\n    res = optimizer.run()\n    assert np.allclose(res.x, np.array([20.13393]), rtol=1e-5, atol=1e-5)\n\n    # Test case 3: Test with nlopt algorithm\n    settings = Settings(developer_mode=True, algorithm_choice=\"nlopt_sbplx\")\n    opt_settings = _get_opt_settings(settings)\n    obj_fcn = get_obj_fcn(settings)\n    optimizer = Optimizer(obj_fcn, x0, bnds, coef_id, settings, opt_settings)\n    res = optimizer.run()\n    assert np.allclose(res.x, np.array([20.13393]), rtol=1e-5, atol=1e-5)"
  },
  {
    "path": "tests/eemeter/daily_model/test_optimize_results.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport pytest\nimport numpy as np\n\nfrom opendsm.eemeter.models.daily.optimize_results import (\n    get_k,\n    reduce_model,\n    OptimizedResult,\n)\n\nfrom opendsm.eemeter.models.daily.parameters import ModelCoefficients\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings as Settings\n\n\ndef test_get_k():\n    # Test case 1: Test when all values are within the bounds\n    X = [60, 0.5, 80, 0.3]\n    T_min_seg = 50\n    T_max_seg = 90\n    assert get_k(X, T_min_seg, T_max_seg) == [70.0, 10.0, 74.0, 6.0]\n\n    # Test case 2: Test when hdd_bp is greater than T_max_seg\n    X = [100, 0.5, 80, 0.3]\n    T_min_seg = 50\n    T_max_seg = 90\n    assert get_k(X, T_min_seg, T_max_seg) == [100, 0.0, 86.0, -6.0]\n\n    # Test case 3: Test when cdd_bp is less than T_min_seg\n    X = [60, 0.5, 20, 0.3]\n    T_min_seg = 50\n    T_max_seg = 90\n    assert get_k(X, T_min_seg, T_max_seg) == [40.0, -20.0, 20, 0.0]\n\n    # Test case 4: Test when both hdd_k and cdd_k are zero\n    X = [100, 0.5, 20, 0.3]\n    T_min_seg = 50\n    T_max_seg = 90\n    assert get_k(X, T_min_seg, T_max_seg) == [20, 0.0, 20, 0.0]\n\n\n@pytest.mark.parametrize(\n    \"hdd_bp, hdd_beta, pct_hdd_k, cdd_bp, cdd_beta, pct_cdd_k, intercept, T_min, T_max, T_min_seg, T_max_seg, model_key, expected_coef_id, expected_x\",\n    [\n        # Test case 1\n        (\n            10,\n            20,\n            30,\n            40,\n            50,\n            60,\n            70,\n            0,\n            100,\n            20,\n            80,\n            \"hdd_tidd_cdd_smooth\",\n            [\"hdd_bp\", \"hdd_beta\", \"hdd_k\", \"cdd_bp\", \"cdd_beta\", \"cdd_k\", \"intercept\"],\n            [10, 20, 30, 40, 50, 60, 70],\n        ),\n        # Test case 2\n        (\n            10,\n            20,\n            0,\n            40,\n            50,\n            60,\n            70,\n            0,\n            100,\n            20,\n            80,\n            \"hdd_tidd_cdd_smooth\",\n            [\"hdd_bp\", \"hdd_beta\", \"hdd_k\", \"cdd_bp\", \"cdd_beta\", \"cdd_k\", \"intercept\"],\n            [10, 20, 0, 40, 50, 60, 70],\n        ),\n        # Test case 3\n        (\n            10,\n            0,\n            30,\n            40,\n            50,\n            60,\n            70,\n            0,\n            100,\n            20,\n            80,\n            \"hdd_tidd_cdd_smooth\",\n            [\"c_hdd_bp\", \"c_hdd_beta\", \"c_hdd_k\", \"intercept\"],\n            [20.0, 50.0, 20.0, 70.0],\n        ),\n        # Test case 4\n        (\n            10,\n            20,\n            0,\n            40,\n            50,\n            0,\n            70,\n            0,\n            100,\n            20,\n            80,\n            \"hdd_tidd_cdd_smooth\",\n            [\"hdd_bp\", \"hdd_beta\", \"cdd_bp\", \"cdd_beta\", \"intercept\"],\n            [10, 20, 40, 50, 70],\n        ),\n        # Test case 5\n        (\n            10,\n            0,\n            0,\n            40,\n            0,\n            0,\n            70,\n            0,\n            100,\n            20,\n            80,\n            \"hdd_tidd_cdd_smooth\",\n            [\"intercept\"],\n            [70],\n        ),\n    ],\n)\ndef test_reduce_model(\n    hdd_bp,\n    hdd_beta,\n    pct_hdd_k,\n    cdd_bp,\n    cdd_beta,\n    pct_cdd_k,\n    intercept,\n    T_min,\n    T_max,\n    T_min_seg,\n    T_max_seg,\n    model_key,\n    expected_coef_id,\n    expected_x,\n):\n    coef_id, x = reduce_model(\n        hdd_bp,\n        hdd_beta,\n        pct_hdd_k,\n        cdd_bp,\n        cdd_beta,\n        pct_cdd_k,\n        intercept,\n        T_min,\n        T_max,\n        T_min_seg,\n        T_max_seg,\n        model_key,\n    )\n\n    assert coef_id == expected_coef_id\n    assert np.allclose(x, expected_x)\n\n\nclass TestOptimizeResult:\n    @pytest.fixture\n    def optimize_result(self):\n        # create an instance of OptimizeResult for testing\n        x = np.array([1, 2, 3, 4, 5, 6, 7])\n        bnds = [[0, 10] * 7]\n        coef_id = [\n            \"hdd_bp\",\n            \"hdd_beta\",\n            \"hdd_k\",\n            \"cdd_bp\",\n            \"cdd_beta\",\n            \"cdd_k\",\n            \"intercept\",\n        ]\n        loss_alpha = 2.0\n        C = np.array([1, 2, 3, 4, 5, 6, 7] * 2)\n        T = np.array([1, 2, 3, 4, 5, 6, 7] * 2)\n        model = np.array([1, 2, 3, 4, 5, 6, 7])\n        resid = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7])\n        weight = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7])\n        settings = Settings()\n        jac = None\n        mean_loss = 0.0\n        TSS = 0.0\n        success = True\n        message = \"Optimization terminated successfully.\"\n        nfev = 10\n        time_elapsed = 1.0\n        return OptimizedResult(\n            x,\n            bnds,\n            coef_id,\n            loss_alpha,\n            T,\n            C,\n            model,\n            weight,\n            resid,\n            jac,\n            mean_loss,\n            TSS,\n            success,\n            message,\n            nfev,\n            time_elapsed,\n            settings,\n        )\n\n    def test_named_coeffs(self, optimize_result):\n        # test that named_coeffs is an instance of ModelCoefficients\n        assert isinstance(optimize_result.named_coeffs, ModelCoefficients)\n\n    def test_prediction_uncertainty(self, optimize_result):\n        # test that _prediction_uncertainty sets f_unc correctly\n        optimize_result._prediction_uncertainty()\n        assert optimize_result.f_unc == pytest.approx(2.1556496051287013)\n\n    def test_set_model_key(self, optimize_result):\n        # test that _set_model_key sets model_key and model_name correctly\n        optimize_result._set_model_key()\n        assert optimize_result.model_key == \"c_hdd_tidd\"\n        assert optimize_result.model_name == \"cdd_tidd\"\n\n    def test_refine_model(self, optimize_result):\n        # test that _refine_model sets coef_id and x correctly\n        optimize_result._refine_model()\n        assert optimize_result.coef_id == [\"c_hdd_bp\", \"c_hdd_beta\", \"intercept\"]\n        assert optimize_result.x == pytest.approx(np.array([4, 5, 7]))\n\n    def test_eval(self, optimize_result):\n        # test that eval returns the correct values\n        T = np.array([1, 2, 3, 4, 5])\n        model, f_unc, hdd_load, cdd_load = optimize_result.eval(T)\n        assert model == pytest.approx(np.array([7, 7, 7, 7, 12]))\n        assert f_unc == pytest.approx(\n            np.array([2.15564961, 2.15564961, 2.15564961, 2.15564961, 2.15564961])\n        )\n        assert hdd_load == pytest.approx(np.array([0, 0, 0, 0, 0]))\n        assert cdd_load == pytest.approx(np.array([0, 0, 0, 0, 5]))\n"
  },
  {
    "path": "tests/eemeter/daily_model/utilities/test_adaptive_loss.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\n\nfrom opendsm.common.stats.adaptive_loss import (\n    remove_outliers,\n    adaptive_weights,\n    adaptive_loss_fcn,\n)\n\n\ndef test_remove_outliers():\n    # Test case 1: No outliers\n    data = np.array([1, 2, 3, 4, 5])\n    data_no_outliers, idx_no_outliers = remove_outliers(data)\n    assert np.array_equal(data, data_no_outliers)\n    assert np.array_equal(idx_no_outliers, np.arange(len(data)))\n\n    # Test case 2: Outliers present\n    data = np.array([1, 2, 3, 4, 5, 100])\n    data_no_outliers, idx_no_outliers = remove_outliers(data)\n    assert np.array_equal(data_no_outliers, np.array([1, 2, 3, 4, 5]))\n    assert np.array_equal(idx_no_outliers, np.arange(len(data) - 1))\n\n    # Test case 3: Weights provided\n    data = np.array([1.0, 2, 3, 4, 5, 100])\n    weights = np.array([1, 1, 1, 1, 1, 0.1])\n    data_no_outliers, idx_no_outliers = remove_outliers(data, weights)\n    assert np.array_equal(data_no_outliers, np.array([1, 2, 3, 4, 5]))\n    assert np.array_equal(idx_no_outliers, np.arange(len(data) - 1))\n\n\ndef test_adaptive_loss_fcn():\n    # Test case 1: Test with all finite values\n    x = np.array([1, 2, 3, 4, 5])\n    loss_fcn_val, loss_alpha = adaptive_loss_fcn(x)\n    assert np.isfinite(loss_fcn_val)\n    assert np.isfinite(loss_alpha)\n\n    # Test case 3: Test with zero values\n    x = np.array([0, 0, 0, 0, 0])\n    loss_fcn_val, loss_alpha = adaptive_loss_fcn(x)\n    assert np.isfinite(loss_fcn_val)\n    assert np.isfinite(loss_alpha)\n\n    # Test case 4: Test with negative values\n    x = np.array([-1, -2, -3, -4, -5])\n    loss_fcn_val, loss_alpha = adaptive_loss_fcn(x)\n    assert np.isfinite(loss_fcn_val)\n    assert np.isfinite(loss_alpha)\n\n    # Test case 5: Test with large values\n    x = np.array([1e10, 2e10, 3e10, 4e10, 5e10])\n    loss_fcn_val, loss_alpha = adaptive_loss_fcn(x)\n    assert np.isfinite(loss_fcn_val)\n    assert np.isfinite(loss_alpha)\n\n\ndef test_adaptive_weights():\n    # Test case 1: x has not been standardized\n    x = np.array([1, 2, 3, 4, 5])\n    weights, C, alpha = adaptive_weights(x)\n    assert np.allclose(weights, np.array([1, 1, 1, 0.99993894, 0.99992852]), atol=1e-3)\n    assert np.isclose(C, 4.4478)\n    assert np.isclose(alpha, 2.0)\n\n    # Test case 2: x has been standardized\n    x = np.array([1, 2, 3, 4, 5])\n    mu = np.mean(x)\n    sigma = np.std(x)\n    x = (x - mu) / sigma\n    weights, C, alpha = adaptive_weights(x, alpha=3)\n    assert np.allclose(weights, np.array([1, 1, 1, 1.02496275, 1.09644634]), atol=1e-3)\n    assert np.isclose(C, 3.1450695413615257)\n    assert np.isclose(alpha, 3.0)\n\n    # Test case 3: x contains non-finite values\n    x = np.array([1, 2, np.nan, 4, 5])\n    weights, C, alpha = adaptive_weights(x)\n    assert np.allclose(\n        weights, np.array([1, 1, 0.99993282, 0.99994308, 0.99993282]), atol=1e-3\n    )\n    assert np.isclose(C, 5.55975)\n    assert np.isclose(alpha, 2.0)\n\n    # Test case 4: x contains outliers\n    x = np.array([1, 2, 3, 4, 5, 100])\n    weights, C, alpha = adaptive_weights(x)\n    assert np.allclose(weights, np.array([1, 1, 1, 0.9865, 0.9479, 0.0011]), atol=1e-3)\n    assert np.isclose(C, 6.05975)\n    assert np.isclose(alpha, -1.0928, atol=1e-2)\n"
  },
  {
    "path": "tests/eemeter/daily_model/utilities/test_base_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pytest\nfrom scipy.stats import linregress, theilslopes\n\nfrom opendsm.eemeter.models.daily.utilities.settings import DailySettings\nfrom opendsm.eemeter.models.daily.utilities.base_model import (\n    get_slope,\n    linear_fit,\n    get_smooth_coeffs,\n    fix_identical_bnds,\n    get_intercept,\n)\n\n\n@pytest.fixture\ndef get_settings():\n    settings = DailySettings()\n    return settings\n\n\ndef test_get_intercept():\n    # Test case 1: alpha = 2, y has positive values\n    y = np.array([1, 2, 3, 4, 5])\n    assert get_intercept(y) == 3.0\n\n    # Test case 2: alpha = 2, y has negative values\n    y = np.array([-5, -4, -3, -2, -1])\n    assert get_intercept(y) == -3.0\n\n    # Test case 3: alpha = 1, y has positive values\n    y = np.array([1, 2, 3, 4, 5])\n    assert get_intercept(y, alpha=1) == 3.0\n\n    # Test case 4: alpha = 1, y has negative values\n    y = np.array([-5, -4, -3, -2, -1])\n    assert get_intercept(y, alpha=1) == -3.0\n\n    # Test case 5: alpha = 2, y has both positive and negative values\n    y = np.array([-5, -4, -3, 2, 1])\n    assert get_intercept(y) == -1.8\n\n    # Test case 6: alpha = 1, y has both positive and negative values\n    y = np.array([-5, -4, -3, 2, 1])\n    assert get_intercept(y, alpha=1) == -3.0\n\n\ndef test_get_slope():\n    # Test case 1: Test with alpha=2\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([2, 4, 6, 8, 10])\n    x_bp = 3\n    intercept = 0\n    alpha = 2\n    expected_slope = 2.0102\n    assert np.isclose(\n        get_slope(x, y, x_bp, intercept, alpha), expected_slope, atol=1e-3\n    )\n\n    # Test case 2: Test with alpha=1\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([2, 4, 6, 8, 10])\n    x_bp = 3\n    intercept = 0\n    alpha = 1\n    expected_slope = 2.125\n    assert np.isclose(\n        get_slope(x, y, x_bp, intercept, alpha), expected_slope, atol=1e-3\n    )\n\n    # Test case 3: Test with negative y values\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([-2, -4, -6, -8, -10])\n    x_bp = 3\n    intercept = 0\n    alpha = 2\n    expected_slope = -2.016\n    assert np.isclose(\n        get_slope(x, y, x_bp, intercept, alpha), expected_slope, atol=1e-3\n    )\n\n    # Test case 4: Test with non-zero intercept\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([2, 4, 6, 8, 10])\n    x_bp = 3\n    intercept = 1\n    alpha = 2\n    expected_slope = 2.0143\n    assert np.isclose(\n        get_slope(x, y, x_bp, intercept, alpha), expected_slope, atol=1e-3\n    )\n\n\ndef test_linear_fit():\n    # Test case 1: Test with alpha = 2\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([2, 4, 6, 8, 10])\n    alpha = 2\n    slope, intercept = linear_fit(x, y, alpha)\n    res = linregress(x, y)\n    assert slope == res.slope\n    assert intercept == res.intercept\n\n    # Test case 2: Test with alpha = 0.95\n    x = np.array([1, 2, 3, 4, 5])\n    y = np.array([2, 4, 6, 8, 10])\n    alpha = 0.95\n    slope, intercept = linear_fit(x, y, alpha)\n    res = theilslopes(y, x, alpha=0.95)\n    assert slope == res[0]\n    assert intercept == res[1]\n\n    # Test case 3: Test with alpha = 2 and identical observations\n    x = np.array([10, 10, 10, 10, 10])\n    y = np.array([2, 4, 6, 8, 10])\n    alpha = 2\n    slope, intercept = linear_fit(x, y, alpha)\n    with pytest.raises(ValueError):\n        res = linregress(x, y)\n    assert slope == 0\n    assert intercept == x[0]\n\n\ndef test_get_smooth_coeffs():\n    # Test case 1: Both pct_hdd_k and pct_cdd_k are less than min_pct_k\n    coeffs = get_smooth_coeffs(10, 0.005, 20, 0.005, min_pct_k=0.01)\n    assert np.allclose(coeffs, np.array([10, 0, 20, 0]))\n\n    # Test case 2: pct_hdd_k is greater than min_pct_k and pct_cdd_k is less than min_pct_k\n    coeffs = get_smooth_coeffs(10, 0.1, 20, 0.005, min_pct_k=0.01)\n    assert np.allclose(coeffs, np.array([11, 1, 19.95, 0.05]))\n\n    # Test case 3: pct_cdd_k is greater than min_pct_k and pct_hdd_k is less than min_pct_k\n    coeffs = get_smooth_coeffs(10, 0.005, 20, 0.1, min_pct_k=0.01)\n    assert np.allclose(coeffs, np.array([10.05, 0.05, 19, 1]))\n\n    # Test case 4: pct_hdd_k and pct_cdd_k are both greater than min_pct_k and sum to less than or equal to 1\n    coeffs = get_smooth_coeffs(10, 0.1, 20, 0.2, min_pct_k=0.01)\n    assert np.allclose(coeffs, np.array([11, 1, 18, 2]))\n\n    # Test case 5: pct_hdd_k and pct_cdd_k are both greater than min_pct_k and sum to greater than 1\n    coeffs = get_smooth_coeffs(10, 0.5, 20, 0.6, min_pct_k=0.01)\n    assert np.allclose(\n        coeffs, np.array([14.54545455, 4.54545455, 14.54545455, 5.45454545])\n    )\n\n    # Test case 6: pct_match is 1.0, so the smoothed curve should converge at - or + inf\n    coeffs = get_smooth_coeffs(10, 0.1, 20, 0.2, min_pct_k=0.01)\n    assert np.allclose(coeffs, np.array([11, 1, 18, 2]))\n\n\ndef test_fix_identical_bnds():\n    # Test case 1: No bounds are identical\n    bnds = np.array([[1, 2], [3, 4], [5, 6]])\n    assert np.array_equal(fix_identical_bnds(bnds), bnds)\n\n    # Test case 2: One bound is identical\n    bnds = np.array([[1, 2], [3, 3], [5, 6]])\n    expected_output = np.array([[1, 2], [2, 4], [5, 6]])\n    assert np.array_equal(fix_identical_bnds(bnds), expected_output)\n\n    # Test case 3: Multiple bounds are identical\n    bnds = np.array([[1, 2], [3, 3], [5, 5]])\n    expected_output = np.array([[1, 2], [2, 4], [4, 6]])\n    assert np.array_equal(fix_identical_bnds(bnds), expected_output)\n\n    # Test case 4: All bounds are identical\n    bnds = np.array([[1, 1], [1, 1], [1, 1]])\n    expected_output = np.array([[0, 2], [0, 2], [0, 2]])\n    assert np.array_equal(fix_identical_bnds(bnds), expected_output)\n"
  },
  {
    "path": "tests/eemeter/daily_model/utilities/test_config.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport pytest\n\nfrom opendsm.eemeter.models.daily.utilities.settings import (\n    DailySettings,\n)\n\n\ndef test_default_settings():\n    settings = DailySettings()\n    assert settings.developer_mode is False\n    assert settings.algorithm_choice.lower() == \"nlopt_sbplx\"\n    assert settings.initial_guess_algorithm_choice.lower() == \"nlopt_direct\"\n    assert settings.alpha_selection == 2.0\n    assert settings.alpha_final == \"adaptive\"\n    assert settings.alpha_final_type == \"last\"\n    assert settings.regularization_alpha == 0.001\n    assert settings.regularization_percent_lasso == 1.0\n    assert settings.allow_smooth_model is True\n    assert settings.split_selection.allow_separate_summer is True\n    assert settings.split_selection.allow_separate_shoulder is True\n    assert settings.split_selection.allow_separate_winter is True\n    assert settings.split_selection.allow_separate_weekday_weekend is True\n    assert settings.split_selection.reduce_splits_by_gaussian is True\n    assert settings.segment_minimum_count == 6\n\n\ndef test_custom_settings():\n    settings_dict = {\n        \"developer_mode\": True,\n        \"algorithm_choice\": \"scipy_SLSQP\",\n        \"initial_guess_algorithm_choice\": \"nlopt_DIRECT_L\",\n        \"alpha_selection\": 1.5,\n        \"alpha_final\": 1.5,\n        \"alpha_final_type\": \"last\",\n        \"regularization_alpha\": 0.01,\n        \"regularization_percent_lasso\": 0.5,\n        \"allow_smooth_model\": True,\n        \"split_selection\": {\n            \"allow_separate_summer\": True,\n            \"allow_separate_shoulder\": True,\n            \"allow_separate_winter\": True,\n            \"allow_separate_weekday_weekend\": True,\n            \"reduce_splits_by_gaussian\": True,\n        },\n        \"segment_minimum_count\": 20,\n    }\n    settings = DailySettings(**settings_dict)\n\n    assert settings.developer_mode is True\n    assert settings.algorithm_choice.lower() == \"scipy_slsqp\"\n    assert settings.initial_guess_algorithm_choice.lower() == \"nlopt_direct_l\"\n    assert settings.alpha_selection == 1.5\n    assert settings.alpha_final == 1.5\n    assert settings.alpha_final_type == \"last\"\n    assert settings.regularization_alpha == 0.01\n    assert settings.regularization_percent_lasso == 0.5\n    assert settings.allow_smooth_model is True\n    assert settings.split_selection.allow_separate_summer is True\n    assert settings.split_selection.allow_separate_shoulder is True\n    assert settings.split_selection.allow_separate_winter is True\n    assert settings.split_selection.allow_separate_weekday_weekend is True\n    assert settings.split_selection.reduce_splits_by_gaussian is True\n    assert settings.segment_minimum_count == 20\n\n\ndef test_invalid_settings():\n    with pytest.raises(ValueError):\n        DailySettings(developer_mode=False, algorithm_choice=\"invalid_algorithm\")\n    with pytest.raises(ValueError):\n        DailySettings(developer_mode=False, alpha_selection=0.5)\n    with pytest.raises(ValueError):\n        DailySettings(developer_mode=False, alpha_selection=1.5)\n    with pytest.raises(ValueError):\n        DailySettings(developer_mode=False, alpha_final_type=\"invalid_type\")\n"
  },
  {
    "path": "tests/eemeter/daily_model/utilities/test_ellipsoid_test.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nimport pandas as pd\n\nfrom opendsm.eemeter.models.daily.utilities.ellipsoid_test import (\n    ellipsoid_intersection_test,\n    ellipsoid_K_function,\n    robust_confidence_ellipse,\n    ellipsoid_split_filter,\n)\n\n\ndef test_ellipsoid_intersection_test():\n    # test case 1: ellipsoids intersect\n    mu_A = np.array([0, 0])\n    mu_B = np.array([1, 1])\n    cov_A = np.array([[1, 0], [0, 1]])\n    cov_B = np.array([[1, 0], [0, 1]])\n    assert ellipsoid_intersection_test(mu_A, mu_B, cov_A, cov_B) == True\n\n    # test case 2: ellipsoids do not intersect\n    mu_A = np.array([0, 0])\n    mu_B = np.array([3, 3])\n    cov_A = np.array([[1, 0], [0, 1]])\n    cov_B = np.array([[1, 0], [0, 1]])\n    assert ellipsoid_intersection_test(mu_A, mu_B, cov_A, cov_B) == False\n\n    # test case 3: ellipsoids intersect at a single point\n    mu_A = np.array([0, 0])\n    mu_B = np.array([1, 0])\n    cov_A = np.array([[1, 0], [0, 1]])\n    cov_B = np.array([[1, 0], [0, 1]])\n    assert ellipsoid_intersection_test(mu_A, mu_B, cov_A, cov_B) == True\n\n\ndef test_ellipsoid_K_function():\n    # test case 1: ss = 0.5 -> doesn't match?\n    ss = 0.5\n    lambdas = np.array([1, 2, 3])\n    v_squared = np.array([1, 2, 3])\n    assert np.isclose(ellipsoid_K_function(ss, lambdas, v_squared), 0.0417, atol=1e-3)\n\n    # test case 2: ss = 0\n    ss = 0\n    lambdas = np.array([1, 2])\n    v_squared = np.array([1, 2])\n    assert np.isclose(ellipsoid_K_function(ss, lambdas, v_squared), 1)\n\n    # test case 3: ss = 1\n    ss = 1\n    lambdas = np.array([1, 2])\n    v_squared = np.array([1, 2])\n    assert np.isclose(ellipsoid_K_function(ss, lambdas, v_squared), 1)\n\n\ndef test_robust_confidence_ellipse():\n    # these answers should remain constant\n    def assert_close(mu, cov, a, b, phi):\n        assert np.allclose(mu, np.array([10.4, 18.1]), rtol=0.1)\n        assert np.allclose(cov, np.array([[27.2, -17], [-17, 28.4]]), rtol=0.1)\n        assert np.isclose(a, 3.3, rtol=0.1)\n        assert np.isclose(b, 6.7, rtol=0.1)\n        assert np.isclose(phi, -2.4, rtol=0.1)\n\n    # test case 1: no outliers\n    # fmt: off\n    x = np.array([ 4.9,  5.2,  6.4,  6.8,  8.6,  8.6, 10.3, 10.4, 10.9, 11.4, 12.2, 12.9, 15.1, 15.3, 17.1])\n    y = np.array([24.5, 23.5, 13.8, 25.5, 13.6, 16.4, 15.4, 22.6,  8.9, 21.0, 13.6, 23.2, 21.7, 13.7, 14.1])\n    # fmt: on\n    mu, cov, a, b, phi = robust_confidence_ellipse(x, y)\n    assert_close(mu, cov, a, b, phi)\n\n    # test case 2: with outliers\n    x = np.append(x, [8.7, 9.9, 10.5, 13, 13.4])\n    y = np.append(y, [34.7, 4.4, 35.3, 5.9, 28.6])\n    mu, cov, a, b, phi = robust_confidence_ellipse(x, y)\n    assert_close(mu, cov, a, b, phi)\n\n\ndef test_ellipsoid_split_filter():\n    # TODO : Add more test cases which contain all seasons and weekday/weekend\n    # Test case 1: Test with a small dataset\n    meter = pd.DataFrame(\n        {\n            \"season\": [\n                \"summer\",\n                \"summer\",\n                \"summer\",\n                \"shoulder\",\n                \"shoulder\",\n                \"shoulder\",\n                \"winter\",\n                \"winter\",\n                \"winter\",\n            ],\n            \"day_of_week\": [1, 2, 3, 4, 5, 6, 7, 1, 2],\n            \"temperature\": [20, 25, 30, 15, 20, 25, 10, 5, 0],\n            \"observed\": [10, 20, 30, 15, 25, 35, 5, 10, 15],\n        }\n    )\n    expected_output = {\n        \"summer\": True,\n        \"shoulder\": True,\n        \"winter\": True,\n        \"weekday_weekend\": True,\n    }\n    assert ellipsoid_split_filter(meter) == expected_output\n"
  },
  {
    "path": "tests/eemeter/daily_model/utilities/test_selection_criteria.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nimport numpy as np\nfrom opendsm.eemeter.models.daily.utilities.selection_criteria import (\n    neg_log_likelihood,\n    selection_criteria,\n)\n\n\ndef test_neg_log_likelihood():\n    # Test case 1: Test with a simple loss and N\n    loss = 1.0\n    N = 10\n    result = neg_log_likelihood(loss, N)\n    expected = -2.6764598670764994\n    assert np.allclose(result, expected)\n\n    # Test case 2: Test with a larger loss and N\n    loss = 10.0\n    N = 100\n    result = neg_log_likelihood(loss, N)\n    expected = -26.764598670764993\n    assert np.allclose(result, expected)\n\n    # Test case 3: Test with a loss of zero (should return infinity)\n    loss = 0.0\n    N = 10\n    result = neg_log_likelihood(loss, N)\n    expected = np.inf\n    assert np.allclose(result, expected)\n\n    # Test case 4: Test with a negative loss (should raise ValueError)\n    loss = -1.0\n    N = 10\n    try:\n        neg_log_likelihood(loss, N)\n    except ValueError as e:\n        assert str(e) == \"loss must be non-negative\"\n\n    # Test case 5: Test with a negative N (should raise ValueError)\n    loss = 1.0\n    N = -10\n    try:\n        neg_log_likelihood(loss, N)\n    except ValueError as e:\n        assert str(e) == \"N must be positive\"\n\n\ndef test_selection_criteria():\n    # Test case 1: Test with RMSE criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"rmse\"\n    )\n    expected = np.sqrt(loss / N)\n    assert np.allclose(result, expected)\n\n    # Test case 2: Test with RMSE adjusted criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"rmse_adj\"\n    )\n    expected = np.sqrt(loss / (N - num_coeffs - 1))\n    assert np.allclose(result, expected)\n\n    # Test case 3: Test with R-squared criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"r_squared\"\n    )\n    expected = (1 - (1 - loss / TSS)) * 10\n    assert np.allclose(result, expected)\n\n    # Test case 4: Test with adjusted R-squared criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"r_squared_adj\"\n    )\n    expected = 1.2857142857142856\n    assert np.allclose(result, expected)\n\n    # Test case 5: Test with FPE criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"fpe\"\n    )\n    expected = 0.18571428571428572\n    assert np.allclose(result, expected)\n\n    # Test case 6: Test with AIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"aic\"\n    )\n    expected = 0.9352919734152998\n    assert np.allclose(result, expected)\n\n    # Test case 7: Test with AICc criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"aicc\"\n    )\n    expected = 1.1067205448438713\n    assert np.allclose(result, expected)\n\n    # Test case 8: Test with CAIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"caic\"\n    )\n    expected = 1.195808992014109\n    assert np.allclose(result, expected)\n\n    # Test case 9: Test with BIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"bic\"\n    )\n    expected = 0.9958089920141091\n    assert np.allclose(result, expected)\n\n    # Test case 10: Test with SABIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    result = selection_criteria(\n        loss, TSS, N, num_coeffs, model_selection_criteria=\"sabic\"\n    )\n    expected = 0.3966625373033108\n    assert np.allclose(result, expected)\n\n    # Test case 11: Test with invalid criterion -> Not possible since we are using the settings config\n    # loss = 1.0\n    # TSS = 10.0\n    # N = 10\n    # num_coeffs = 2\n    # try:\n    #     selection_criteria(loss, TSS, N, num_coeffs, model_selection_criteria=\"invalid\")\n    # except ValueError as e:\n    #     assert str(e) == \"Invalid model selection criterion\"\n\n    # Remove the below test cases once the following criteria are implemented\n\n    # Test case 12: Test with DIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    try:\n        selection_criteria(loss, TSS, N, num_coeffs, model_selection_criteria=\"dic\")\n    except NotImplementedError as e:\n        assert str(e) == \"DIC has not been implmented as a model selection criterion\"\n\n    # Test case 13: Test with WAIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    try:\n        selection_criteria(loss, TSS, N, num_coeffs, model_selection_criteria=\"waic\")\n    except NotImplementedError as e:\n        assert str(e) == \"WAIC has not been implmented as a model selection criterion\"\n\n    # Test case 14: Test with WBIC criterion\n    loss = 1.0\n    TSS = 10.0\n    N = 10\n    num_coeffs = 2\n    try:\n        selection_criteria(loss, TSS, N, num_coeffs, model_selection_criteria=\"wbic\")\n    except NotImplementedError as e:\n        assert str(e) == \"WBIC has not been implmented as a model selection criterion\"\n"
  },
  {
    "path": "tests/eemeter/hourly_model/conftest.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nfrom opendsm.common.test_data import load_test_data\n\n_TEST_METER = 110596\n\n\n@pytest.fixture\ndef hourly_data():\n    baseline, reporting = load_test_data(\"hourly_treatment_data\")\n    return baseline.loc[_TEST_METER], reporting.loc[_TEST_METER]\n\n\n@pytest.fixture\ndef baseline(hourly_data):\n    baseline, _ = hourly_data\n    baseline.loc[baseline[\"observed\"] > 513, \"observed\"] = (\n        0  # quick extreme value removal\n    )\n    return baseline\n\n\n@pytest.fixture\ndef reporting(hourly_data):\n    _, reporting = hourly_data\n    return reporting"
  },
  {
    "path": "tests/eemeter/hourly_model/test_hourly_model.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter import (\n    HourlyBaselineData,\n    HourlyReportingData,\n    HourlyModel,\n    HourlySolarSettings,\n    HourlyNonSolarSettings,\n)\nfrom opendsm.eemeter.models.hourly.settings import BaseHourlySettings\nfrom opendsm.eemeter.common.exceptions import (\n    DataSufficiencyError,\n    DisqualifiedModelError,\n)\nimport numpy as np\nimport pandas as pd\nimport pytest\nfrom math import ceil\n\n\n\ndef test_good_data(baseline, reporting):\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    reporting_data = HourlyReportingData(reporting, is_electricity_data=True)\n    hm = HourlyModel().fit(baseline_data)\n    p1 = hm.predict(reporting_data)\n    assert np.isclose(\n        p1[\"predicted\"].sum(), 1135000, rtol=1e-2\n    )  # quick check that model fit isn't changing drastically\n    serialized = hm.to_json()\n    hm2 = HourlyModel.from_json(serialized)\n    p2 = hm2.predict(reporting_data)\n    assert p1.equals(p2)\n\n\ndef test_misaligned_data(baseline, reporting):\n    reporting.index = reporting.index.shift(8, freq=\"h\")\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    reporting_data = HourlyReportingData(reporting, is_electricity_data=True)\n    hm = HourlyModel().fit(baseline_data)\n    hm.predict(reporting_data)\n\n\ndef test_tz_naive(baseline):\n    baseline.index = baseline.index.tz_localize(None)\n    with pytest.raises(ValueError):\n        HourlyBaselineData(baseline, is_electricity_data=True)\n\n\ndef test_tz_mismatch(baseline):\n    # might allow automatic adjustment from the model in the future, but hard requirement for now\n    baseline.index = baseline.index.tz_convert(\"US/Pacific\")\n    reporting = baseline.copy()\n    reporting.index = reporting.index.tz_convert(\"US/Eastern\")\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    reporting_data = HourlyReportingData(reporting, is_electricity_data=True)\n    hm = HourlyModel().fit(baseline_data)\n    with pytest.raises(ValueError):\n        hm.predict(reporting_data)\n\n\ndef test_predict_missing_fit_features(baseline, reporting):\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    hm = HourlyModel(settings=HourlySolarSettings()).fit(baseline_data)\n    reporting.drop(\"ghi\", axis=1, inplace=True)\n    reporting_data = HourlyReportingData(reporting, is_electricity_data=True)\n    with pytest.raises(ValueError):\n        hm.predict(reporting_data)\n\n\ndef test_nonsolar_predict_with_ghi(baseline, reporting, caplog):\n    baseline.drop(\"ghi\", axis=1, inplace=True)\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    hm = HourlyModel().fit(baseline_data)\n    reporting_data = HourlyReportingData(reporting, is_electricity_data=True)\n    with caplog.at_level(\"WARNING\"):\n        hm.predict(reporting_data)\n        assert \"GHI\" in caplog.text\n\n\ndef test_forced_solar_model_fit_no_ghi(baseline):\n    baseline = baseline.drop(\"ghi\", axis=1)\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    with pytest.raises(ValueError):\n        HourlyModel(settings=HourlySolarSettings()).fit(baseline_data)\n\n\ndef test_forced_nonsolar_model_fit_with_ghi(baseline):\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    hm = HourlyModel(settings=HourlyNonSolarSettings()).fit(baseline_data)\n    assert [\n        w for w in hm.warnings if w.qualified_name == \"eemeter.potential_model_mismatch\"\n    ]\n\n\ndef test_no_data(baseline):\n    baseline[\"observed\"] = 0\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n\ndef test_negative_meter_values(baseline):\n    baseline.loc[\"2018-01-08\", \"observed\"] = -1\n\n    # gas data can't be negative\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=False)\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n    # elec can\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    HourlyModel().fit(baseline_data)\n\n\ndef test_invalid_baseline_lengths(baseline):\n    # TODO import min/max length from constants\n    MAX_BASELINE_HOURS = 8760\n    MIN_BASELINE_HOURS = ceil(MAX_BASELINE_HOURS * 0.9) - 24\n    short_df = baseline.iloc[:MIN_BASELINE_HOURS]\n\n    extra_days = baseline.iloc[-24*2:]\n    extra_days.index += pd.Timedelta(days=2)\n    long_df = pd.concat([baseline, extra_days])\n\n    short_baseline = HourlyBaselineData(short_df, is_electricity_data=True)\n    long_baseline = HourlyBaselineData(long_df, is_electricity_data=True)\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(short_baseline)\n    hm_short = HourlyModel().fit(short_baseline, ignore_disqualification=True)\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(long_baseline)\n    hm_long = HourlyModel().fit(long_baseline, ignore_disqualification=True)\n\n\ndef test_low_freq_temp(baseline):\n    baseline[\"temperature\"] = baseline[\"temperature\"].resample(\"D\").mean()\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    assert_dq(\n        baseline_data,\n        [\"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\"],\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n\ndef test_low_freq_meter(baseline):\n    baseline[\"observed\"] = baseline[\"observed\"].resample(\"D\").mean()\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    assert_dq(\n        baseline_data,\n        [\"eemeter.sufficiency_criteria.too_many_days_with_missing_observed_data\"],\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n\ndef test_monthly_percentage(baseline):\n    missing_idx = pd.date_range(\n        start=baseline.index.min(), end=baseline.index.max(), freq=\"h\"\n    )\n    # create datetimeindex where a little over 10% of days are missing in feb, but still 90% overall\n    missing_idx = missing_idx[missing_idx.day < 4]\n    invalid_baseline = baseline[~baseline.index.isin(missing_idx)]\n    # create datetimeindex where a little under 10% of days are missing in feb\n    missing_idx = missing_idx[missing_idx.day < 3]\n    valid_baseline = baseline[~baseline.index.isin(missing_idx)]\n\n    invalid_temp = baseline.copy()\n    invalid_temp.loc[invalid_temp.index.day < 5, \"temperature\"] = np.nan\n\n    invalid_meter = baseline.copy()\n    invalid_meter.loc[invalid_meter.index.day < 5, \"observed\"] = np.nan\n\n    baseline_data = HourlyBaselineData(invalid_baseline, is_electricity_data=True)\n    assert_dq(\n        baseline_data, [\"eemeter.sufficiency_criteria.missing_monthly_temperature_data\"]\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n    baseline_data = HourlyBaselineData(valid_baseline, is_electricity_data=True)\n    HourlyModel().fit(baseline_data)\n\n    baseline_data = HourlyBaselineData(invalid_temp, is_electricity_data=True)\n    assert_dq(\n        baseline_data,\n        [\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n            \"eemeter.sufficiency_criteria.missing_monthly_temperature_data\",\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_temperature_data\",\n        ],\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n    baseline_data = HourlyBaselineData(invalid_meter, is_electricity_data=True)\n    assert_dq(\n        baseline_data,\n        [\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_joint_data\",\n            \"eemeter.sufficiency_criteria.missing_monthly_observed_data\",\n            \"eemeter.sufficiency_criteria.too_many_days_with_missing_observed_data\",\n        ],\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n\ndef test_monthly_ghi_percentage(baseline):\n    # create datetimeindex where a little over 10% of days are missing in feb, but still 90% overall\n    missing_idx = pd.date_range(\n        start=baseline.index.min(), end=baseline.index.max(), freq=\"h\"\n    )\n    missing_idx = missing_idx[missing_idx.day < 4]\n\n    invalid_ghi = baseline.copy()\n    invalid_ghi.loc[invalid_ghi.index.day < 5, \"ghi\"] = np.nan\n\n    baseline_data = HourlyBaselineData(invalid_ghi, is_electricity_data=True)\n    assert_dq(\n        baseline_data,\n        [\n            \"eemeter.sufficiency_criteria.missing_monthly_ghi_data\",\n        ],\n    )\n    with pytest.raises(DataSufficiencyError):\n        HourlyModel().fit(baseline_data)\n\n\ndef test_hourly_fit_daily_threshold(baseline):\n    \"\"\"confirm that days with >50% interpolated data are excluded from fit step\"\"\"\n\n    # bit fragile testing private methods this way, but fine for now\n    m = HourlyModel()\n    b1 = baseline.copy()\n    b1.loc[\"2018-01-08\":\"2018-01-08 11\", \"temperature\"] = np.nan\n    b1 = m._add_categorical_features(b1)\n    b1 = m._daily_fitting_sufficiency(b1)\n    assert b1.loc[\"2018-01-08\", \"include_date\"].sum() == 24\n\n    b2 = baseline.copy()\n    b2.loc[\"2018-01-08\":\"2018-01-08 12\", \"temperature\"] = np.nan\n    b2 = m._add_categorical_features(b2)\n    b2 = m._daily_fitting_sufficiency(b2)\n    assert b2.loc[\"2018-01-08\", \"include_date\"].sum() == 0\n    assert b2.loc[\"2018-01-09\", \"include_date\"].sum() == 24\n\n\n@pytest.mark.filterwarnings(\"ignore:Objective did not converge.\")\ndef test_hourly_error_metric_dq(baseline):\n    baseline[\"observed\"] = np.random.normal(-1, 10, len(baseline)) ** 3\n    baseline_data = HourlyBaselineData(baseline, is_electricity_data=True)\n    model = HourlyModel().fit(baseline_data)\n    assert_dq(baseline_data, [\"eemeter.model_fit_metrics\"])\n    with pytest.raises(DisqualifiedModelError):\n        model.predict(baseline_data)\n\n\ndef assert_dq(data, expected_disqualifications):\n    remaining_dq = set(expected_disqualifications)\n    for dq in data.disqualification:\n        if dq.qualified_name in remaining_dq:\n            remaining_dq.remove(dq.qualified_name)\n    assert not remaining_dq\n\n\ndef test_hourly_dict_settings():\n    m = HourlyModel(settings={\"train_features\": [\"feature_col\"]})\n    assert isinstance(m.settings, HourlyNonSolarSettings)\n    assert set(m.settings.train_features) == {\"temperature\", \"feature_col\"}\n    m = HourlyModel(settings={\"train_features\": [\"feature_col\", \"ghi\"]})\n    assert isinstance(m.settings, HourlySolarSettings)\n    assert set(m.settings.train_features) == {\"temperature\", \"ghi\", \"feature_col\"}\n    m = HourlyModel(settings={\"cvrmse_threshold\": 1.0})\n    assert isinstance(m.settings, BaseHourlySettings)\n    assert m.settings.train_features == None\n"
  },
  {
    "path": "tests/legacy_hourly.json",
    "content": "{\"status\": \"SUCCEEDED\", \"method_name\": \"caltrack_hourly\", \"model\": {\"segment_models\": [{\"segment_name\": \"dec-jan-feb-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 32.15899363448022, \"C(hour_of_week)[1]\": 32.58562881548998, \"C(hour_of_week)[2]\": 32.61812220709161, \"C(hour_of_week)[3]\": 32.65520698632631, \"C(hour_of_week)[4]\": 32.61570655800883, \"C(hour_of_week)[5]\": 32.73311651749455, \"C(hour_of_week)[6]\": 31.856477499270156, \"C(hour_of_week)[7]\": 24.907281832799118, \"C(hour_of_week)[8]\": 22.790287600128057, \"C(hour_of_week)[9]\": 21.11727954258783, \"C(hour_of_week)[10]\": 20.761681291728614, \"C(hour_of_week)[11]\": 20.05395739534302, \"C(hour_of_week)[12]\": 19.852304627158343, \"C(hour_of_week)[13]\": 19.178851313753633, \"C(hour_of_week)[14]\": 19.35026829680495, \"C(hour_of_week)[15]\": 19.332117497691748, \"C(hour_of_week)[16]\": 19.89278593138144, \"C(hour_of_week)[17]\": 21.962458863084876, \"C(hour_of_week)[18]\": 25.94023979782371, \"C(hour_of_week)[19]\": 26.43810836126226, \"C(hour_of_week)[20]\": 31.06093758548402, \"C(hour_of_week)[21]\": 31.214327461098584, \"C(hour_of_week)[22]\": 30.8490200146618, \"C(hour_of_week)[23]\": 30.9096072954312, \"C(hour_of_week)[24]\": 31.29415695921615, \"C(hour_of_week)[25]\": 31.241052261465327, \"C(hour_of_week)[26]\": 31.208300117846093, \"C(hour_of_week)[27]\": 31.352104548740712, \"C(hour_of_week)[28]\": 31.60977188989677, \"C(hour_of_week)[29]\": 31.94923976499217, \"C(hour_of_week)[30]\": 30.919028611260423, \"C(hour_of_week)[31]\": 24.53872328208661, \"C(hour_of_week)[32]\": 22.06403295329572, \"C(hour_of_week)[33]\": 20.54919389704653, \"C(hour_of_week)[34]\": 20.359300048719383, \"C(hour_of_week)[35]\": 20.37131269400872, \"C(hour_of_week)[36]\": 20.16300689354454, \"C(hour_of_week)[37]\": 19.60806710042366, \"C(hour_of_week)[38]\": 19.487260513974743, \"C(hour_of_week)[39]\": 19.332725575623478, \"C(hour_of_week)[40]\": 19.898575384762832, \"C(hour_of_week)[41]\": 22.86172819727339, \"C(hour_of_week)[42]\": 26.54196868161532, \"C(hour_of_week)[43]\": 27.039270645311074, \"C(hour_of_week)[44]\": 32.54572993963664, \"C(hour_of_week)[45]\": 32.806307618295, \"C(hour_of_week)[46]\": 32.73563536913076, \"C(hour_of_week)[47]\": 32.640324594388055, \"C(hour_of_week)[48]\": 32.175234270947, \"C(hour_of_week)[49]\": 32.360014648369194, \"C(hour_of_week)[50]\": 32.415444577446074, \"C(hour_of_week)[51]\": 32.89551645807635, \"C(hour_of_week)[52]\": 32.58032978966754, \"C(hour_of_week)[53]\": 32.63647995817117, \"C(hour_of_week)[54]\": 31.940512378808833, \"C(hour_of_week)[55]\": 24.782044601435196, \"C(hour_of_week)[56]\": 22.979195575028317, \"C(hour_of_week)[57]\": 21.808512840446696, \"C(hour_of_week)[58]\": 21.04900057868996, \"C(hour_of_week)[59]\": 20.900855178060457, \"C(hour_of_week)[60]\": 20.624821206342776, \"C(hour_of_week)[61]\": 19.702274239592672, \"C(hour_of_week)[62]\": 19.62181267265293, \"C(hour_of_week)[63]\": 19.634969997947852, \"C(hour_of_week)[64]\": 20.74016163180321, \"C(hour_of_week)[65]\": 23.28685781544317, \"C(hour_of_week)[66]\": 31.786305059379874, \"C(hour_of_week)[67]\": 31.811282119855832, \"C(hour_of_week)[68]\": 32.09839413276148, \"C(hour_of_week)[69]\": 32.04342560921696, \"C(hour_of_week)[70]\": 32.43327898572922, \"C(hour_of_week)[71]\": 32.27941765600451, \"C(hour_of_week)[72]\": 32.58562063475673, \"C(hour_of_week)[73]\": 32.47219241516897, \"C(hour_of_week)[74]\": 32.17647165457752, \"C(hour_of_week)[75]\": 32.13154760722789, \"C(hour_of_week)[76]\": 32.20425127589996, \"C(hour_of_week)[77]\": 32.361440932455, \"C(hour_of_week)[78]\": 31.830691038280342, \"C(hour_of_week)[79]\": 24.80936239041419, \"C(hour_of_week)[80]\": 22.84051696243142, \"C(hour_of_week)[81]\": 21.21123304872646, \"C(hour_of_week)[82]\": 20.520799974278827, \"C(hour_of_week)[83]\": 20.801360213686554, \"C(hour_of_week)[84]\": 20.70103928391113, \"C(hour_of_week)[85]\": 19.752886769960526, \"C(hour_of_week)[86]\": 19.46488853801918, \"C(hour_of_week)[87]\": 18.955052996483698, \"C(hour_of_week)[88]\": 19.931447105228234, \"C(hour_of_week)[89]\": 21.898293879930506, \"C(hour_of_week)[90]\": 30.015772988415666, \"C(hour_of_week)[91]\": 30.555296969210538, \"C(hour_of_week)[92]\": 30.989890391672127, \"C(hour_of_week)[93]\": 31.374047466073655, \"C(hour_of_week)[94]\": 31.343083725389352, \"C(hour_of_week)[95]\": 30.973472199271495, \"C(hour_of_week)[96]\": 31.306742686599623, \"C(hour_of_week)[97]\": 31.91940897071943, \"C(hour_of_week)[98]\": 32.28268156828579, \"C(hour_of_week)[99]\": 32.41773444149017, \"C(hour_of_week)[100]\": 32.79547133298564, \"C(hour_of_week)[101]\": 33.07163360646684, \"C(hour_of_week)[102]\": 32.01290575736232, \"C(hour_of_week)[103]\": 25.288978132064972, \"C(hour_of_week)[104]\": 23.31227014969002, \"C(hour_of_week)[105]\": 22.401526121695, \"C(hour_of_week)[106]\": 21.89728523568504, \"C(hour_of_week)[107]\": 21.919917341963558, \"C(hour_of_week)[108]\": 21.543081382150692, \"C(hour_of_week)[109]\": 20.902587811403922, \"C(hour_of_week)[110]\": 20.380651013725362, \"C(hour_of_week)[111]\": 20.05736410095646, \"C(hour_of_week)[112]\": 21.09189436034263, \"C(hour_of_week)[113]\": 23.201646471034206, \"C(hour_of_week)[114]\": 31.177628188806736, \"C(hour_of_week)[115]\": 31.811928406068034, \"C(hour_of_week)[116]\": 32.27843354784621, \"C(hour_of_week)[117]\": 31.889051334378788, \"C(hour_of_week)[118]\": 31.96378696780874, \"C(hour_of_week)[119]\": 31.972797111289967, \"C(hour_of_week)[120]\": 32.19533126826024, \"C(hour_of_week)[121]\": 32.0203082485063, \"C(hour_of_week)[122]\": 32.250106464808944, \"C(hour_of_week)[123]\": 32.13989484562883, \"C(hour_of_week)[124]\": 32.050235146574835, \"C(hour_of_week)[125]\": 31.91547119227756, \"C(hour_of_week)[126]\": 32.47376160291762, \"C(hour_of_week)[127]\": 31.486635858680938, \"C(hour_of_week)[128]\": 30.371496890619145, \"C(hour_of_week)[129]\": 24.542281415297484, \"C(hour_of_week)[130]\": 23.080358310115333, \"C(hour_of_week)[131]\": 22.12671249926624, \"C(hour_of_week)[132]\": 21.80261264168631, \"C(hour_of_week)[133]\": 19.124890248441016, \"C(hour_of_week)[134]\": 18.598897570415986, \"C(hour_of_week)[135]\": 18.980580306522366, \"C(hour_of_week)[136]\": 19.711871828479243, \"C(hour_of_week)[137]\": 21.54837878115354, \"C(hour_of_week)[138]\": 24.792448566548646, \"C(hour_of_week)[139]\": 30.411171823863764, \"C(hour_of_week)[140]\": 31.045448566610016, \"C(hour_of_week)[141]\": 30.926724616991372, \"C(hour_of_week)[142]\": 31.20207028937535, \"C(hour_of_week)[143]\": 31.67751224549073, \"C(hour_of_week)[144]\": 31.523676458295895, \"C(hour_of_week)[145]\": 31.630152913436977, \"C(hour_of_week)[146]\": 31.90576798845558, \"C(hour_of_week)[147]\": 31.954484843221444, \"C(hour_of_week)[148]\": 32.22954082342334, \"C(hour_of_week)[149]\": 32.76433080154116, \"C(hour_of_week)[150]\": 32.29874264361644, \"C(hour_of_week)[151]\": 32.31885870545857, \"C(hour_of_week)[152]\": 27.05358027346026, \"C(hour_of_week)[153]\": 24.92577860890868, \"C(hour_of_week)[154]\": 23.645576556682865, \"C(hour_of_week)[155]\": 23.009598389626998, \"C(hour_of_week)[156]\": 22.68232694640618, \"C(hour_of_week)[157]\": 22.071141798131883, \"C(hour_of_week)[158]\": 22.159714687068632, \"C(hour_of_week)[159]\": 22.62162735928998, \"C(hour_of_week)[160]\": 24.406460590973563, \"C(hour_of_week)[161]\": 26.316733814852977, \"C(hour_of_week)[162]\": 31.763118194197624, \"C(hour_of_week)[163]\": 31.853519702329205, \"C(hour_of_week)[164]\": 32.37267868638918, \"C(hour_of_week)[165]\": 32.56731989949366, \"C(hour_of_week)[166]\": 32.46366099353558, \"C(hour_of_week)[167]\": 32.43130473132385, \"bin_0_occupied\": -0.5397879922762127, \"bin_1_occupied\": -0.34397717717226745, \"bin_2_occupied\": -0.14714690337327535, \"bin_0_unoccupied\": -0.3728458101434492, \"bin_1_unoccupied\": -0.2544749189147975, \"bin_2_unoccupied\": 0.0067672015189005775, \"bin_3_unoccupied\": -0.09190119178221492}}, {\"segment_name\": \"jan-feb-mar-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 31.69619495313116, \"C(hour_of_week)[1]\": 31.883173988305636, \"C(hour_of_week)[2]\": 31.99141404639163, \"C(hour_of_week)[3]\": 32.09074739677064, \"C(hour_of_week)[4]\": 32.23600462548064, \"C(hour_of_week)[5]\": 32.26958780022159, \"C(hour_of_week)[6]\": 31.607163998402076, \"C(hour_of_week)[7]\": 25.224210851491552, \"C(hour_of_week)[8]\": 23.200868092940475, \"C(hour_of_week)[9]\": 21.756239708839956, \"C(hour_of_week)[10]\": 21.45053533307132, \"C(hour_of_week)[11]\": 20.974927752771226, \"C(hour_of_week)[12]\": 20.726945207164654, \"C(hour_of_week)[13]\": 19.956064449212096, \"C(hour_of_week)[14]\": 20.004053519252523, \"C(hour_of_week)[15]\": 20.039374644764944, \"C(hour_of_week)[16]\": 20.256460456181017, \"C(hour_of_week)[17]\": 21.926348984992135, \"C(hour_of_week)[18]\": 25.784389808153406, \"C(hour_of_week)[19]\": 29.844102116791973, \"C(hour_of_week)[20]\": 31.05749203415121, \"C(hour_of_week)[21]\": 31.43759830970828, \"C(hour_of_week)[22]\": 30.994696096981446, \"C(hour_of_week)[23]\": 31.148536871586423, \"C(hour_of_week)[24]\": 31.445733791551778, \"C(hour_of_week)[25]\": 31.656250562688555, \"C(hour_of_week)[26]\": 31.45673484407656, \"C(hour_of_week)[27]\": 31.058893471617367, \"C(hour_of_week)[28]\": 31.62665283708522, \"C(hour_of_week)[29]\": 32.055577625013214, \"C(hour_of_week)[30]\": 31.071109066481203, \"C(hour_of_week)[31]\": 24.923003177418057, \"C(hour_of_week)[32]\": 22.8536054467803, \"C(hour_of_week)[33]\": 21.565074912090754, \"C(hour_of_week)[34]\": 21.421183327212113, \"C(hour_of_week)[35]\": 21.286096590096317, \"C(hour_of_week)[36]\": 20.959338027433493, \"C(hour_of_week)[37]\": 20.358976799048513, \"C(hour_of_week)[38]\": 20.392517036677425, \"C(hour_of_week)[39]\": 20.22605769454511, \"C(hour_of_week)[40]\": 20.20912765270942, \"C(hour_of_week)[41]\": 22.545761199548352, \"C(hour_of_week)[42]\": 29.805129485793895, \"C(hour_of_week)[43]\": 30.397627823860567, \"C(hour_of_week)[44]\": 31.646015985238012, \"C(hour_of_week)[45]\": 31.76166077797324, \"C(hour_of_week)[46]\": 31.412894996543244, \"C(hour_of_week)[47]\": 31.747111834666857, \"C(hour_of_week)[48]\": 31.497813724873673, \"C(hour_of_week)[49]\": 31.971052517868568, \"C(hour_of_week)[50]\": 31.796321327068743, \"C(hour_of_week)[51]\": 32.31670125990251, \"C(hour_of_week)[52]\": 32.016875417911976, \"C(hour_of_week)[53]\": 32.26366972625077, \"C(hour_of_week)[54]\": 31.21080511896095, \"C(hour_of_week)[55]\": 25.02366667038599, \"C(hour_of_week)[56]\": 23.194660829499313, \"C(hour_of_week)[57]\": 21.724244935041582, \"C(hour_of_week)[58]\": 21.338611215217874, \"C(hour_of_week)[59]\": 21.196324884446863, \"C(hour_of_week)[60]\": 21.014778751151216, \"C(hour_of_week)[61]\": 20.239522385552984, \"C(hour_of_week)[62]\": 20.121881243936457, \"C(hour_of_week)[63]\": 19.816600446880404, \"C(hour_of_week)[64]\": 19.968190293020605, \"C(hour_of_week)[65]\": 21.89468657983282, \"C(hour_of_week)[66]\": 25.58955554272984, \"C(hour_of_week)[67]\": 26.275135234761965, \"C(hour_of_week)[68]\": 30.516714895153605, \"C(hour_of_week)[69]\": 30.68421335931127, \"C(hour_of_week)[70]\": 31.12011660306499, \"C(hour_of_week)[71]\": 31.056096798756233, \"C(hour_of_week)[72]\": 31.692996319309923, \"C(hour_of_week)[73]\": 31.644433823156373, \"C(hour_of_week)[74]\": 31.76206205137058, \"C(hour_of_week)[75]\": 31.597534349965404, \"C(hour_of_week)[76]\": 31.795408618870823, \"C(hour_of_week)[77]\": 32.003085745028, \"C(hour_of_week)[78]\": 31.41905159462924, \"C(hour_of_week)[79]\": 25.19410149086313, \"C(hour_of_week)[80]\": 23.280632189041466, \"C(hour_of_week)[81]\": 21.236633397225464, \"C(hour_of_week)[82]\": 20.80546521259806, \"C(hour_of_week)[83]\": 20.97298122325837, \"C(hour_of_week)[84]\": 20.693438105825326, \"C(hour_of_week)[85]\": 19.79730731816196, \"C(hour_of_week)[86]\": 19.64639621138425, \"C(hour_of_week)[87]\": 19.454786997318166, \"C(hour_of_week)[88]\": 19.984503658417623, \"C(hour_of_week)[89]\": 21.60309625790427, \"C(hour_of_week)[90]\": 25.250521323557383, \"C(hour_of_week)[91]\": 26.285118160160078, \"C(hour_of_week)[92]\": 27.009533579232546, \"C(hour_of_week)[93]\": 30.680938246152486, \"C(hour_of_week)[94]\": 30.408523133580893, \"C(hour_of_week)[95]\": 30.40163570350816, \"C(hour_of_week)[96]\": 30.702092253129234, \"C(hour_of_week)[97]\": 31.289194097096217, \"C(hour_of_week)[98]\": 31.675475965146518, \"C(hour_of_week)[99]\": 31.9019823441271, \"C(hour_of_week)[100]\": 32.194093951688025, \"C(hour_of_week)[101]\": 32.41128619621574, \"C(hour_of_week)[102]\": 31.403093694041896, \"C(hour_of_week)[103]\": 25.396670362001547, \"C(hour_of_week)[104]\": 23.574908903592085, \"C(hour_of_week)[105]\": 22.347264550064004, \"C(hour_of_week)[106]\": 21.783729974786027, \"C(hour_of_week)[107]\": 21.715308036471715, \"C(hour_of_week)[108]\": 21.371582721441662, \"C(hour_of_week)[109]\": 20.57920317267788, \"C(hour_of_week)[110]\": 20.162147035309946, \"C(hour_of_week)[111]\": 20.17328294898878, \"C(hour_of_week)[112]\": 20.267415557908443, \"C(hour_of_week)[113]\": 21.588898958557735, \"C(hour_of_week)[114]\": 25.68719547714796, \"C(hour_of_week)[115]\": 26.88784809866192, \"C(hour_of_week)[116]\": 30.384521093224844, \"C(hour_of_week)[117]\": 30.713591998043963, \"C(hour_of_week)[118]\": 30.70428179311557, \"C(hour_of_week)[119]\": 27.90456419487642, \"C(hour_of_week)[120]\": 31.342643904310705, \"C(hour_of_week)[121]\": 31.396692134779563, \"C(hour_of_week)[122]\": 31.720058588928353, \"C(hour_of_week)[123]\": 31.83784054000408, \"C(hour_of_week)[124]\": 31.969932838053104, \"C(hour_of_week)[125]\": 32.33787214691998, \"C(hour_of_week)[126]\": 32.87128911229978, \"C(hour_of_week)[127]\": 31.75711099419046, \"C(hour_of_week)[128]\": 27.304643595077042, \"C(hour_of_week)[129]\": 25.913003298340104, \"C(hour_of_week)[130]\": 24.7318936523136, \"C(hour_of_week)[131]\": 23.56012282272306, \"C(hour_of_week)[132]\": 23.027322175986097, \"C(hour_of_week)[133]\": 20.599415452585323, \"C(hour_of_week)[134]\": 20.140523234050118, \"C(hour_of_week)[135]\": 20.378163840883204, \"C(hour_of_week)[136]\": 20.612977365130988, \"C(hour_of_week)[137]\": 22.0241785149091, \"C(hour_of_week)[138]\": 25.15122864759263, \"C(hour_of_week)[139]\": 26.901887332692464, \"C(hour_of_week)[140]\": 31.08559919721048, \"C(hour_of_week)[141]\": 31.46762358423757, \"C(hour_of_week)[142]\": 31.578309248819625, \"C(hour_of_week)[143]\": 31.80370806875947, \"C(hour_of_week)[144]\": 31.871049252849307, \"C(hour_of_week)[145]\": 31.8902285482373, \"C(hour_of_week)[146]\": 32.377531002109585, \"C(hour_of_week)[147]\": 32.23360856527041, \"C(hour_of_week)[148]\": 32.39370227874754, \"C(hour_of_week)[149]\": 32.76910914776338, \"C(hour_of_week)[150]\": 32.406658984411145, \"C(hour_of_week)[151]\": 32.021657985064834, \"C(hour_of_week)[152]\": 27.302047873495646, \"C(hour_of_week)[153]\": 25.25129490626848, \"C(hour_of_week)[154]\": 23.927027824336275, \"C(hour_of_week)[155]\": 23.11330868447653, \"C(hour_of_week)[156]\": 23.092483732841984, \"C(hour_of_week)[157]\": 22.670839177305698, \"C(hour_of_week)[158]\": 22.34396350990764, \"C(hour_of_week)[159]\": 22.46316163196926, \"C(hour_of_week)[160]\": 23.24015708557393, \"C(hour_of_week)[161]\": 25.183133758437204, \"C(hour_of_week)[162]\": 29.852980325507737, \"C(hour_of_week)[163]\": 30.502918781669155, \"C(hour_of_week)[164]\": 31.014737451484837, \"C(hour_of_week)[165]\": 31.213454859392417, \"C(hour_of_week)[166]\": 31.406168995022067, \"C(hour_of_week)[167]\": 31.36338670892857, \"bin_0_occupied\": -0.527614366229834, \"bin_1_occupied\": -0.3671416187265185, \"bin_2_occupied\": -0.17678692012547065, \"bin_0_unoccupied\": -0.41436649971578454, \"bin_1_unoccupied\": -0.21027104555092474, \"bin_2_unoccupied\": -0.030931109147582472, \"bin_3_unoccupied\": 0.035954795964798086}}, {\"segment_name\": \"feb-mar-apr-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 29.606509743611365, \"C(hour_of_week)[1]\": 29.570577507170487, \"C(hour_of_week)[2]\": 29.894551074352773, \"C(hour_of_week)[3]\": 30.201982912437487, \"C(hour_of_week)[4]\": 30.458006803474902, \"C(hour_of_week)[5]\": 30.575538190141774, \"C(hour_of_week)[6]\": 29.891861064545132, \"C(hour_of_week)[7]\": 25.422202404675495, \"C(hour_of_week)[8]\": 23.681571762480907, \"C(hour_of_week)[9]\": 22.709993126856723, \"C(hour_of_week)[10]\": 22.813744673000112, \"C(hour_of_week)[11]\": 22.67661849098906, \"C(hour_of_week)[12]\": 22.243257315276175, \"C(hour_of_week)[13]\": 21.825394871803365, \"C(hour_of_week)[14]\": 21.76419112178408, \"C(hour_of_week)[15]\": 21.457409200447408, \"C(hour_of_week)[16]\": 21.25247198499195, \"C(hour_of_week)[17]\": 21.994671452675277, \"C(hour_of_week)[18]\": 24.385311979686048, \"C(hour_of_week)[19]\": 26.036060784572296, \"C(hour_of_week)[20]\": 29.98411167398938, \"C(hour_of_week)[21]\": 30.27470604919401, \"C(hour_of_week)[22]\": 30.12798018150804, \"C(hour_of_week)[23]\": 30.390619112519033, \"C(hour_of_week)[24]\": 30.567741367664933, \"C(hour_of_week)[25]\": 30.90686651216744, \"C(hour_of_week)[26]\": 30.759065158081107, \"C(hour_of_week)[27]\": 30.36244589356716, \"C(hour_of_week)[28]\": 30.81727375751529, \"C(hour_of_week)[29]\": 31.138325987190207, \"C(hour_of_week)[30]\": 30.46410231328193, \"C(hour_of_week)[31]\": 28.055484606844658, \"C(hour_of_week)[32]\": 23.895616718262502, \"C(hour_of_week)[33]\": 23.001850398079505, \"C(hour_of_week)[34]\": 22.96403235230715, \"C(hour_of_week)[35]\": 22.65598914478821, \"C(hour_of_week)[36]\": 22.313599898520728, \"C(hour_of_week)[37]\": 22.044729415280795, \"C(hour_of_week)[38]\": 22.16014095970541, \"C(hour_of_week)[39]\": 21.88045469380224, \"C(hour_of_week)[40]\": 21.606597790019624, \"C(hour_of_week)[41]\": 22.450565018541212, \"C(hour_of_week)[42]\": 24.65425520679909, \"C(hour_of_week)[43]\": 28.447471516088296, \"C(hour_of_week)[44]\": 29.433285504445916, \"C(hour_of_week)[45]\": 29.50420050378277, \"C(hour_of_week)[46]\": 29.438769586535024, \"C(hour_of_week)[47]\": 29.976369142033022, \"C(hour_of_week)[48]\": 30.09679618962316, \"C(hour_of_week)[49]\": 30.35913852561247, \"C(hour_of_week)[50]\": 30.38680007049313, \"C(hour_of_week)[51]\": 30.9213831969174, \"C(hour_of_week)[52]\": 30.98224325864481, \"C(hour_of_week)[53]\": 31.140436285525446, \"C(hour_of_week)[54]\": 30.1846466459844, \"C(hour_of_week)[55]\": 25.41643473636842, \"C(hour_of_week)[56]\": 23.77245108344277, \"C(hour_of_week)[57]\": 22.215789258220827, \"C(hour_of_week)[58]\": 22.373457987146136, \"C(hour_of_week)[59]\": 22.235275040920378, \"C(hour_of_week)[60]\": 22.057567555427134, \"C(hour_of_week)[61]\": 21.696944881008182, \"C(hour_of_week)[62]\": 21.38798420900042, \"C(hour_of_week)[63]\": 20.733100996044413, \"C(hour_of_week)[64]\": 20.347537247278407, \"C(hour_of_week)[65]\": 20.754600526000136, \"C(hour_of_week)[66]\": 22.85023465882854, \"C(hour_of_week)[67]\": 24.859119357188366, \"C(hour_of_week)[68]\": 28.031947550457993, \"C(hour_of_week)[69]\": 28.192189338992257, \"C(hour_of_week)[70]\": 28.87329726573205, \"C(hour_of_week)[71]\": 28.814073214555155, \"C(hour_of_week)[72]\": 29.272795891520655, \"C(hour_of_week)[73]\": 29.463350313690682, \"C(hour_of_week)[74]\": 29.818250997342826, \"C(hour_of_week)[75]\": 29.700215186437074, \"C(hour_of_week)[76]\": 29.82893239247302, \"C(hour_of_week)[77]\": 30.1884935938724, \"C(hour_of_week)[78]\": 29.746660687045047, \"C(hour_of_week)[79]\": 25.119547274272676, \"C(hour_of_week)[80]\": 23.20917927338823, \"C(hour_of_week)[81]\": 22.07010185351045, \"C(hour_of_week)[82]\": 22.163712233924254, \"C(hour_of_week)[83]\": 22.020099848496077, \"C(hour_of_week)[84]\": 21.686955369179568, \"C(hour_of_week)[85]\": 21.358676823474806, \"C(hour_of_week)[86]\": 21.270366535753688, \"C(hour_of_week)[87]\": 20.907198842526405, \"C(hour_of_week)[88]\": 20.79434626401162, \"C(hour_of_week)[89]\": 21.017544223045704, \"C(hour_of_week)[90]\": 22.715054789525517, \"C(hour_of_week)[91]\": 24.4523716369436, \"C(hour_of_week)[92]\": 25.7676288845483, \"C(hour_of_week)[93]\": 26.14382239030146, \"C(hour_of_week)[94]\": 28.192777181335057, \"C(hour_of_week)[95]\": 28.27534938058132, \"C(hour_of_week)[96]\": 28.62022804185484, \"C(hour_of_week)[97]\": 29.02635597931267, \"C(hour_of_week)[98]\": 29.391081966169565, \"C(hour_of_week)[99]\": 29.698164274642615, \"C(hour_of_week)[100]\": 29.9212153978975, \"C(hour_of_week)[101]\": 30.26373856579102, \"C(hour_of_week)[102]\": 29.398533923918382, \"C(hour_of_week)[103]\": 24.707224656623723, \"C(hour_of_week)[104]\": 22.91794334076898, \"C(hour_of_week)[105]\": 22.263957312824157, \"C(hour_of_week)[106]\": 22.167354715588576, \"C(hour_of_week)[107]\": 21.965734169873702, \"C(hour_of_week)[108]\": 21.68677649798916, \"C(hour_of_week)[109]\": 21.366649960986255, \"C(hour_of_week)[110]\": 21.143496695548734, \"C(hour_of_week)[111]\": 20.690958534781053, \"C(hour_of_week)[112]\": 20.20273317845596, \"C(hour_of_week)[113]\": 20.301971188011024, \"C(hour_of_week)[114]\": 23.294334859680777, \"C(hour_of_week)[115]\": 25.11836818746646, \"C(hour_of_week)[116]\": 25.98323265600618, \"C(hour_of_week)[117]\": 26.580701582516248, \"C(hour_of_week)[118]\": 28.296192086429787, \"C(hour_of_week)[119]\": 26.733974603799894, \"C(hour_of_week)[120]\": 29.060730011454314, \"C(hour_of_week)[121]\": 29.619631938151976, \"C(hour_of_week)[122]\": 29.873905566019566, \"C(hour_of_week)[123]\": 30.290500324699167, \"C(hour_of_week)[124]\": 30.618365325283833, \"C(hour_of_week)[125]\": 31.281371184110824, \"C(hour_of_week)[126]\": 31.460024032466265, \"C(hour_of_week)[127]\": 30.576885459225505, \"C(hour_of_week)[128]\": 27.133113875058292, \"C(hour_of_week)[129]\": 25.858501688544358, \"C(hour_of_week)[130]\": 24.771611361912026, \"C(hour_of_week)[131]\": 23.809754193915268, \"C(hour_of_week)[132]\": 23.2233510625287, \"C(hour_of_week)[133]\": 21.80368545587041, \"C(hour_of_week)[134]\": 21.1855224723839, \"C(hour_of_week)[135]\": 21.41209723327908, \"C(hour_of_week)[136]\": 21.403765914054894, \"C(hour_of_week)[137]\": 21.98517017076719, \"C(hour_of_week)[138]\": 23.801834448277816, \"C(hour_of_week)[139]\": 25.511299123655746, \"C(hour_of_week)[140]\": 26.948034713113525, \"C(hour_of_week)[141]\": 29.697698815170355, \"C(hour_of_week)[142]\": 29.663212816709894, \"C(hour_of_week)[143]\": 30.13592171301013, \"C(hour_of_week)[144]\": 30.202063592995245, \"C(hour_of_week)[145]\": 30.18749849014261, \"C(hour_of_week)[146]\": 30.6765439457621, \"C(hour_of_week)[147]\": 30.681133416998353, \"C(hour_of_week)[148]\": 30.72073215230051, \"C(hour_of_week)[149]\": 31.080416635810433, \"C(hour_of_week)[150]\": 31.01233766914123, \"C(hour_of_week)[151]\": 30.38993895255213, \"C(hour_of_week)[152]\": 26.346436403718744, \"C(hour_of_week)[153]\": 24.135369577851325, \"C(hour_of_week)[154]\": 22.762205243751108, \"C(hour_of_week)[155]\": 21.783985327221774, \"C(hour_of_week)[156]\": 21.643093126157503, \"C(hour_of_week)[157]\": 21.26894561921078, \"C(hour_of_week)[158]\": 20.69703175805654, \"C(hour_of_week)[159]\": 20.56203478834863, \"C(hour_of_week)[160]\": 20.98784975661244, \"C(hour_of_week)[161]\": 22.03310475347403, \"C(hour_of_week)[162]\": 23.263109542729204, \"C(hour_of_week)[163]\": 24.992151137728143, \"C(hour_of_week)[164]\": 27.938597960330135, \"C(hour_of_week)[165]\": 28.465518649621863, \"C(hour_of_week)[166]\": 28.997096494272682, \"C(hour_of_week)[167]\": 28.89222807673672, \"bin_0_occupied\": -0.4776947145908701, \"bin_1_occupied\": -0.41218741798488834, \"bin_2_occupied\": -0.26483554007848864, \"bin_0_unoccupied\": -0.427042266557898, \"bin_1_unoccupied\": -0.2725225469513356, \"bin_2_unoccupied\": -0.05816825424413172, \"bin_3_unoccupied\": 0.07191198678486313}}, {\"segment_name\": \"mar-apr-may-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 24.94532743686949, \"C(hour_of_week)[1]\": 27.95841406218917, \"C(hour_of_week)[2]\": 25.358484280629895, \"C(hour_of_week)[3]\": 25.744341840041947, \"C(hour_of_week)[4]\": 25.898107831755766, \"C(hour_of_week)[5]\": 26.0402711176253, \"C(hour_of_week)[6]\": 25.53820628391771, \"C(hour_of_week)[7]\": 24.536994186235095, \"C(hour_of_week)[8]\": 25.77249582845344, \"C(hour_of_week)[9]\": 25.561562466249136, \"C(hour_of_week)[10]\": 25.987951683731154, \"C(hour_of_week)[11]\": 25.866548120420333, \"C(hour_of_week)[12]\": 25.33571241229781, \"C(hour_of_week)[13]\": 25.232945477832104, \"C(hour_of_week)[14]\": 25.16874398222068, \"C(hour_of_week)[15]\": 24.664661646312002, \"C(hour_of_week)[16]\": 24.40152524499718, \"C(hour_of_week)[17]\": 24.40693726739642, \"C(hour_of_week)[18]\": 24.719060019965383, \"C(hour_of_week)[19]\": 25.968789610571033, \"C(hour_of_week)[20]\": 27.32933893696577, \"C(hour_of_week)[21]\": 25.600695134797192, \"C(hour_of_week)[22]\": 27.85776619426793, \"C(hour_of_week)[23]\": 25.896463300682992, \"C(hour_of_week)[24]\": 25.915253650294986, \"C(hour_of_week)[25]\": 26.203732139486092, \"C(hour_of_week)[26]\": 26.34948351949061, \"C(hour_of_week)[27]\": 26.321727961827598, \"C(hour_of_week)[28]\": 26.368111312033435, \"C(hour_of_week)[29]\": 26.529355875594163, \"C(hour_of_week)[30]\": 25.983318109039157, \"C(hour_of_week)[31]\": 24.83028753462647, \"C(hour_of_week)[32]\": 25.661910100866514, \"C(hour_of_week)[33]\": 25.822409614023723, \"C(hour_of_week)[34]\": 26.171142081867863, \"C(hour_of_week)[35]\": 26.00260542153902, \"C(hour_of_week)[36]\": 25.862119618955145, \"C(hour_of_week)[37]\": 25.755446859416747, \"C(hour_of_week)[38]\": 25.90749566238141, \"C(hour_of_week)[39]\": 25.423021839938183, \"C(hour_of_week)[40]\": 25.31274570446314, \"C(hour_of_week)[41]\": 25.010149282201883, \"C(hour_of_week)[42]\": 24.559993985482414, \"C(hour_of_week)[43]\": 25.698525382657543, \"C(hour_of_week)[44]\": 26.725472137746152, \"C(hour_of_week)[45]\": 26.995802186671888, \"C(hour_of_week)[46]\": 25.228583453292227, \"C(hour_of_week)[47]\": 25.129457200504834, \"C(hour_of_week)[48]\": 25.35101821558891, \"C(hour_of_week)[49]\": 27.905117654831173, \"C(hour_of_week)[50]\": 25.788363588066684, \"C(hour_of_week)[51]\": 26.244918075458266, \"C(hour_of_week)[52]\": 26.36863046590338, \"C(hour_of_week)[53]\": 26.42758133312134, \"C(hour_of_week)[54]\": 26.166572141874077, \"C(hour_of_week)[55]\": 26.5757019332486, \"C(hour_of_week)[56]\": 26.070053470062348, \"C(hour_of_week)[57]\": 25.45164738799054, \"C(hour_of_week)[58]\": 25.93485548847692, \"C(hour_of_week)[59]\": 25.853494741792083, \"C(hour_of_week)[60]\": 25.646634376429848, \"C(hour_of_week)[61]\": 25.57081855189757, \"C(hour_of_week)[62]\": 25.036061606190135, \"C(hour_of_week)[63]\": 24.23240089187477, \"C(hour_of_week)[64]\": 24.060044315867447, \"C(hour_of_week)[65]\": 23.343784128619163, \"C(hour_of_week)[66]\": 23.246268302642783, \"C(hour_of_week)[67]\": 24.89364375294519, \"C(hour_of_week)[68]\": 25.87887224008219, \"C(hour_of_week)[69]\": 26.076815003373525, \"C(hour_of_week)[70]\": 26.74100153206202, \"C(hour_of_week)[71]\": 26.52082619643934, \"C(hour_of_week)[72]\": 26.74890607326323, \"C(hour_of_week)[73]\": 27.036065481783783, \"C(hour_of_week)[74]\": 27.408118860844144, \"C(hour_of_week)[75]\": 27.35347283753901, \"C(hour_of_week)[76]\": 27.54697835118427, \"C(hour_of_week)[77]\": 27.839808685094784, \"C(hour_of_week)[78]\": 27.143559890759278, \"C(hour_of_week)[79]\": 25.862332250363707, \"C(hour_of_week)[80]\": 24.914850834891862, \"C(hour_of_week)[81]\": 25.353449812475823, \"C(hour_of_week)[82]\": 26.111755803366897, \"C(hour_of_week)[83]\": 26.060879965869432, \"C(hour_of_week)[84]\": 25.99794520982336, \"C(hour_of_week)[85]\": 26.018476296817866, \"C(hour_of_week)[86]\": 25.806988457851304, \"C(hour_of_week)[87]\": 25.304203075319034, \"C(hour_of_week)[88]\": 24.59493455741477, \"C(hour_of_week)[89]\": 23.618955868438267, \"C(hour_of_week)[90]\": 22.621459168417342, \"C(hour_of_week)[91]\": 24.088094289772066, \"C(hour_of_week)[92]\": 25.433137245328258, \"C(hour_of_week)[93]\": 25.750115804185434, \"C(hour_of_week)[94]\": 26.365038611702374, \"C(hour_of_week)[95]\": 26.447387093795466, \"C(hour_of_week)[96]\": 26.858935630674857, \"C(hour_of_week)[97]\": 24.6354735413625, \"C(hour_of_week)[98]\": 24.808826654636864, \"C(hour_of_week)[99]\": 25.09592383956292, \"C(hour_of_week)[100]\": 25.33086242752335, \"C(hour_of_week)[101]\": 25.4394246512882, \"C(hour_of_week)[102]\": 24.639157271900658, \"C(hour_of_week)[103]\": 25.70842121599432, \"C(hour_of_week)[104]\": 24.80871998929216, \"C(hour_of_week)[105]\": 25.181411878416725, \"C(hour_of_week)[106]\": 25.625782416090768, \"C(hour_of_week)[107]\": 25.41022670060498, \"C(hour_of_week)[108]\": 25.28441101786722, \"C(hour_of_week)[109]\": 25.289050239681718, \"C(hour_of_week)[110]\": 24.93680085388033, \"C(hour_of_week)[111]\": 24.333916541413082, \"C(hour_of_week)[112]\": 23.733820905231248, \"C(hour_of_week)[113]\": 22.970669537342157, \"C(hour_of_week)[114]\": 23.973514844608673, \"C(hour_of_week)[115]\": 25.46415435951515, \"C(hour_of_week)[116]\": 26.326992779380763, \"C(hour_of_week)[117]\": 26.806774372647318, \"C(hour_of_week)[118]\": 27.202866306910494, \"C(hour_of_week)[119]\": 26.929993309857025, \"C(hour_of_week)[120]\": 24.877083772691382, \"C(hour_of_week)[121]\": 28.226027307530742, \"C(hour_of_week)[122]\": 25.690835905562135, \"C(hour_of_week)[123]\": 25.94869204946375, \"C(hour_of_week)[124]\": 26.346371490621117, \"C(hour_of_week)[125]\": 26.84049651702878, \"C(hour_of_week)[126]\": 26.485671085004906, \"C(hour_of_week)[127]\": 25.698818666081177, \"C(hour_of_week)[128]\": 27.250226360530313, \"C(hour_of_week)[129]\": 26.173351005778787, \"C(hour_of_week)[130]\": 25.194607984116143, \"C(hour_of_week)[131]\": 24.645146626599946, \"C(hour_of_week)[132]\": 24.53951860892081, \"C(hour_of_week)[133]\": 24.298384295753422, \"C(hour_of_week)[134]\": 23.758008890121964, \"C(hour_of_week)[135]\": 23.961562275219332, \"C(hour_of_week)[136]\": 23.96891322745426, \"C(hour_of_week)[137]\": 24.01492467826856, \"C(hour_of_week)[138]\": 24.360551574327545, \"C(hour_of_week)[139]\": 25.29713523037705, \"C(hour_of_week)[140]\": 26.732067993600822, \"C(hour_of_week)[141]\": 27.758539725416902, \"C(hour_of_week)[142]\": 27.71157252664556, \"C(hour_of_week)[143]\": 28.48251411541564, \"C(hour_of_week)[144]\": 28.357869732278, \"C(hour_of_week)[145]\": 25.23293848669381, \"C(hour_of_week)[146]\": 25.601493800507523, \"C(hour_of_week)[147]\": 26.191385429824926, \"C(hour_of_week)[148]\": 26.213582233037545, \"C(hour_of_week)[149]\": 26.70499819557181, \"C(hour_of_week)[150]\": 26.348590431553188, \"C(hour_of_week)[151]\": 25.894865756458742, \"C(hour_of_week)[152]\": 27.087064092936618, \"C(hour_of_week)[153]\": 25.23317263387358, \"C(hour_of_week)[154]\": 24.203527728659093, \"C(hour_of_week)[155]\": 23.448851444628612, \"C(hour_of_week)[156]\": 23.119404192346185, \"C(hour_of_week)[157]\": 22.68992700465754, \"C(hour_of_week)[158]\": 22.185281587432044, \"C(hour_of_week)[159]\": 22.22954552331926, \"C(hour_of_week)[160]\": 22.629722255028625, \"C(hour_of_week)[161]\": 23.016870240921893, \"C(hour_of_week)[162]\": 23.697884545227616, \"C(hour_of_week)[163]\": 24.863912317428934, \"C(hour_of_week)[164]\": 26.34355973620864, \"C(hour_of_week)[165]\": 27.024162553658797, \"C(hour_of_week)[166]\": 24.543922117247234, \"C(hour_of_week)[167]\": 27.4523665173592, \"bin_0_occupied\": -0.31288354776816923, \"bin_1_occupied\": -0.5097332126768013, \"bin_2_occupied\": -0.4911792137768626, \"bin_3_occupied\": -0.1947890203086559, \"bin_4_occupied\": 0.02633850336936243, \"bin_0_unoccupied\": -0.4540651803675606, \"bin_1_unoccupied\": -0.417641000573493, \"bin_2_unoccupied\": -0.13743962287054115, \"bin_3_unoccupied\": -0.09455961013287216, \"bin_4_unoccupied\": 0.2557824771519784, \"bin_5_unoccupied\": 0.15473917176742324}}, {\"segment_name\": \"apr-may-jun-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 26.809299112573363, \"C(hour_of_week)[1]\": 26.551498757039358, \"C(hour_of_week)[2]\": 26.938705209900018, \"C(hour_of_week)[3]\": 27.1313786780544, \"C(hour_of_week)[4]\": 27.172983140202216, \"C(hour_of_week)[5]\": 26.96831795372348, \"C(hour_of_week)[6]\": 26.95941952825705, \"C(hour_of_week)[7]\": 20.150308914511932, \"C(hour_of_week)[8]\": 20.1426273995468, \"C(hour_of_week)[9]\": 20.552273497778227, \"C(hour_of_week)[10]\": 21.13320000052441, \"C(hour_of_week)[11]\": 20.976549145620783, \"C(hour_of_week)[12]\": 20.533476789006887, \"C(hour_of_week)[13]\": 20.578323633546773, \"C(hour_of_week)[14]\": 20.408591335246566, \"C(hour_of_week)[15]\": 19.896114082385715, \"C(hour_of_week)[16]\": 19.68187172961265, \"C(hour_of_week)[17]\": 18.92996899552198, \"C(hour_of_week)[18]\": 25.48482171016112, \"C(hour_of_week)[19]\": 25.8856834256579, \"C(hour_of_week)[20]\": 26.954644400633313, \"C(hour_of_week)[21]\": 27.131123786141156, \"C(hour_of_week)[22]\": 27.195669758412798, \"C(hour_of_week)[23]\": 26.953286466349375, \"C(hour_of_week)[24]\": 26.9753761890457, \"C(hour_of_week)[25]\": 27.215801155041625, \"C(hour_of_week)[26]\": 27.377567748715165, \"C(hour_of_week)[27]\": 27.389100899643164, \"C(hour_of_week)[28]\": 27.295071140604858, \"C(hour_of_week)[29]\": 27.20397616446049, \"C(hour_of_week)[30]\": 26.745965386674204, \"C(hour_of_week)[31]\": 19.529863766550136, \"C(hour_of_week)[32]\": 19.484199658063467, \"C(hour_of_week)[33]\": 20.42284442324013, \"C(hour_of_week)[34]\": 21.0595118734399, \"C(hour_of_week)[35]\": 20.917092797436045, \"C(hour_of_week)[36]\": 20.946401250720996, \"C(hour_of_week)[37]\": 20.89880243654278, \"C(hour_of_week)[38]\": 20.766285880249484, \"C(hour_of_week)[39]\": 20.286564103531216, \"C(hour_of_week)[40]\": 20.108868345615566, \"C(hour_of_week)[41]\": 19.020782560636103, \"C(hour_of_week)[42]\": 25.186836271766055, \"C(hour_of_week)[43]\": 25.231605568467, \"C(hour_of_week)[44]\": 25.68068654886593, \"C(hour_of_week)[45]\": 26.132197428360264, \"C(hour_of_week)[46]\": 26.37799859851469, \"C(hour_of_week)[47]\": 25.85296071253909, \"C(hour_of_week)[48]\": 25.839234609335143, \"C(hour_of_week)[49]\": 26.002942227535875, \"C(hour_of_week)[50]\": 26.153274305211607, \"C(hour_of_week)[51]\": 26.410947829373985, \"C(hour_of_week)[52]\": 26.503519454689403, \"C(hour_of_week)[53]\": 26.39823420438387, \"C(hour_of_week)[54]\": 26.654285619701792, \"C(hour_of_week)[55]\": 27.267488161727904, \"C(hour_of_week)[56]\": 19.702287002441388, \"C(hour_of_week)[57]\": 20.439633668254416, \"C(hour_of_week)[58]\": 20.952164530259584, \"C(hour_of_week)[59]\": 20.90481054773177, \"C(hour_of_week)[60]\": 20.82284900132815, \"C(hour_of_week)[61]\": 20.703873695015975, \"C(hour_of_week)[62]\": 20.373740143900303, \"C(hour_of_week)[63]\": 19.87481878149373, \"C(hour_of_week)[64]\": 19.777626529359075, \"C(hour_of_week)[65]\": 27.079096581432772, \"C(hour_of_week)[66]\": 25.304126482436605, \"C(hour_of_week)[67]\": 25.3931014640617, \"C(hour_of_week)[68]\": 25.75805379462641, \"C(hour_of_week)[69]\": 26.00883536393592, \"C(hour_of_week)[70]\": 26.145793213753475, \"C(hour_of_week)[71]\": 25.624694317513363, \"C(hour_of_week)[72]\": 25.641119677886422, \"C(hour_of_week)[73]\": 25.89502225124429, \"C(hour_of_week)[74]\": 26.256414332705667, \"C(hour_of_week)[75]\": 26.156061266557703, \"C(hour_of_week)[76]\": 26.485210179001562, \"C(hour_of_week)[77]\": 26.31274749660708, \"C(hour_of_week)[78]\": 25.69186355711933, \"C(hour_of_week)[79]\": 26.807108835197273, \"C(hour_of_week)[80]\": 19.161329966185352, \"C(hour_of_week)[81]\": 20.112123497154972, \"C(hour_of_week)[82]\": 20.920786571207856, \"C(hour_of_week)[83]\": 21.090813186650017, \"C(hour_of_week)[84]\": 21.070696747648434, \"C(hour_of_week)[85]\": 20.94206994010004, \"C(hour_of_week)[86]\": 20.831455299383503, \"C(hour_of_week)[87]\": 20.432770153818296, \"C(hour_of_week)[88]\": 19.837208215885322, \"C(hour_of_week)[89]\": 18.720700060722002, \"C(hour_of_week)[90]\": 24.51941679164439, \"C(hour_of_week)[91]\": 24.785106689742236, \"C(hour_of_week)[92]\": 25.22361610777215, \"C(hour_of_week)[93]\": 25.5718042399158, \"C(hour_of_week)[94]\": 25.93375658716609, \"C(hour_of_week)[95]\": 26.004916011817755, \"C(hour_of_week)[96]\": 26.117458375109976, \"C(hour_of_week)[97]\": 26.390760060647345, \"C(hour_of_week)[98]\": 26.583149354123055, \"C(hour_of_week)[99]\": 26.674709875082932, \"C(hour_of_week)[100]\": 26.796303063862418, \"C(hour_of_week)[101]\": 26.589072385184664, \"C(hour_of_week)[102]\": 26.01109473869544, \"C(hour_of_week)[103]\": 26.55934039477706, \"C(hour_of_week)[104]\": 26.97720851109976, \"C(hour_of_week)[105]\": 20.472139921322494, \"C(hour_of_week)[106]\": 21.085520239088996, \"C(hour_of_week)[107]\": 20.95821062000301, \"C(hour_of_week)[108]\": 20.91044990325138, \"C(hour_of_week)[109]\": 20.93548230212962, \"C(hour_of_week)[110]\": 20.702062830802152, \"C(hour_of_week)[111]\": 20.49225181314433, \"C(hour_of_week)[112]\": 20.077691804450016, \"C(hour_of_week)[113]\": 19.08718431225351, \"C(hour_of_week)[114]\": 25.123592460839312, \"C(hour_of_week)[115]\": 25.47596709685693, \"C(hour_of_week)[116]\": 25.86232217086545, \"C(hour_of_week)[117]\": 26.550615497284202, \"C(hour_of_week)[118]\": 26.625727065370388, \"C(hour_of_week)[119]\": 26.18060374869981, \"C(hour_of_week)[120]\": 26.608854584293685, \"C(hour_of_week)[121]\": 26.99762174039028, \"C(hour_of_week)[122]\": 27.231821143439362, \"C(hour_of_week)[123]\": 27.265092129919413, \"C(hour_of_week)[124]\": 27.547731360871442, \"C(hour_of_week)[125]\": 20.840440740529317, \"C(hour_of_week)[126]\": 26.839521214450155, \"C(hour_of_week)[127]\": 26.164765689817372, \"C(hour_of_week)[128]\": 25.773710108487254, \"C(hour_of_week)[129]\": 25.346179887862448, \"C(hour_of_week)[130]\": 24.8040600073316, \"C(hour_of_week)[131]\": 24.641922542738467, \"C(hour_of_week)[132]\": 25.54459541735725, \"C(hour_of_week)[133]\": 26.372081181953043, \"C(hour_of_week)[134]\": 26.195537179065585, \"C(hour_of_week)[135]\": 26.30351765717446, \"C(hour_of_week)[136]\": 26.391721538050447, \"C(hour_of_week)[137]\": 26.451718244242794, \"C(hour_of_week)[138]\": 25.71952640630832, \"C(hour_of_week)[139]\": 25.30660033113384, \"C(hour_of_week)[140]\": 26.024688670754642, \"C(hour_of_week)[141]\": 26.66582880037223, \"C(hour_of_week)[142]\": 26.37878321156504, \"C(hour_of_week)[143]\": 26.751227952496425, \"C(hour_of_week)[144]\": 26.628604281963653, \"C(hour_of_week)[145]\": 26.792090771626146, \"C(hour_of_week)[146]\": 26.99211375215611, \"C(hour_of_week)[147]\": 27.498165622667354, \"C(hour_of_week)[148]\": 27.319734054294454, \"C(hour_of_week)[149]\": 27.739273338832692, \"C(hour_of_week)[150]\": 26.905485165889754, \"C(hour_of_week)[151]\": 26.648602916498753, \"C(hour_of_week)[152]\": 25.979104063954694, \"C(hour_of_week)[153]\": 25.13385773425032, \"C(hour_of_week)[154]\": 24.725603503812223, \"C(hour_of_week)[155]\": 24.278149962880693, \"C(hour_of_week)[156]\": 24.099580828346195, \"C(hour_of_week)[157]\": 23.84554253500937, \"C(hour_of_week)[158]\": 23.59797133693031, \"C(hour_of_week)[159]\": 23.933947926820576, \"C(hour_of_week)[160]\": 24.06488323890634, \"C(hour_of_week)[161]\": 24.00375930205891, \"C(hour_of_week)[162]\": 24.263083681926314, \"C(hour_of_week)[163]\": 24.277690665334394, \"C(hour_of_week)[164]\": 25.56488867200746, \"C(hour_of_week)[165]\": 26.04189754518007, \"C(hour_of_week)[166]\": 26.212838622203574, \"C(hour_of_week)[167]\": 26.326027800924063, \"bin_0_occupied\": -0.33037572797740017, \"bin_1_occupied\": 0.016895543781330024, \"bin_2_occupied\": 0.03901815834378147, \"bin_3_occupied\": 0.10537627117913564, \"bin_4_occupied\": 0.15222368750805618, \"bin_0_unoccupied\": -0.3768106512847225, \"bin_1_unoccupied\": -0.5165553933295964, \"bin_2_unoccupied\": -0.45010531099154466, \"bin_3_unoccupied\": -0.09471166401732745, \"bin_4_unoccupied\": 0.10978579316320011, \"bin_5_unoccupied\": 0.22803938324709278}}, {\"segment_name\": \"may-jun-jul-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 60.67650222249238, \"C(hour_of_week)[1]\": 60.41615726825565, \"C(hour_of_week)[2]\": 60.56919699780234, \"C(hour_of_week)[3]\": 60.54327696233125, \"C(hour_of_week)[4]\": 60.51813496364416, \"C(hour_of_week)[5]\": 60.07959813480906, \"C(hour_of_week)[6]\": 61.01089807260293, \"C(hour_of_week)[7]\": 10.496281621340083, \"C(hour_of_week)[8]\": 11.551539968320114, \"C(hour_of_week)[9]\": 12.574809200315517, \"C(hour_of_week)[10]\": 13.439553601055259, \"C(hour_of_week)[11]\": 13.447531109359979, \"C(hour_of_week)[12]\": 13.038852679153836, \"C(hour_of_week)[13]\": 13.128031287525578, \"C(hour_of_week)[14]\": 12.848683863330688, \"C(hour_of_week)[15]\": 12.420163460708771, \"C(hour_of_week)[16]\": 12.093431686083136, \"C(hour_of_week)[17]\": 10.781624419804842, \"C(hour_of_week)[18]\": 61.62419468469082, \"C(hour_of_week)[19]\": 61.15215404192815, \"C(hour_of_week)[20]\": 61.311892828157035, \"C(hour_of_week)[21]\": 61.65154481923449, \"C(hour_of_week)[22]\": 61.17931221167856, \"C(hour_of_week)[23]\": 60.605960822988116, \"C(hour_of_week)[24]\": 60.48749973691623, \"C(hour_of_week)[25]\": 60.676057139002516, \"C(hour_of_week)[26]\": 60.73149943255651, \"C(hour_of_week)[27]\": 60.67629561739463, \"C(hour_of_week)[28]\": 60.632034036986056, \"C(hour_of_week)[29]\": 60.29463149375192, \"C(hour_of_week)[30]\": 60.54177823206151, \"C(hour_of_week)[31]\": 10.24029358342246, \"C(hour_of_week)[32]\": 11.113915065162196, \"C(hour_of_week)[33]\": 12.378758825065303, \"C(hour_of_week)[34]\": 13.021778477471052, \"C(hour_of_week)[35]\": 12.985859972067773, \"C(hour_of_week)[36]\": 13.01805799988418, \"C(hour_of_week)[37]\": 12.911494400238302, \"C(hour_of_week)[38]\": 12.602396034555971, \"C(hour_of_week)[39]\": 12.445214297663174, \"C(hour_of_week)[40]\": 11.990414758971365, \"C(hour_of_week)[41]\": 10.650295582228507, \"C(hour_of_week)[42]\": 61.01257826063294, \"C(hour_of_week)[43]\": 60.244329700533974, \"C(hour_of_week)[44]\": 60.26222933752997, \"C(hour_of_week)[45]\": 60.797013370521974, \"C(hour_of_week)[46]\": 60.35302542264533, \"C(hour_of_week)[47]\": 59.84931082920367, \"C(hour_of_week)[48]\": 59.61755666468906, \"C(hour_of_week)[49]\": 59.73499480137057, \"C(hour_of_week)[50]\": 59.677079103681095, \"C(hour_of_week)[51]\": 59.659700958645736, \"C(hour_of_week)[52]\": 59.87401940592316, \"C(hour_of_week)[53]\": 59.59385906744005, \"C(hour_of_week)[54]\": 60.161990676624306, \"C(hour_of_week)[55]\": 9.714265287676835, \"C(hour_of_week)[56]\": 11.021608092015974, \"C(hour_of_week)[57]\": 12.203021668587295, \"C(hour_of_week)[58]\": 12.851261566082647, \"C(hour_of_week)[59]\": 12.979001889910284, \"C(hour_of_week)[60]\": 12.833224194181913, \"C(hour_of_week)[61]\": 12.729352201221701, \"C(hour_of_week)[62]\": 12.586692498361575, \"C(hour_of_week)[63]\": 12.26526532363907, \"C(hour_of_week)[64]\": 11.954647735080119, \"C(hour_of_week)[65]\": 10.664518656870976, \"C(hour_of_week)[66]\": 61.815708840699735, \"C(hour_of_week)[67]\": 60.83634628040101, \"C(hour_of_week)[68]\": 60.706176783278, \"C(hour_of_week)[69]\": 60.95048601153016, \"C(hour_of_week)[70]\": 60.48605555362514, \"C(hour_of_week)[71]\": 59.91614041366424, \"C(hour_of_week)[72]\": 59.761095472551794, \"C(hour_of_week)[73]\": 60.013429340400016, \"C(hour_of_week)[74]\": 60.17746460940701, \"C(hour_of_week)[75]\": 60.119328864710525, \"C(hour_of_week)[76]\": 60.293503577681186, \"C(hour_of_week)[77]\": 59.8617298049054, \"C(hour_of_week)[78]\": 59.924317236326885, \"C(hour_of_week)[79]\": 9.644211301030907, \"C(hour_of_week)[80]\": 10.88969749468703, \"C(hour_of_week)[81]\": 12.022132530450477, \"C(hour_of_week)[82]\": 12.909880926342245, \"C(hour_of_week)[83]\": 13.014643723287275, \"C(hour_of_week)[84]\": 12.979356370349485, \"C(hour_of_week)[85]\": 12.780678240210719, \"C(hour_of_week)[86]\": 12.55777040936484, \"C(hour_of_week)[87]\": 12.317913551915105, \"C(hour_of_week)[88]\": 11.67382348519997, \"C(hour_of_week)[89]\": 10.455729135607472, \"C(hour_of_week)[90]\": 60.89462573265589, \"C(hour_of_week)[91]\": 60.33096947760878, \"C(hour_of_week)[92]\": 60.355212048621155, \"C(hour_of_week)[93]\": 60.71708260521646, \"C(hour_of_week)[94]\": 60.451938458612325, \"C(hour_of_week)[95]\": 60.17023122914937, \"C(hour_of_week)[96]\": 60.07810941290483, \"C(hour_of_week)[97]\": 60.27023867521617, \"C(hour_of_week)[98]\": 60.44132937090376, \"C(hour_of_week)[99]\": 60.46628591715299, \"C(hour_of_week)[100]\": 60.51481253527568, \"C(hour_of_week)[101]\": 60.17622852633339, \"C(hour_of_week)[102]\": 60.12964569904602, \"C(hour_of_week)[103]\": 10.161771732146853, \"C(hour_of_week)[104]\": 11.093660184718221, \"C(hour_of_week)[105]\": 12.153239391511189, \"C(hour_of_week)[106]\": 12.980735545584574, \"C(hour_of_week)[107]\": 13.022951715784332, \"C(hour_of_week)[108]\": 12.779569569958396, \"C(hour_of_week)[109]\": 12.776138039460646, \"C(hour_of_week)[110]\": 12.695124961533525, \"C(hour_of_week)[111]\": 12.528975044967314, \"C(hour_of_week)[112]\": 12.039864316190881, \"C(hour_of_week)[113]\": 10.766999815109605, \"C(hour_of_week)[114]\": 60.859929293281525, \"C(hour_of_week)[115]\": 60.352244284844915, \"C(hour_of_week)[116]\": 60.42675831148539, \"C(hour_of_week)[117]\": 60.92752400844525, \"C(hour_of_week)[118]\": 60.45373728918682, \"C(hour_of_week)[119]\": 60.017984775612106, \"C(hour_of_week)[120]\": 60.38956488956445, \"C(hour_of_week)[121]\": 60.759128252267296, \"C(hour_of_week)[122]\": 60.91559325372759, \"C(hour_of_week)[123]\": 60.99469863862249, \"C(hour_of_week)[124]\": 61.106248471587875, \"C(hour_of_week)[125]\": 60.73010081187387, \"C(hour_of_week)[126]\": 60.234494618160554, \"C(hour_of_week)[127]\": 59.78096948038826, \"C(hour_of_week)[128]\": 59.55201658438834, \"C(hour_of_week)[129]\": 59.547432298702205, \"C(hour_of_week)[130]\": 59.49759622221076, \"C(hour_of_week)[131]\": 59.5688094670692, \"C(hour_of_week)[132]\": 61.2698132148873, \"C(hour_of_week)[133]\": 9.948181034481177, \"C(hour_of_week)[134]\": 62.57421636805126, \"C(hour_of_week)[135]\": 62.63883118498022, \"C(hour_of_week)[136]\": 62.573012502350636, \"C(hour_of_week)[137]\": 10.00797977879207, \"C(hour_of_week)[138]\": 61.4387669039969, \"C(hour_of_week)[139]\": 60.46405171344144, \"C(hour_of_week)[140]\": 60.401760456019225, \"C(hour_of_week)[141]\": 60.77425931700524, \"C(hour_of_week)[142]\": 60.361412119049774, \"C(hour_of_week)[143]\": 60.51066993619704, \"C(hour_of_week)[144]\": 60.4202778156332, \"C(hour_of_week)[145]\": 60.47420148645551, \"C(hour_of_week)[146]\": 60.523397305160515, \"C(hour_of_week)[147]\": 60.76512466692259, \"C(hour_of_week)[148]\": 60.49488052823777, \"C(hour_of_week)[149]\": 60.51731427220481, \"C(hour_of_week)[150]\": 59.715315344683056, \"C(hour_of_week)[151]\": 59.864318387509385, \"C(hour_of_week)[152]\": 59.97744725484236, \"C(hour_of_week)[153]\": 60.00379045198551, \"C(hour_of_week)[154]\": 59.98791256467151, \"C(hour_of_week)[155]\": 59.68118744783514, \"C(hour_of_week)[156]\": 59.59927338810505, \"C(hour_of_week)[157]\": 59.549460201486916, \"C(hour_of_week)[158]\": 59.67809756872401, \"C(hour_of_week)[159]\": 60.02988638432881, \"C(hour_of_week)[160]\": 60.05005397345205, \"C(hour_of_week)[161]\": 59.78137420724008, \"C(hour_of_week)[162]\": 59.84918607726996, \"C(hour_of_week)[163]\": 59.61925297319014, \"C(hour_of_week)[164]\": 60.103651498209295, \"C(hour_of_week)[165]\": 60.26664980110008, \"C(hour_of_week)[166]\": 60.23539797374825, \"C(hour_of_week)[167]\": 60.43285637595931, \"bin_0_occupied\": -0.1180231413287284, \"bin_1_occupied\": 0.035710448406035544, \"bin_2_occupied\": 0.13256090210357388, \"bin_3_occupied\": 0.15161162776715645, \"bin_0_unoccupied\": -1.1879322949037743, \"bin_1_unoccupied\": -0.4572954874548413, \"bin_2_unoccupied\": -0.07465341460632846, \"bin_3_unoccupied\": 0.05387622395435813, \"bin_4_unoccupied\": 0.20783973236877415}}, {\"segment_name\": \"jun-jul-aug-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 9.690608528702803, \"C(hour_of_week)[1]\": 9.486031769326488, \"C(hour_of_week)[2]\": 9.476208657812649, \"C(hour_of_week)[3]\": 9.469355697109204, \"C(hour_of_week)[4]\": 9.48020848789061, \"C(hour_of_week)[5]\": 9.138654537755862, \"C(hour_of_week)[6]\": 10.50139521103516, \"C(hour_of_week)[7]\": 3.026473487878018, \"C(hour_of_week)[8]\": 4.164834483329621, \"C(hour_of_week)[9]\": 5.159964341155153, \"C(hour_of_week)[10]\": 6.120641568612342, \"C(hour_of_week)[11]\": 6.189967044658065, \"C(hour_of_week)[12]\": 5.841975234502378, \"C(hour_of_week)[13]\": 5.963738810878539, \"C(hour_of_week)[14]\": 5.606135409137146, \"C(hour_of_week)[15]\": 5.259871821794395, \"C(hour_of_week)[16]\": 4.890277886188841, \"C(hour_of_week)[17]\": 3.4018480143955045, \"C(hour_of_week)[18]\": 12.453251669378911, \"C(hour_of_week)[19]\": 11.41754419845419, \"C(hour_of_week)[20]\": 11.047692201357775, \"C(hour_of_week)[21]\": 11.262842984575432, \"C(hour_of_week)[22]\": 10.223110212847795, \"C(hour_of_week)[23]\": 9.599073921600011, \"C(hour_of_week)[24]\": 9.34731468560463, \"C(hour_of_week)[25]\": 9.492907687062704, \"C(hour_of_week)[26]\": 9.494584418508275, \"C(hour_of_week)[27]\": 9.44207223436857, \"C(hour_of_week)[28]\": 9.380749478287083, \"C(hour_of_week)[29]\": 9.133539510588065, \"C(hour_of_week)[30]\": 10.176607609148979, \"C(hour_of_week)[31]\": 2.7125564560188877, \"C(hour_of_week)[32]\": 3.856894574221677, \"C(hour_of_week)[33]\": 5.167560741511714, \"C(hour_of_week)[34]\": 5.669139661098907, \"C(hour_of_week)[35]\": 5.84305365463517, \"C(hour_of_week)[36]\": 5.853302020012606, \"C(hour_of_week)[37]\": 5.554538909985226, \"C(hour_of_week)[38]\": 5.266390895866113, \"C(hour_of_week)[39]\": 5.345703784077667, \"C(hour_of_week)[40]\": 4.509612428006058, \"C(hour_of_week)[41]\": 3.2221254766347123, \"C(hour_of_week)[42]\": 11.857196362604089, \"C(hour_of_week)[43]\": 10.575819332825622, \"C(hour_of_week)[44]\": 10.723007104193108, \"C(hour_of_week)[45]\": 11.085197494664634, \"C(hour_of_week)[46]\": 9.985435461042863, \"C(hour_of_week)[47]\": 9.590830858726525, \"C(hour_of_week)[48]\": 9.383640686105913, \"C(hour_of_week)[49]\": 9.51452972507969, \"C(hour_of_week)[50]\": 9.669717249702693, \"C(hour_of_week)[51]\": 9.539717582598705, \"C(hour_of_week)[52]\": 9.495832425852477, \"C(hour_of_week)[53]\": 9.164968818982377, \"C(hour_of_week)[54]\": 9.636261136588782, \"C(hour_of_week)[55]\": 2.4733069168080632, \"C(hour_of_week)[56]\": 3.952557754659086, \"C(hour_of_week)[57]\": 4.824373653422509, \"C(hour_of_week)[58]\": 5.681078364080193, \"C(hour_of_week)[59]\": 5.956278599405682, \"C(hour_of_week)[60]\": 5.599689469151592, \"C(hour_of_week)[61]\": 5.585886044991774, \"C(hour_of_week)[62]\": 5.60941065121418, \"C(hour_of_week)[63]\": 5.23083254754291, \"C(hour_of_week)[64]\": 4.750825129653364, \"C(hour_of_week)[65]\": 3.501635328022452, \"C(hour_of_week)[66]\": 12.195487038116822, \"C(hour_of_week)[67]\": 10.99388884199698, \"C(hour_of_week)[68]\": 10.861581140504214, \"C(hour_of_week)[69]\": 10.922154358626887, \"C(hour_of_week)[70]\": 10.135150867103194, \"C(hour_of_week)[71]\": 9.500744327032876, \"C(hour_of_week)[72]\": 9.24702844055628, \"C(hour_of_week)[73]\": 9.412008798620079, \"C(hour_of_week)[74]\": 9.459768313031066, \"C(hour_of_week)[75]\": 9.391993800952102, \"C(hour_of_week)[76]\": 9.400803430986365, \"C(hour_of_week)[77]\": 9.112216401776855, \"C(hour_of_week)[78]\": 9.807952604036341, \"C(hour_of_week)[79]\": 2.430110382512389, \"C(hour_of_week)[80]\": 3.5931153924017565, \"C(hour_of_week)[81]\": 4.861792653484315, \"C(hour_of_week)[82]\": 5.680447383180336, \"C(hour_of_week)[83]\": 5.644504188007382, \"C(hour_of_week)[84]\": 5.732866384921891, \"C(hour_of_week)[85]\": 5.526078557123668, \"C(hour_of_week)[86]\": 5.178891601192831, \"C(hour_of_week)[87]\": 5.256626728770961, \"C(hour_of_week)[88]\": 4.594650921690995, \"C(hour_of_week)[89]\": 3.077187801022447, \"C(hour_of_week)[90]\": 11.781757562222172, \"C(hour_of_week)[91]\": 10.667844821366698, \"C(hour_of_week)[92]\": 10.64264271431917, \"C(hour_of_week)[93]\": 11.007679754798186, \"C(hour_of_week)[94]\": 9.96951247026564, \"C(hour_of_week)[95]\": 9.548436243251839, \"C(hour_of_week)[96]\": 9.339139776692813, \"C(hour_of_week)[97]\": 9.506651461945825, \"C(hour_of_week)[98]\": 9.576879124520953, \"C(hour_of_week)[99]\": 9.581288894559504, \"C(hour_of_week)[100]\": 9.684318698467631, \"C(hour_of_week)[101]\": 9.321861675995034, \"C(hour_of_week)[102]\": 9.788217124642216, \"C(hour_of_week)[103]\": 2.631414963633457, \"C(hour_of_week)[104]\": 3.9609344622914726, \"C(hour_of_week)[105]\": 4.872482505078288, \"C(hour_of_week)[106]\": 5.762052469888751, \"C(hour_of_week)[107]\": 5.897379504296278, \"C(hour_of_week)[108]\": 5.535621431254118, \"C(hour_of_week)[109]\": 5.511556778330792, \"C(hour_of_week)[110]\": 5.6140072708471465, \"C(hour_of_week)[111]\": 5.210474108377973, \"C(hour_of_week)[112]\": 4.708922146791419, \"C(hour_of_week)[113]\": 3.405571187308798, \"C(hour_of_week)[114]\": 11.553166435928183, \"C(hour_of_week)[115]\": 10.686937636858392, \"C(hour_of_week)[116]\": 10.654737928003918, \"C(hour_of_week)[117]\": 10.734639091209036, \"C(hour_of_week)[118]\": 10.077109386907573, \"C(hour_of_week)[119]\": 9.564076499523331, \"C(hour_of_week)[120]\": 9.945349984751845, \"C(hour_of_week)[121]\": 10.313001000789182, \"C(hour_of_week)[122]\": 10.447956489496498, \"C(hour_of_week)[123]\": 10.496011912851976, \"C(hour_of_week)[124]\": 10.47768107691954, \"C(hour_of_week)[125]\": 10.144246176063962, \"C(hour_of_week)[126]\": 9.544628575078887, \"C(hour_of_week)[127]\": 9.196818787202142, \"C(hour_of_week)[128]\": 8.98860014225951, \"C(hour_of_week)[129]\": 9.027471205179914, \"C(hour_of_week)[130]\": 9.195754180851015, \"C(hour_of_week)[131]\": 9.34070051864865, \"C(hour_of_week)[132]\": 11.529954506448561, \"C(hour_of_week)[133]\": 2.6665388362130926, \"C(hour_of_week)[134]\": 12.826694733110468, \"C(hour_of_week)[135]\": 12.884832097550618, \"C(hour_of_week)[136]\": 12.75847742148832, \"C(hour_of_week)[137]\": 12.734913480618829, \"C(hour_of_week)[138]\": 11.533113534084674, \"C(hour_of_week)[139]\": 10.596478174191892, \"C(hour_of_week)[140]\": 10.331590514517611, \"C(hour_of_week)[141]\": 10.334936189018215, \"C(hour_of_week)[142]\": 9.948445833852459, \"C(hour_of_week)[143]\": 10.035821696596692, \"C(hour_of_week)[144]\": 9.52134694732889, \"C(hour_of_week)[145]\": 9.402793377863794, \"C(hour_of_week)[146]\": 9.421615039160972, \"C(hour_of_week)[147]\": 9.443651353701934, \"C(hour_of_week)[148]\": 9.464382403437227, \"C(hour_of_week)[149]\": 9.24388023993942, \"C(hour_of_week)[150]\": 8.660613700100452, \"C(hour_of_week)[151]\": 9.153641335414093, \"C(hour_of_week)[152]\": 9.4746885190151, \"C(hour_of_week)[153]\": 9.763445103133677, \"C(hour_of_week)[154]\": 9.82295234528213, \"C(hour_of_week)[155]\": 9.57141777282848, \"C(hour_of_week)[156]\": 9.514246878104686, \"C(hour_of_week)[157]\": 9.558029637806825, \"C(hour_of_week)[158]\": 9.849898383891663, \"C(hour_of_week)[159]\": 10.21045202267707, \"C(hour_of_week)[160]\": 10.3377426165394, \"C(hour_of_week)[161]\": 10.039732129291064, \"C(hour_of_week)[162]\": 9.802109129494273, \"C(hour_of_week)[163]\": 9.381721622331458, \"C(hour_of_week)[164]\": 9.519178632178523, \"C(hour_of_week)[165]\": 9.484588701932717, \"C(hour_of_week)[166]\": 9.459396395565031, \"C(hour_of_week)[167]\": 9.549925393707973, \"bin_0_occupied\": 0.01905918165387921, \"bin_1_occupied\": 0.1519842620276844, \"bin_2_occupied\": 0.13554187373401674, \"bin_0_unoccupied\": -0.14052662682892825, \"bin_1_unoccupied\": -0.053106211540812034, \"bin_2_unoccupied\": 0.04475248775873241, \"bin_3_unoccupied\": 0.18459889342587046}}, {\"segment_name\": \"jul-aug-sep-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 16.37987014202073, \"C(hour_of_week)[1]\": 16.15764192634435, \"C(hour_of_week)[2]\": 16.12726627366887, \"C(hour_of_week)[3]\": 16.064562550298902, \"C(hour_of_week)[4]\": 16.013834877663324, \"C(hour_of_week)[5]\": 15.901576717792164, \"C(hour_of_week)[6]\": 17.48766220063024, \"C(hour_of_week)[7]\": -0.13081474027286788, \"C(hour_of_week)[8]\": 1.004395911684945, \"C(hour_of_week)[9]\": 1.9961886062205245, \"C(hour_of_week)[10]\": 3.043108028885957, \"C(hour_of_week)[11]\": 3.0922399814005566, \"C(hour_of_week)[12]\": 2.7990836914463415, \"C(hour_of_week)[13]\": 2.8837186611263625, \"C(hour_of_week)[14]\": 2.508957090378118, \"C(hour_of_week)[15]\": 2.1464256202268928, \"C(hour_of_week)[16]\": 1.7406396648914217, \"C(hour_of_week)[17]\": 0.22128525924161835, \"C(hour_of_week)[18]\": 19.248786605754013, \"C(hour_of_week)[19]\": 18.260987822605692, \"C(hour_of_week)[20]\": 17.875533498326714, \"C(hour_of_week)[21]\": 18.091760004835194, \"C(hour_of_week)[22]\": 16.811670771140125, \"C(hour_of_week)[23]\": 16.220060938367475, \"C(hour_of_week)[24]\": 15.961164004862113, \"C(hour_of_week)[25]\": 16.10762216587754, \"C(hour_of_week)[26]\": 16.090760983479587, \"C(hour_of_week)[27]\": 16.023280380147362, \"C(hour_of_week)[28]\": 15.99108464334176, \"C(hour_of_week)[29]\": 15.961821871054825, \"C(hour_of_week)[30]\": 17.23352302430612, \"C(hour_of_week)[31]\": -0.5919856540579111, \"C(hour_of_week)[32]\": 0.7011756614081719, \"C(hour_of_week)[33]\": 2.068518903698477, \"C(hour_of_week)[34]\": 2.4827684332781335, \"C(hour_of_week)[35]\": 2.7640131395403174, \"C(hour_of_week)[36]\": 2.8207280302667748, \"C(hour_of_week)[37]\": 2.394020635230076, \"C(hour_of_week)[38]\": 2.183827743566116, \"C(hour_of_week)[39]\": 2.2707045773430146, \"C(hour_of_week)[40]\": 1.1946955537737693, \"C(hour_of_week)[41]\": -0.014568046172136917, \"C(hour_of_week)[42]\": 18.754923767801387, \"C(hour_of_week)[43]\": 17.47360992820215, \"C(hour_of_week)[44]\": 17.813560117824238, \"C(hour_of_week)[45]\": 17.970261421720185, \"C(hour_of_week)[46]\": 16.683780707287454, \"C(hour_of_week)[47]\": 16.378119631701445, \"C(hour_of_week)[48]\": 16.130373570719552, \"C(hour_of_week)[49]\": 16.236576845979748, \"C(hour_of_week)[50]\": 16.454077893270988, \"C(hour_of_week)[51]\": 16.20954024917702, \"C(hour_of_week)[52]\": 16.066420683976137, \"C(hour_of_week)[53]\": 15.999480782509423, \"C(hour_of_week)[54]\": 16.473691179637882, \"C(hour_of_week)[55]\": -0.6171564986047997, \"C(hour_of_week)[56]\": 0.8947708503884328, \"C(hour_of_week)[57]\": 1.6625695639954898, \"C(hour_of_week)[58]\": 2.746240452513886, \"C(hour_of_week)[59]\": 3.062912892188155, \"C(hour_of_week)[60]\": 2.5726640186117855, \"C(hour_of_week)[61]\": 2.679118204376252, \"C(hour_of_week)[62]\": 2.729948829056415, \"C(hour_of_week)[63]\": 2.186606025240854, \"C(hour_of_week)[64]\": 1.7024696928862078, \"C(hour_of_week)[65]\": 0.43636043454694473, \"C(hour_of_week)[66]\": 18.574063896354424, \"C(hour_of_week)[67]\": 17.808542492328648, \"C(hour_of_week)[68]\": 17.647560513747493, \"C(hour_of_week)[69]\": 17.55100604028615, \"C(hour_of_week)[70]\": 16.950629114685103, \"C(hour_of_week)[71]\": 16.284554491329025, \"C(hour_of_week)[72]\": 16.020217756499363, \"C(hour_of_week)[73]\": 16.124770033721557, \"C(hour_of_week)[74]\": 16.171047837860062, \"C(hour_of_week)[75]\": 16.115435776125786, \"C(hour_of_week)[76]\": 16.1064462353893, \"C(hour_of_week)[77]\": 16.061127203232967, \"C(hour_of_week)[78]\": 16.732758652040886, \"C(hour_of_week)[79]\": -0.5203886476926733, \"C(hour_of_week)[80]\": 0.491438291421737, \"C(hour_of_week)[81]\": 1.9797083199230254, \"C(hour_of_week)[82]\": 2.7170688960673406, \"C(hour_of_week)[83]\": 2.646991473368331, \"C(hour_of_week)[84]\": 2.8484108596531197, \"C(hour_of_week)[85]\": 2.6157573207441587, \"C(hour_of_week)[86]\": 2.2572898901360787, \"C(hour_of_week)[87]\": 2.498241241512, \"C(hour_of_week)[88]\": 1.8015187435402495, \"C(hour_of_week)[89]\": 0.17147395475086924, \"C(hour_of_week)[90]\": 18.816353587395916, \"C(hour_of_week)[91]\": 17.551063425252515, \"C(hour_of_week)[92]\": 17.513325537024336, \"C(hour_of_week)[93]\": 17.87293153159593, \"C(hour_of_week)[94]\": 16.633980896368417, \"C(hour_of_week)[95]\": 16.231094897405754, \"C(hour_of_week)[96]\": 15.991416471422452, \"C(hour_of_week)[97]\": 16.148055221140936, \"C(hour_of_week)[98]\": 16.18178321954306, \"C(hour_of_week)[99]\": 16.203751447286248, \"C(hour_of_week)[100]\": 16.361800161960623, \"C(hour_of_week)[101]\": 16.18228194835862, \"C(hour_of_week)[102]\": 16.655491189016164, \"C(hour_of_week)[103]\": -0.6845188122180161, \"C(hour_of_week)[104]\": 0.8939794221595996, \"C(hour_of_week)[105]\": 1.7180627304854674, \"C(hour_of_week)[106]\": 2.58178715314196, \"C(hour_of_week)[107]\": 2.890769840879912, \"C(hour_of_week)[108]\": 2.5116141516958024, \"C(hour_of_week)[109]\": 2.4863708981695005, \"C(hour_of_week)[110]\": 2.668022407407344, \"C(hour_of_week)[111]\": 2.0900268199597996, \"C(hour_of_week)[112]\": 1.6097795944659303, \"C(hour_of_week)[113]\": 0.4025451529956001, \"C(hour_of_week)[114]\": 18.239615137820984, \"C(hour_of_week)[115]\": 17.609531347877727, \"C(hour_of_week)[116]\": 17.656124662762153, \"C(hour_of_week)[117]\": 17.53032523263551, \"C(hour_of_week)[118]\": 17.043088783790804, \"C(hour_of_week)[119]\": 16.38809692611024, \"C(hour_of_week)[120]\": 16.751514139735786, \"C(hour_of_week)[121]\": 17.063956655200236, \"C(hour_of_week)[122]\": 17.156723849965427, \"C(hour_of_week)[123]\": 17.174265343121164, \"C(hour_of_week)[124]\": 17.12951471221455, \"C(hour_of_week)[125]\": 17.013621609500305, \"C(hour_of_week)[126]\": 16.49184996069778, \"C(hour_of_week)[127]\": 15.962112249943978, \"C(hour_of_week)[128]\": 15.773385487046838, \"C(hour_of_week)[129]\": 15.809415384485, \"C(hour_of_week)[130]\": 15.978388533351259, \"C(hour_of_week)[131]\": 16.120786115957387, \"C(hour_of_week)[132]\": 18.459854578151116, \"C(hour_of_week)[133]\": -0.27356795117660226, \"C(hour_of_week)[134]\": 19.742481605522784, \"C(hour_of_week)[135]\": 19.819979501329364, \"C(hour_of_week)[136]\": -0.4838250483234745, \"C(hour_of_week)[137]\": -0.501422995518741, \"C(hour_of_week)[138]\": 18.270624396075306, \"C(hour_of_week)[139]\": 17.414458684125712, \"C(hour_of_week)[140]\": 17.27213356377706, \"C(hour_of_week)[141]\": 17.114853467043247, \"C(hour_of_week)[142]\": 16.77671715991275, \"C(hour_of_week)[143]\": 16.923653477932447, \"C(hour_of_week)[144]\": 16.326815993756632, \"C(hour_of_week)[145]\": 16.191744581853445, \"C(hour_of_week)[146]\": 16.24423960549348, \"C(hour_of_week)[147]\": 16.258538664060307, \"C(hour_of_week)[148]\": 16.256429067407865, \"C(hour_of_week)[149]\": 16.28708006163922, \"C(hour_of_week)[150]\": 15.861247441115868, \"C(hour_of_week)[151]\": 16.11186787694335, \"C(hour_of_week)[152]\": 16.424848588050054, \"C(hour_of_week)[153]\": 16.779465695717743, \"C(hour_of_week)[154]\": 16.883991709663725, \"C(hour_of_week)[155]\": 16.512503805152676, \"C(hour_of_week)[156]\": 16.520205329414228, \"C(hour_of_week)[157]\": 16.683993703898096, \"C(hour_of_week)[158]\": 16.958591661056033, \"C(hour_of_week)[159]\": 17.339246404469037, \"C(hour_of_week)[160]\": 17.52856290654105, \"C(hour_of_week)[161]\": 17.13510162457645, \"C(hour_of_week)[162]\": 16.651636375902026, \"C(hour_of_week)[163]\": 16.29417887903526, \"C(hour_of_week)[164]\": 16.42422981928218, \"C(hour_of_week)[165]\": 16.292387262759448, \"C(hour_of_week)[166]\": 16.277765043246944, \"C(hour_of_week)[167]\": 16.40872258714465, \"bin_0_occupied\": 0.07126384596382582, \"bin_1_occupied\": 0.11357546592288134, \"bin_2_occupied\": 0.14392311090663848, \"bin_0_unoccupied\": -0.26741671237889175, \"bin_1_unoccupied\": -0.030856837708973746, \"bin_2_unoccupied\": 0.04298660213586147, \"bin_3_unoccupied\": 0.15488918492927964}}, {\"segment_name\": \"aug-sep-oct-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 23.366228293275242, \"C(hour_of_week)[1]\": 23.126631268722353, \"C(hour_of_week)[2]\": 23.137403860760774, \"C(hour_of_week)[3]\": 23.165974714283937, \"C(hour_of_week)[4]\": 23.08227079699936, \"C(hour_of_week)[5]\": 23.138793799434218, \"C(hour_of_week)[6]\": 24.078518807637604, \"C(hour_of_week)[7]\": 19.997699082425566, \"C(hour_of_week)[8]\": 20.64825286007556, \"C(hour_of_week)[9]\": 21.4061968627101, \"C(hour_of_week)[10]\": 22.365204160978223, \"C(hour_of_week)[11]\": 22.496519535913652, \"C(hour_of_week)[12]\": 22.247956481034148, \"C(hour_of_week)[13]\": 22.26691004404814, \"C(hour_of_week)[14]\": 21.939908553645104, \"C(hour_of_week)[15]\": 21.559295953397573, \"C(hour_of_week)[16]\": 21.10824955244883, \"C(hour_of_week)[17]\": 19.792698792480092, \"C(hour_of_week)[18]\": 23.843299457775665, \"C(hour_of_week)[19]\": 23.671220781913753, \"C(hour_of_week)[20]\": 23.555826683662726, \"C(hour_of_week)[21]\": 23.76861539603402, \"C(hour_of_week)[22]\": 23.038193241361732, \"C(hour_of_week)[23]\": 22.543532454496148, \"C(hour_of_week)[24]\": 22.436125147460935, \"C(hour_of_week)[25]\": 22.563301236518075, \"C(hour_of_week)[26]\": 22.706756539937757, \"C(hour_of_week)[27]\": 22.64776529522092, \"C(hour_of_week)[28]\": 22.79576009298976, \"C(hour_of_week)[29]\": 22.798621601789474, \"C(hour_of_week)[30]\": 23.759531067092404, \"C(hour_of_week)[31]\": 19.578815150082495, \"C(hour_of_week)[32]\": 20.34328926168453, \"C(hour_of_week)[33]\": 21.671107476899984, \"C(hour_of_week)[34]\": 22.136276173830208, \"C(hour_of_week)[35]\": 22.25993172253686, \"C(hour_of_week)[36]\": 22.385479302045113, \"C(hour_of_week)[37]\": 22.04615817756068, \"C(hour_of_week)[38]\": 21.789008273992984, \"C(hour_of_week)[39]\": 21.81246323120298, \"C(hour_of_week)[40]\": 20.934330680923445, \"C(hour_of_week)[41]\": 19.649757291585132, \"C(hour_of_week)[42]\": 23.693368945227437, \"C(hour_of_week)[43]\": 23.26527656972177, \"C(hour_of_week)[44]\": 23.466816819879458, \"C(hour_of_week)[45]\": 23.738100976256458, \"C(hour_of_week)[46]\": 22.85380183331455, \"C(hour_of_week)[47]\": 22.57121324821127, \"C(hour_of_week)[48]\": 22.373550883513865, \"C(hour_of_week)[49]\": 22.525320088152892, \"C(hour_of_week)[50]\": 22.70354441009877, \"C(hour_of_week)[51]\": 22.584109663702822, \"C(hour_of_week)[52]\": 22.408504432061985, \"C(hour_of_week)[53]\": 22.50648692079134, \"C(hour_of_week)[54]\": 23.053580563451213, \"C(hour_of_week)[55]\": 19.395344070122395, \"C(hour_of_week)[56]\": 20.50767232650263, \"C(hour_of_week)[57]\": 21.390893809357898, \"C(hour_of_week)[58]\": 22.337513318057688, \"C(hour_of_week)[59]\": 22.671428110517375, \"C(hour_of_week)[60]\": 22.34735053037412, \"C(hour_of_week)[61]\": 22.324814190468274, \"C(hour_of_week)[62]\": 22.308778824903694, \"C(hour_of_week)[63]\": 21.845271306466003, \"C(hour_of_week)[64]\": 21.271974297464563, \"C(hour_of_week)[65]\": 20.037911259492038, \"C(hour_of_week)[66]\": 23.285417650528196, \"C(hour_of_week)[67]\": 23.117321998169334, \"C(hour_of_week)[68]\": 23.087453269906838, \"C(hour_of_week)[69]\": 22.932025007536932, \"C(hour_of_week)[70]\": 22.615011182257565, \"C(hour_of_week)[71]\": 22.180455464271454, \"C(hour_of_week)[72]\": 22.02084741700281, \"C(hour_of_week)[73]\": 22.09019331079449, \"C(hour_of_week)[74]\": 22.23804053508062, \"C(hour_of_week)[75]\": 22.283739054279398, \"C(hour_of_week)[76]\": 22.377485055619672, \"C(hour_of_week)[77]\": 22.397023911607242, \"C(hour_of_week)[78]\": 22.817982954753308, \"C(hour_of_week)[79]\": 19.445324011424148, \"C(hour_of_week)[80]\": 20.317719731703214, \"C(hour_of_week)[81]\": 21.70535522869095, \"C(hour_of_week)[82]\": 22.46693399681773, \"C(hour_of_week)[83]\": 22.453911300466473, \"C(hour_of_week)[84]\": 22.590473529516593, \"C(hour_of_week)[85]\": 22.411492574056282, \"C(hour_of_week)[86]\": 22.146187472929263, \"C(hour_of_week)[87]\": 22.257622772353802, \"C(hour_of_week)[88]\": 21.627142144122697, \"C(hour_of_week)[89]\": 20.131901100687365, \"C(hour_of_week)[90]\": 23.677701747336684, \"C(hour_of_week)[91]\": 23.104613038752127, \"C(hour_of_week)[92]\": 23.120972916159886, \"C(hour_of_week)[93]\": 23.625370792377968, \"C(hour_of_week)[94]\": 22.76497057117175, \"C(hour_of_week)[95]\": 22.475112534683138, \"C(hour_of_week)[96]\": 22.293204707702692, \"C(hour_of_week)[97]\": 22.344914760894284, \"C(hour_of_week)[98]\": 22.488163891056093, \"C(hour_of_week)[99]\": 22.48719593671759, \"C(hour_of_week)[100]\": 22.682255958049502, \"C(hour_of_week)[101]\": 22.73401065535415, \"C(hour_of_week)[102]\": 23.31244498627201, \"C(hour_of_week)[103]\": 19.457787304536307, \"C(hour_of_week)[104]\": 20.669129901510274, \"C(hour_of_week)[105]\": 21.43322750682772, \"C(hour_of_week)[106]\": 22.094811251150926, \"C(hour_of_week)[107]\": 22.39772577896682, \"C(hour_of_week)[108]\": 22.166268296316723, \"C(hour_of_week)[109]\": 22.004791052954296, \"C(hour_of_week)[110]\": 22.072956468214905, \"C(hour_of_week)[111]\": 21.59098856499101, \"C(hour_of_week)[112]\": 21.062013255429367, \"C(hour_of_week)[113]\": 20.04864166954059, \"C(hour_of_week)[114]\": 23.485980160154487, \"C(hour_of_week)[115]\": 23.658839837892096, \"C(hour_of_week)[116]\": 23.644627063495122, \"C(hour_of_week)[117]\": 23.671555812076583, \"C(hour_of_week)[118]\": 23.38352595492042, \"C(hour_of_week)[119]\": 22.79801169539237, \"C(hour_of_week)[120]\": 23.067655871312628, \"C(hour_of_week)[121]\": 23.257319960573646, \"C(hour_of_week)[122]\": 23.409957880552685, \"C(hour_of_week)[123]\": 23.418717573128202, \"C(hour_of_week)[124]\": 23.468376166715775, \"C(hour_of_week)[125]\": 23.374299447062583, \"C(hour_of_week)[126]\": 23.32991905647293, \"C(hour_of_week)[127]\": 22.49463812058477, \"C(hour_of_week)[128]\": 22.069501344754325, \"C(hour_of_week)[129]\": 21.890855378259552, \"C(hour_of_week)[130]\": 21.882696903399193, \"C(hour_of_week)[131]\": 21.859353675824575, \"C(hour_of_week)[132]\": 23.579002408433993, \"C(hour_of_week)[133]\": 24.91633840318825, \"C(hour_of_week)[134]\": 24.812682658938254, \"C(hour_of_week)[135]\": 24.809965239246104, \"C(hour_of_week)[136]\": 24.513850810240825, \"C(hour_of_week)[137]\": 24.52143349997702, \"C(hour_of_week)[138]\": 23.512910470980273, \"C(hour_of_week)[139]\": 23.231248042317077, \"C(hour_of_week)[140]\": 23.167370809322215, \"C(hour_of_week)[141]\": 23.150189758729876, \"C(hour_of_week)[142]\": 22.983914062386347, \"C(hour_of_week)[143]\": 23.263984469654766, \"C(hour_of_week)[144]\": 22.91983866137546, \"C(hour_of_week)[145]\": 22.892051824183554, \"C(hour_of_week)[146]\": 23.050354369105634, \"C(hour_of_week)[147]\": 23.198817774144405, \"C(hour_of_week)[148]\": 23.091426238698755, \"C(hour_of_week)[149]\": 23.373358274777175, \"C(hour_of_week)[150]\": 23.325183390544865, \"C(hour_of_week)[151]\": 23.12427393655535, \"C(hour_of_week)[152]\": 23.006103549808486, \"C(hour_of_week)[153]\": 22.97247763537626, \"C(hour_of_week)[154]\": 22.98033032109599, \"C(hour_of_week)[155]\": 22.4778608414167, \"C(hour_of_week)[156]\": 22.583536893177044, \"C(hour_of_week)[157]\": 22.77022032363438, \"C(hour_of_week)[158]\": 22.95215902263051, \"C(hour_of_week)[159]\": 23.238031595501177, \"C(hour_of_week)[160]\": 23.382534276603256, \"C(hour_of_week)[161]\": 23.054326830274718, \"C(hour_of_week)[162]\": 22.851021002648523, \"C(hour_of_week)[163]\": 22.995882154598554, \"C(hour_of_week)[164]\": 23.1360361473538, \"C(hour_of_week)[165]\": 23.012436403964674, \"C(hour_of_week)[166]\": 23.166174086057786, \"C(hour_of_week)[167]\": 23.304412856322813, \"bin_0_occupied\": -0.3592009167601687, \"bin_1_occupied\": 0.015247893689167305, \"bin_2_occupied\": 0.09272818647205205, \"bin_3_occupied\": 0.1043730755194128, \"bin_4_occupied\": 0.17052095527619673, \"bin_0_unoccupied\": -0.3896117828283877, \"bin_1_unoccupied\": -0.30657109823613604, \"bin_2_unoccupied\": -0.06977012927096878, \"bin_3_unoccupied\": 0.06573835148384805, \"bin_4_unoccupied\": 0.15641711821939408}}, {\"segment_name\": \"sep-oct-nov-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_5_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 30.27655536591546, \"C(hour_of_week)[1]\": 30.03114746932107, \"C(hour_of_week)[2]\": 30.070224938555917, \"C(hour_of_week)[3]\": 18.013508017689915, \"C(hour_of_week)[4]\": 29.719558659615082, \"C(hour_of_week)[5]\": 18.17224589346977, \"C(hour_of_week)[6]\": 18.2195703249951, \"C(hour_of_week)[7]\": 17.202488530767376, \"C(hour_of_week)[8]\": 16.5955873635177, \"C(hour_of_week)[9]\": 16.686275063106656, \"C(hour_of_week)[10]\": 17.249109355395756, \"C(hour_of_week)[11]\": 17.394977053076268, \"C(hour_of_week)[12]\": 17.286415968664176, \"C(hour_of_week)[13]\": 17.2266315061564, \"C(hour_of_week)[14]\": 16.999919673822223, \"C(hour_of_week)[15]\": 16.65317481812397, \"C(hour_of_week)[16]\": 29.588993122020852, \"C(hour_of_week)[17]\": 28.843400457823133, \"C(hour_of_week)[18]\": 27.880979065065908, \"C(hour_of_week)[19]\": 28.416283714819205, \"C(hour_of_week)[20]\": 28.58434451026054, \"C(hour_of_week)[21]\": 28.684047592985518, \"C(hour_of_week)[22]\": 28.644384242052233, \"C(hour_of_week)[23]\": 28.313349245476946, \"C(hour_of_week)[24]\": 28.435373783284202, \"C(hour_of_week)[25]\": 28.513939145893833, \"C(hour_of_week)[26]\": 28.81639852444017, \"C(hour_of_week)[27]\": 28.825740652131365, \"C(hour_of_week)[28]\": 29.02116571110834, \"C(hour_of_week)[29]\": 29.05734887425925, \"C(hour_of_week)[30]\": 29.33734373267347, \"C(hour_of_week)[31]\": 28.81328001893316, \"C(hour_of_week)[32]\": 28.67415645460644, \"C(hour_of_week)[33]\": 16.774359128174748, \"C(hour_of_week)[34]\": 17.217244164784464, \"C(hour_of_week)[35]\": 17.267745934847316, \"C(hour_of_week)[36]\": 17.369064205040253, \"C(hour_of_week)[37]\": 17.208596335319232, \"C(hour_of_week)[38]\": 16.914263639909066, \"C(hour_of_week)[39]\": 16.718677976730323, \"C(hour_of_week)[40]\": 16.56863984786057, \"C(hour_of_week)[41]\": 28.691015596396145, \"C(hour_of_week)[42]\": 27.6964157759401, \"C(hour_of_week)[43]\": 28.09177937460716, \"C(hour_of_week)[44]\": 28.23332435851736, \"C(hour_of_week)[45]\": 28.51037887444903, \"C(hour_of_week)[46]\": 28.192625141874014, \"C(hour_of_week)[47]\": 27.806730758975487, \"C(hour_of_week)[48]\": 27.76574453718197, \"C(hour_of_week)[49]\": 28.08822390459864, \"C(hour_of_week)[50]\": 28.248134392489053, \"C(hour_of_week)[51]\": 28.298167754357223, \"C(hour_of_week)[52]\": 28.333259929904287, \"C(hour_of_week)[53]\": 28.54490347183069, \"C(hour_of_week)[54]\": 28.68555420298292, \"C(hour_of_week)[55]\": 28.245642338027803, \"C(hour_of_week)[56]\": 28.256144861565385, \"C(hour_of_week)[57]\": 29.094972054297397, \"C(hour_of_week)[58]\": 29.681588187748133, \"C(hour_of_week)[59]\": 17.096686983232782, \"C(hour_of_week)[60]\": 17.20069174576422, \"C(hour_of_week)[61]\": 17.065520749278306, \"C(hour_of_week)[62]\": 16.91369536938222, \"C(hour_of_week)[63]\": 16.848944516674283, \"C(hour_of_week)[64]\": 29.228962844611072, \"C(hour_of_week)[65]\": 28.374592225348962, \"C(hour_of_week)[66]\": 27.1944216304326, \"C(hour_of_week)[67]\": 27.478666441618742, \"C(hour_of_week)[68]\": 27.870418637309083, \"C(hour_of_week)[69]\": 27.74366871749255, \"C(hour_of_week)[70]\": 27.755174188925196, \"C(hour_of_week)[71]\": 27.52426559548981, \"C(hour_of_week)[72]\": 27.657891570347285, \"C(hour_of_week)[73]\": 27.845126341180297, \"C(hour_of_week)[74]\": 28.180205086128993, \"C(hour_of_week)[75]\": 28.219615303617996, \"C(hour_of_week)[76]\": 28.413623419085557, \"C(hour_of_week)[77]\": 28.523721472638364, \"C(hour_of_week)[78]\": 28.563663066835815, \"C(hour_of_week)[79]\": 27.765365020032966, \"C(hour_of_week)[80]\": 28.349612438542366, \"C(hour_of_week)[81]\": 29.385053168189362, \"C(hour_of_week)[82]\": 29.961246184545562, \"C(hour_of_week)[83]\": 30.130836685899624, \"C(hour_of_week)[84]\": 17.62424539121027, \"C(hour_of_week)[85]\": 17.447889491576607, \"C(hour_of_week)[86]\": 17.32779975363923, \"C(hour_of_week)[87]\": 17.241891424389607, \"C(hour_of_week)[88]\": 29.61747224880832, \"C(hour_of_week)[89]\": 28.75788160267457, \"C(hour_of_week)[90]\": 28.168973818595447, \"C(hour_of_week)[91]\": 28.68634439289838, \"C(hour_of_week)[92]\": 28.739875510482506, \"C(hour_of_week)[93]\": 29.32012657867501, \"C(hour_of_week)[94]\": 28.98407446183369, \"C(hour_of_week)[95]\": 28.83332230742362, \"C(hour_of_week)[96]\": 28.80162125138472, \"C(hour_of_week)[97]\": 28.703192017233302, \"C(hour_of_week)[98]\": 29.05036481878761, \"C(hour_of_week)[99]\": 28.94193564775637, \"C(hour_of_week)[100]\": 29.061663551904847, \"C(hour_of_week)[101]\": 29.198365928242637, \"C(hour_of_week)[102]\": 29.475954756150553, \"C(hour_of_week)[103]\": 28.62435007820763, \"C(hour_of_week)[104]\": 28.856768028142188, \"C(hour_of_week)[105]\": 16.84114452231205, \"C(hour_of_week)[106]\": 29.652497929918685, \"C(hour_of_week)[107]\": 29.660743985225377, \"C(hour_of_week)[108]\": 17.015124309849213, \"C(hour_of_week)[109]\": 29.425955585154625, \"C(hour_of_week)[110]\": 29.2979274861928, \"C(hour_of_week)[111]\": 28.986901778435904, \"C(hour_of_week)[112]\": 28.69931897764228, \"C(hour_of_week)[113]\": 28.317752754659228, \"C(hour_of_week)[114]\": 28.578287651597606, \"C(hour_of_week)[115]\": 29.35308630597045, \"C(hour_of_week)[116]\": 29.278985541109158, \"C(hour_of_week)[117]\": 29.570633238241516, \"C(hour_of_week)[118]\": 29.378746486105914, \"C(hour_of_week)[119]\": 29.043916300555836, \"C(hour_of_week)[120]\": 29.089565072331908, \"C(hour_of_week)[121]\": 29.19842939621716, \"C(hour_of_week)[122]\": 29.370787671995405, \"C(hour_of_week)[123]\": 29.390076457206977, \"C(hour_of_week)[124]\": 29.409357344317634, \"C(hour_of_week)[125]\": 29.387390216798305, \"C(hour_of_week)[126]\": 29.78858391058893, \"C(hour_of_week)[127]\": 28.701909050802886, \"C(hour_of_week)[128]\": 27.848367581018696, \"C(hour_of_week)[129]\": 27.215599880456303, \"C(hour_of_week)[130]\": 26.869053563354772, \"C(hour_of_week)[131]\": 26.5816237139736, \"C(hour_of_week)[132]\": 27.336512199186238, \"C(hour_of_week)[133]\": 27.75639384005387, \"C(hour_of_week)[134]\": 27.611365361544706, \"C(hour_of_week)[135]\": 27.634559939237306, \"C(hour_of_week)[136]\": 27.623253145124878, \"C(hour_of_week)[137]\": 27.792912161025374, \"C(hour_of_week)[138]\": 28.279168128312477, \"C(hour_of_week)[139]\": 28.82991709294854, \"C(hour_of_week)[140]\": 28.780850180801107, \"C(hour_of_week)[141]\": 28.927924317111117, \"C(hour_of_week)[142]\": 28.891599078362983, \"C(hour_of_week)[143]\": 29.299036924743625, \"C(hour_of_week)[144]\": 29.173137679248235, \"C(hour_of_week)[145]\": 29.367837384619037, \"C(hour_of_week)[146]\": 29.46570455082897, \"C(hour_of_week)[147]\": 18.04904238713644, \"C(hour_of_week)[148]\": 29.55662046136834, \"C(hour_of_week)[149]\": 29.877935735565107, \"C(hour_of_week)[150]\": 18.488922116633127, \"C(hour_of_week)[151]\": 17.799438428178348, \"C(hour_of_week)[152]\": 29.035601680404454, \"C(hour_of_week)[153]\": 28.562183729581506, \"C(hour_of_week)[154]\": 28.175658966221523, \"C(hour_of_week)[155]\": 27.725853387860056, \"C(hour_of_week)[156]\": 27.713950130290225, \"C(hour_of_week)[157]\": 27.75660665907822, \"C(hour_of_week)[158]\": 27.876736367707498, \"C(hour_of_week)[159]\": 27.93842248465982, \"C(hour_of_week)[160]\": 28.295917857649506, \"C(hour_of_week)[161]\": 28.42222397873188, \"C(hour_of_week)[162]\": 28.657549741357236, \"C(hour_of_week)[163]\": 29.204111439074815, \"C(hour_of_week)[164]\": 29.399040338061916, \"C(hour_of_week)[165]\": 29.366573385300626, \"C(hour_of_week)[166]\": 29.837953804415797, \"C(hour_of_week)[167]\": 29.967505146319947, \"bin_0_occupied\": -0.2381895696019812, \"bin_1_occupied\": -0.23838868703383748, \"bin_2_occupied\": -0.10001059504365054, \"bin_3_occupied\": -0.014031048613680153, \"bin_4_occupied\": 0.24976180351099614, \"bin_5_occupied\": 0.15885793312588484, \"bin_0_unoccupied\": -0.5246241633383874, \"bin_1_unoccupied\": -0.4801754794862052, \"bin_2_unoccupied\": -0.24284495015951646, \"bin_3_unoccupied\": -0.08148064561400563, \"bin_4_unoccupied\": 0.13700209114734171, \"bin_5_unoccupied\": 0.13877956899084032}}, {\"segment_name\": \"oct-nov-dec-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 28.711758124766707, \"C(hour_of_week)[1]\": 28.543747182696446, \"C(hour_of_week)[2]\": 28.46600046780161, \"C(hour_of_week)[3]\": 28.011829409448303, \"C(hour_of_week)[4]\": 27.924853191459892, \"C(hour_of_week)[5]\": 28.147669206784137, \"C(hour_of_week)[6]\": 27.50619703368132, \"C(hour_of_week)[7]\": 21.97994191538849, \"C(hour_of_week)[8]\": 20.35600634017883, \"C(hour_of_week)[9]\": 19.539863947055753, \"C(hour_of_week)[10]\": 19.732536239702, \"C(hour_of_week)[11]\": 19.84605667720476, \"C(hour_of_week)[12]\": 19.868371614170368, \"C(hour_of_week)[13]\": 19.71864511933228, \"C(hour_of_week)[14]\": 19.582735391963908, \"C(hour_of_week)[15]\": 19.13061578991631, \"C(hour_of_week)[16]\": 19.66878653736818, \"C(hour_of_week)[17]\": 20.365461409173086, \"C(hour_of_week)[18]\": 21.83477701486796, \"C(hour_of_week)[19]\": 22.611114071815493, \"C(hour_of_week)[20]\": 22.984280520056362, \"C(hour_of_week)[21]\": 22.901056855225104, \"C(hour_of_week)[22]\": 23.335528561956405, \"C(hour_of_week)[23]\": 23.19910690719484, \"C(hour_of_week)[24]\": 23.50970523381244, \"C(hour_of_week)[25]\": 23.53627605313309, \"C(hour_of_week)[26]\": 23.691844336806557, \"C(hour_of_week)[27]\": 27.48942141866559, \"C(hour_of_week)[28]\": 23.94440679899692, \"C(hour_of_week)[29]\": 27.74531030712705, \"C(hour_of_week)[30]\": 27.48182521796086, \"C(hour_of_week)[31]\": 21.738270455562112, \"C(hour_of_week)[32]\": 20.215803818488066, \"C(hour_of_week)[33]\": 19.694389713135184, \"C(hour_of_week)[34]\": 19.81967047848799, \"C(hour_of_week)[35]\": 19.822284453914744, \"C(hour_of_week)[36]\": 19.776278027690644, \"C(hour_of_week)[37]\": 19.694825079056933, \"C(hour_of_week)[38]\": 19.408221732252468, \"C(hour_of_week)[39]\": 18.92051477083984, \"C(hour_of_week)[40]\": 19.517269179176107, \"C(hour_of_week)[41]\": 20.395990522386548, \"C(hour_of_week)[42]\": 21.784374240647264, \"C(hour_of_week)[43]\": 22.55727541887872, \"C(hour_of_week)[44]\": 22.873695445909725, \"C(hour_of_week)[45]\": 22.929428932684026, \"C(hour_of_week)[46]\": 23.199360272641226, \"C(hour_of_week)[47]\": 22.858799583581373, \"C(hour_of_week)[48]\": 26.60780687190149, \"C(hour_of_week)[49]\": 26.879366544137454, \"C(hour_of_week)[50]\": 27.05915720837217, \"C(hour_of_week)[51]\": 27.265655417419843, \"C(hour_of_week)[52]\": 27.3947153147074, \"C(hour_of_week)[53]\": 27.498859729420516, \"C(hour_of_week)[54]\": 27.073208347633283, \"C(hour_of_week)[55]\": 21.638403815991992, \"C(hour_of_week)[56]\": 20.163678253942116, \"C(hour_of_week)[57]\": 20.032556719062278, \"C(hour_of_week)[58]\": 19.809305084996367, \"C(hour_of_week)[59]\": 19.683714459680132, \"C(hour_of_week)[60]\": 19.67198156946672, \"C(hour_of_week)[61]\": 19.365226164464737, \"C(hour_of_week)[62]\": 19.04211196454203, \"C(hour_of_week)[63]\": 19.190344224819395, \"C(hour_of_week)[64]\": 19.7875150090469, \"C(hour_of_week)[65]\": 20.3950968508413, \"C(hour_of_week)[66]\": 21.853665459727868, \"C(hour_of_week)[67]\": 22.49243035452334, \"C(hour_of_week)[68]\": 22.828849503428135, \"C(hour_of_week)[69]\": 22.740067984439328, \"C(hour_of_week)[70]\": 23.27491452645309, \"C(hour_of_week)[71]\": 22.92127300189337, \"C(hour_of_week)[72]\": 22.90294205498723, \"C(hour_of_week)[73]\": 23.219510149328087, \"C(hour_of_week)[74]\": 23.401476519298214, \"C(hour_of_week)[75]\": 23.337547169743164, \"C(hour_of_week)[76]\": 27.070296174790624, \"C(hour_of_week)[77]\": 27.26201004625137, \"C(hour_of_week)[78]\": 27.105407618012393, \"C(hour_of_week)[79]\": 21.49169658942448, \"C(hour_of_week)[80]\": 20.688419374088703, \"C(hour_of_week)[81]\": 20.755034948243786, \"C(hour_of_week)[82]\": 20.60683923935888, \"C(hour_of_week)[83]\": 20.51724533070212, \"C(hour_of_week)[84]\": 20.515155042197303, \"C(hour_of_week)[85]\": 20.103637248009225, \"C(hour_of_week)[86]\": 19.973472789160326, \"C(hour_of_week)[87]\": 19.775099397662352, \"C(hour_of_week)[88]\": 20.311631994164735, \"C(hour_of_week)[89]\": 21.00014449782292, \"C(hour_of_week)[90]\": 22.84918281719662, \"C(hour_of_week)[91]\": 27.4229437027431, \"C(hour_of_week)[92]\": 27.479429255615763, \"C(hour_of_week)[93]\": 27.875679283651827, \"C(hour_of_week)[94]\": 27.838312417991638, \"C(hour_of_week)[95]\": 27.65335267988237, \"C(hour_of_week)[96]\": 27.62390677452466, \"C(hour_of_week)[97]\": 27.460122865434585, \"C(hour_of_week)[98]\": 27.81401610991429, \"C(hour_of_week)[99]\": 27.695077903272878, \"C(hour_of_week)[100]\": 27.697185757914042, \"C(hour_of_week)[101]\": 27.917711146550275, \"C(hour_of_week)[102]\": 27.532015564383965, \"C(hour_of_week)[103]\": 21.574590291881414, \"C(hour_of_week)[104]\": 20.43011925498432, \"C(hour_of_week)[105]\": 20.398545637271425, \"C(hour_of_week)[106]\": 20.396046454456954, \"C(hour_of_week)[107]\": 20.236635867001844, \"C(hour_of_week)[108]\": 20.060713325700846, \"C(hour_of_week)[109]\": 19.898317822136008, \"C(hour_of_week)[110]\": 19.68901192972416, \"C(hour_of_week)[111]\": 19.13389181520952, \"C(hour_of_week)[112]\": 19.649912689807095, \"C(hour_of_week)[113]\": 20.450742652160443, \"C(hour_of_week)[114]\": 22.482725042970507, \"C(hour_of_week)[115]\": 27.136968896690252, \"C(hour_of_week)[116]\": 27.224259631477786, \"C(hour_of_week)[117]\": 27.488775870197646, \"C(hour_of_week)[118]\": 27.447035417080137, \"C(hour_of_week)[119]\": 27.26581744699056, \"C(hour_of_week)[120]\": 27.55041238927323, \"C(hour_of_week)[121]\": 27.601136725954227, \"C(hour_of_week)[122]\": 27.770025625731176, \"C(hour_of_week)[123]\": 27.746262171024128, \"C(hour_of_week)[124]\": 27.775673980584884, \"C(hour_of_week)[125]\": 27.764390711034117, \"C(hour_of_week)[126]\": 28.36438574592641, \"C(hour_of_week)[127]\": 27.147849702518698, \"C(hour_of_week)[128]\": 22.505866212956757, \"C(hour_of_week)[129]\": 21.140897452220003, \"C(hour_of_week)[130]\": 20.212813663047548, \"C(hour_of_week)[131]\": 19.803255686941583, \"C(hour_of_week)[132]\": 19.70892147617806, \"C(hour_of_week)[133]\": 18.564094990065296, \"C(hour_of_week)[134]\": 18.37956481223923, \"C(hour_of_week)[135]\": 18.641870890631402, \"C(hour_of_week)[136]\": 19.345146958929508, \"C(hour_of_week)[137]\": 20.272256560409037, \"C(hour_of_week)[138]\": 22.167351126194585, \"C(hour_of_week)[139]\": 26.91977962567692, \"C(hour_of_week)[140]\": 23.28751993771277, \"C(hour_of_week)[141]\": 27.159861286410653, \"C(hour_of_week)[142]\": 23.77051199193381, \"C(hour_of_week)[143]\": 27.981430717282695, \"C(hour_of_week)[144]\": 27.736689076274537, \"C(hour_of_week)[145]\": 27.8156148967581, \"C(hour_of_week)[146]\": 27.853742724016445, \"C(hour_of_week)[147]\": 28.06406086704453, \"C(hour_of_week)[148]\": 28.117208972141047, \"C(hour_of_week)[149]\": 28.38021480503514, \"C(hour_of_week)[150]\": 28.414240855227042, \"C(hour_of_week)[151]\": 28.011683734458025, \"C(hour_of_week)[152]\": 27.02827364805027, \"C(hour_of_week)[153]\": 22.1131538707186, \"C(hour_of_week)[154]\": 21.122288039582962, \"C(hour_of_week)[155]\": 20.469431442899538, \"C(hour_of_week)[156]\": 20.056073818276772, \"C(hour_of_week)[157]\": 19.66258801493068, \"C(hour_of_week)[158]\": 19.685490561903904, \"C(hour_of_week)[159]\": 19.837166232963945, \"C(hour_of_week)[160]\": 21.21940315782187, \"C(hour_of_week)[161]\": 22.327445128865026, \"C(hour_of_week)[162]\": 22.894332562020114, \"C(hour_of_week)[163]\": 23.430844650897, \"C(hour_of_week)[164]\": 27.746343175858478, \"C(hour_of_week)[165]\": 27.941731739356324, \"C(hour_of_week)[166]\": 28.384048196985592, \"C(hour_of_week)[167]\": 28.564249380533447, \"bin_0_occupied\": -0.4054337293818377, \"bin_1_occupied\": -0.49667653919236643, \"bin_2_occupied\": -0.35049457752621377, \"bin_3_occupied\": -0.18622454544907033, \"bin_0_unoccupied\": -0.3138042860374506, \"bin_1_unoccupied\": -0.36298571227470716, \"bin_2_unoccupied\": -0.11349222561511674, \"bin_3_unoccupied\": -0.1742483194082457, \"bin_4_unoccupied\": 0.09967681123790502, \"bin_5_unoccupied\": 0.44943678761599093}}, {\"segment_name\": \"nov-dec-jan-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 31.98895156419638, \"C(hour_of_week)[1]\": 32.177718390904325, \"C(hour_of_week)[2]\": 32.13650550124076, \"C(hour_of_week)[3]\": 31.972267771709618, \"C(hour_of_week)[4]\": 31.87244908586085, \"C(hour_of_week)[5]\": 32.072619277460625, \"C(hour_of_week)[6]\": 31.13663143880229, \"C(hour_of_week)[7]\": 24.0173751665062, \"C(hour_of_week)[8]\": 21.906662080924583, \"C(hour_of_week)[9]\": 20.312815159181294, \"C(hour_of_week)[10]\": 20.197503732511386, \"C(hour_of_week)[11]\": 19.86108787779591, \"C(hour_of_week)[12]\": 19.691211986807268, \"C(hour_of_week)[13]\": 19.313576948745368, \"C(hour_of_week)[14]\": 19.415385265358886, \"C(hour_of_week)[15]\": 19.10945981062134, \"C(hour_of_week)[16]\": 19.879018358551324, \"C(hour_of_week)[17]\": 21.574035723326787, \"C(hour_of_week)[18]\": 29.67577783070654, \"C(hour_of_week)[19]\": 30.194334501643784, \"C(hour_of_week)[20]\": 30.691152391161104, \"C(hour_of_week)[21]\": 30.550405911005885, \"C(hour_of_week)[22]\": 30.55935385728337, \"C(hour_of_week)[23]\": 30.48088742255164, \"C(hour_of_week)[24]\": 30.946357164168752, \"C(hour_of_week)[25]\": 30.830990713030708, \"C(hour_of_week)[26]\": 30.809610265687194, \"C(hour_of_week)[27]\": 31.163595380035012, \"C(hour_of_week)[28]\": 31.186084518954342, \"C(hour_of_week)[29]\": 31.37117440755988, \"C(hour_of_week)[30]\": 30.75043606830743, \"C(hour_of_week)[31]\": 23.791485688430672, \"C(hour_of_week)[32]\": 21.51502245381142, \"C(hour_of_week)[33]\": 20.18767777946643, \"C(hour_of_week)[34]\": 19.993754743267548, \"C(hour_of_week)[35]\": 19.943524469151942, \"C(hour_of_week)[36]\": 19.767670332463076, \"C(hour_of_week)[37]\": 19.492542795312353, \"C(hour_of_week)[38]\": 19.222997795186174, \"C(hour_of_week)[39]\": 18.827245932600206, \"C(hour_of_week)[40]\": 19.77963941817016, \"C(hour_of_week)[41]\": 22.215420595175914, \"C(hour_of_week)[42]\": 25.33428561663138, \"C(hour_of_week)[43]\": 30.684160826684998, \"C(hour_of_week)[44]\": 31.521461786923695, \"C(hour_of_week)[45]\": 31.654396331038296, \"C(hour_of_week)[46]\": 31.955295151456657, \"C(hour_of_week)[47]\": 31.67434889930864, \"C(hour_of_week)[48]\": 31.429445532855063, \"C(hour_of_week)[49]\": 31.381890931592917, \"C(hour_of_week)[50]\": 31.582692074929547, \"C(hour_of_week)[51]\": 32.01983279064792, \"C(hour_of_week)[52]\": 31.9800347390195, \"C(hour_of_week)[53]\": 31.89823568553781, \"C(hour_of_week)[54]\": 31.3644344789662, \"C(hour_of_week)[55]\": 24.021529464439787, \"C(hour_of_week)[56]\": 22.192138717807254, \"C(hour_of_week)[57]\": 21.291898631928785, \"C(hour_of_week)[58]\": 20.60087262244678, \"C(hour_of_week)[59]\": 20.419218632522405, \"C(hour_of_week)[60]\": 20.157527139235384, \"C(hour_of_week)[61]\": 19.50034265183374, \"C(hour_of_week)[62]\": 19.31033546799707, \"C(hour_of_week)[63]\": 19.508913507524273, \"C(hour_of_week)[64]\": 21.05861051539218, \"C(hour_of_week)[65]\": 22.9647490695246, \"C(hour_of_week)[66]\": 31.316166247649573, \"C(hour_of_week)[67]\": 31.812781326074116, \"C(hour_of_week)[68]\": 31.987593219613018, \"C(hour_of_week)[69]\": 31.79329340105898, \"C(hour_of_week)[70]\": 32.53687874757611, \"C(hour_of_week)[71]\": 32.11744926642573, \"C(hour_of_week)[72]\": 31.62735126696792, \"C(hour_of_week)[73]\": 31.801792944564497, \"C(hour_of_week)[74]\": 31.61809007959986, \"C(hour_of_week)[75]\": 31.5776290767317, \"C(hour_of_week)[76]\": 31.449158818117574, \"C(hour_of_week)[77]\": 31.6499965864865, \"C(hour_of_week)[78]\": 31.404268664799837, \"C(hour_of_week)[79]\": 23.8826997466251, \"C(hour_of_week)[80]\": 22.20025175292171, \"C(hour_of_week)[81]\": 21.529494913805937, \"C(hour_of_week)[82]\": 20.920631415677228, \"C(hour_of_week)[83]\": 20.86370404763257, \"C(hour_of_week)[84]\": 20.858969510416287, \"C(hour_of_week)[85]\": 20.215418652600107, \"C(hour_of_week)[86]\": 19.895104029458835, \"C(hour_of_week)[87]\": 19.367329386574454, \"C(hour_of_week)[88]\": 20.479148131878897, \"C(hour_of_week)[89]\": 21.998138977954515, \"C(hour_of_week)[90]\": 30.35598094555071, \"C(hour_of_week)[91]\": 30.88531024059744, \"C(hour_of_week)[92]\": 31.120715223258415, \"C(hour_of_week)[93]\": 31.320675963350794, \"C(hour_of_week)[94]\": 31.44124326371025, \"C(hour_of_week)[95]\": 31.002164316261904, \"C(hour_of_week)[96]\": 31.159272121237514, \"C(hour_of_week)[97]\": 31.376372872870935, \"C(hour_of_week)[98]\": 31.733431732116035, \"C(hour_of_week)[99]\": 31.738754555126842, \"C(hour_of_week)[100]\": 31.971499977998924, \"C(hour_of_week)[101]\": 32.35598835623367, \"C(hour_of_week)[102]\": 31.53264445230679, \"C(hour_of_week)[103]\": 23.923007451110177, \"C(hour_of_week)[104]\": 22.018717272660435, \"C(hour_of_week)[105]\": 21.53234906007758, \"C(hour_of_week)[106]\": 21.399276212032262, \"C(hour_of_week)[107]\": 21.359956111472723, \"C(hour_of_week)[108]\": 20.98771271292019, \"C(hour_of_week)[109]\": 20.733733104920457, \"C(hour_of_week)[110]\": 20.393872098375663, \"C(hour_of_week)[111]\": 19.68460540453138, \"C(hour_of_week)[112]\": 20.872074486016746, \"C(hour_of_week)[113]\": 22.55507802970553, \"C(hour_of_week)[114]\": 30.47942507432029, \"C(hour_of_week)[115]\": 31.176134560328833, \"C(hour_of_week)[116]\": 31.56560749989059, \"C(hour_of_week)[117]\": 31.216751101795136, \"C(hour_of_week)[118]\": 31.369058223861504, \"C(hour_of_week)[119]\": 31.381759687116293, \"C(hour_of_week)[120]\": 31.605880204931914, \"C(hour_of_week)[121]\": 31.63841885317086, \"C(hour_of_week)[122]\": 31.848061307062533, \"C(hour_of_week)[123]\": 31.75055262295338, \"C(hour_of_week)[124]\": 31.68288193895297, \"C(hour_of_week)[125]\": 31.401826594676454, \"C(hour_of_week)[126]\": 31.980824951175777, \"C(hour_of_week)[127]\": 31.065046627173786, \"C(hour_of_week)[128]\": 30.18669344712927, \"C(hour_of_week)[129]\": 23.30491185109104, \"C(hour_of_week)[130]\": 21.982363096925276, \"C(hour_of_week)[131]\": 21.377087255865863, \"C(hour_of_week)[132]\": 21.0937158575841, \"C(hour_of_week)[133]\": 18.927393416227037, \"C(hour_of_week)[134]\": 18.45406871126409, \"C(hour_of_week)[135]\": 18.905333610864087, \"C(hour_of_week)[136]\": 19.965486757483877, \"C(hour_of_week)[137]\": 21.53048885355686, \"C(hour_of_week)[138]\": 24.033842495318847, \"C(hour_of_week)[139]\": 25.325060291572, \"C(hour_of_week)[140]\": 30.580505579163212, \"C(hour_of_week)[141]\": 30.628318948201525, \"C(hour_of_week)[142]\": 30.91143573147886, \"C(hour_of_week)[143]\": 31.630568248300623, \"C(hour_of_week)[144]\": 31.310654966802474, \"C(hour_of_week)[145]\": 31.197222682556298, \"C(hour_of_week)[146]\": 31.323665824442518, \"C(hour_of_week)[147]\": 31.444790781459737, \"C(hour_of_week)[148]\": 31.66163487642701, \"C(hour_of_week)[149]\": 32.04025959822834, \"C(hour_of_week)[150]\": 31.835273298465854, \"C(hour_of_week)[151]\": 31.723241959450164, \"C(hour_of_week)[152]\": 30.695900219021834, \"C(hour_of_week)[153]\": 23.707660768793176, \"C(hour_of_week)[154]\": 22.37528352061392, \"C(hour_of_week)[155]\": 21.661029652499398, \"C(hour_of_week)[156]\": 21.111808171535273, \"C(hour_of_week)[157]\": 20.414434336519477, \"C(hour_of_week)[158]\": 20.51293423997956, \"C(hour_of_week)[159]\": 20.968217984122212, \"C(hour_of_week)[160]\": 23.155601971005225, \"C(hour_of_week)[161]\": 30.271929584170444, \"C(hour_of_week)[162]\": 30.72315239117088, \"C(hour_of_week)[163]\": 31.033755412575744, \"C(hour_of_week)[164]\": 31.536330175519105, \"C(hour_of_week)[165]\": 31.867192158805633, \"C(hour_of_week)[166]\": 31.94706953117463, \"C(hour_of_week)[167]\": 32.10651980575815, \"bin_0_occupied\": -0.5377872114998198, \"bin_1_occupied\": -0.35838551642613065, \"bin_2_occupied\": -0.09763135622759472, \"bin_0_unoccupied\": -0.35895440770368875, \"bin_1_unoccupied\": -0.2545363019920993, \"bin_2_unoccupied\": -0.041317096752884204}}], \"model_lookup\": {\"jan\": {\"segment_name\": \"dec-jan-feb-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 32.15899363448022, \"C(hour_of_week)[1]\": 32.58562881548998, \"C(hour_of_week)[2]\": 32.61812220709161, \"C(hour_of_week)[3]\": 32.65520698632631, \"C(hour_of_week)[4]\": 32.61570655800883, \"C(hour_of_week)[5]\": 32.73311651749455, \"C(hour_of_week)[6]\": 31.856477499270156, \"C(hour_of_week)[7]\": 24.907281832799118, \"C(hour_of_week)[8]\": 22.790287600128057, \"C(hour_of_week)[9]\": 21.11727954258783, \"C(hour_of_week)[10]\": 20.761681291728614, \"C(hour_of_week)[11]\": 20.05395739534302, \"C(hour_of_week)[12]\": 19.852304627158343, \"C(hour_of_week)[13]\": 19.178851313753633, \"C(hour_of_week)[14]\": 19.35026829680495, \"C(hour_of_week)[15]\": 19.332117497691748, \"C(hour_of_week)[16]\": 19.89278593138144, \"C(hour_of_week)[17]\": 21.962458863084876, \"C(hour_of_week)[18]\": 25.94023979782371, \"C(hour_of_week)[19]\": 26.43810836126226, \"C(hour_of_week)[20]\": 31.06093758548402, \"C(hour_of_week)[21]\": 31.214327461098584, \"C(hour_of_week)[22]\": 30.8490200146618, \"C(hour_of_week)[23]\": 30.9096072954312, \"C(hour_of_week)[24]\": 31.29415695921615, \"C(hour_of_week)[25]\": 31.241052261465327, \"C(hour_of_week)[26]\": 31.208300117846093, \"C(hour_of_week)[27]\": 31.352104548740712, \"C(hour_of_week)[28]\": 31.60977188989677, \"C(hour_of_week)[29]\": 31.94923976499217, \"C(hour_of_week)[30]\": 30.919028611260423, \"C(hour_of_week)[31]\": 24.53872328208661, \"C(hour_of_week)[32]\": 22.06403295329572, \"C(hour_of_week)[33]\": 20.54919389704653, \"C(hour_of_week)[34]\": 20.359300048719383, \"C(hour_of_week)[35]\": 20.37131269400872, \"C(hour_of_week)[36]\": 20.16300689354454, \"C(hour_of_week)[37]\": 19.60806710042366, \"C(hour_of_week)[38]\": 19.487260513974743, \"C(hour_of_week)[39]\": 19.332725575623478, \"C(hour_of_week)[40]\": 19.898575384762832, \"C(hour_of_week)[41]\": 22.86172819727339, \"C(hour_of_week)[42]\": 26.54196868161532, \"C(hour_of_week)[43]\": 27.039270645311074, \"C(hour_of_week)[44]\": 32.54572993963664, \"C(hour_of_week)[45]\": 32.806307618295, \"C(hour_of_week)[46]\": 32.73563536913076, \"C(hour_of_week)[47]\": 32.640324594388055, \"C(hour_of_week)[48]\": 32.175234270947, \"C(hour_of_week)[49]\": 32.360014648369194, \"C(hour_of_week)[50]\": 32.415444577446074, \"C(hour_of_week)[51]\": 32.89551645807635, \"C(hour_of_week)[52]\": 32.58032978966754, \"C(hour_of_week)[53]\": 32.63647995817117, \"C(hour_of_week)[54]\": 31.940512378808833, \"C(hour_of_week)[55]\": 24.782044601435196, \"C(hour_of_week)[56]\": 22.979195575028317, \"C(hour_of_week)[57]\": 21.808512840446696, \"C(hour_of_week)[58]\": 21.04900057868996, \"C(hour_of_week)[59]\": 20.900855178060457, \"C(hour_of_week)[60]\": 20.624821206342776, \"C(hour_of_week)[61]\": 19.702274239592672, \"C(hour_of_week)[62]\": 19.62181267265293, \"C(hour_of_week)[63]\": 19.634969997947852, \"C(hour_of_week)[64]\": 20.74016163180321, \"C(hour_of_week)[65]\": 23.28685781544317, \"C(hour_of_week)[66]\": 31.786305059379874, \"C(hour_of_week)[67]\": 31.811282119855832, \"C(hour_of_week)[68]\": 32.09839413276148, \"C(hour_of_week)[69]\": 32.04342560921696, \"C(hour_of_week)[70]\": 32.43327898572922, \"C(hour_of_week)[71]\": 32.27941765600451, \"C(hour_of_week)[72]\": 32.58562063475673, \"C(hour_of_week)[73]\": 32.47219241516897, \"C(hour_of_week)[74]\": 32.17647165457752, \"C(hour_of_week)[75]\": 32.13154760722789, \"C(hour_of_week)[76]\": 32.20425127589996, \"C(hour_of_week)[77]\": 32.361440932455, \"C(hour_of_week)[78]\": 31.830691038280342, \"C(hour_of_week)[79]\": 24.80936239041419, \"C(hour_of_week)[80]\": 22.84051696243142, \"C(hour_of_week)[81]\": 21.21123304872646, \"C(hour_of_week)[82]\": 20.520799974278827, \"C(hour_of_week)[83]\": 20.801360213686554, \"C(hour_of_week)[84]\": 20.70103928391113, \"C(hour_of_week)[85]\": 19.752886769960526, \"C(hour_of_week)[86]\": 19.46488853801918, \"C(hour_of_week)[87]\": 18.955052996483698, \"C(hour_of_week)[88]\": 19.931447105228234, \"C(hour_of_week)[89]\": 21.898293879930506, \"C(hour_of_week)[90]\": 30.015772988415666, \"C(hour_of_week)[91]\": 30.555296969210538, \"C(hour_of_week)[92]\": 30.989890391672127, \"C(hour_of_week)[93]\": 31.374047466073655, \"C(hour_of_week)[94]\": 31.343083725389352, \"C(hour_of_week)[95]\": 30.973472199271495, \"C(hour_of_week)[96]\": 31.306742686599623, \"C(hour_of_week)[97]\": 31.91940897071943, \"C(hour_of_week)[98]\": 32.28268156828579, \"C(hour_of_week)[99]\": 32.41773444149017, \"C(hour_of_week)[100]\": 32.79547133298564, \"C(hour_of_week)[101]\": 33.07163360646684, \"C(hour_of_week)[102]\": 32.01290575736232, \"C(hour_of_week)[103]\": 25.288978132064972, \"C(hour_of_week)[104]\": 23.31227014969002, \"C(hour_of_week)[105]\": 22.401526121695, \"C(hour_of_week)[106]\": 21.89728523568504, \"C(hour_of_week)[107]\": 21.919917341963558, \"C(hour_of_week)[108]\": 21.543081382150692, \"C(hour_of_week)[109]\": 20.902587811403922, \"C(hour_of_week)[110]\": 20.380651013725362, \"C(hour_of_week)[111]\": 20.05736410095646, \"C(hour_of_week)[112]\": 21.09189436034263, \"C(hour_of_week)[113]\": 23.201646471034206, \"C(hour_of_week)[114]\": 31.177628188806736, \"C(hour_of_week)[115]\": 31.811928406068034, \"C(hour_of_week)[116]\": 32.27843354784621, \"C(hour_of_week)[117]\": 31.889051334378788, \"C(hour_of_week)[118]\": 31.96378696780874, \"C(hour_of_week)[119]\": 31.972797111289967, \"C(hour_of_week)[120]\": 32.19533126826024, \"C(hour_of_week)[121]\": 32.0203082485063, \"C(hour_of_week)[122]\": 32.250106464808944, \"C(hour_of_week)[123]\": 32.13989484562883, \"C(hour_of_week)[124]\": 32.050235146574835, \"C(hour_of_week)[125]\": 31.91547119227756, \"C(hour_of_week)[126]\": 32.47376160291762, \"C(hour_of_week)[127]\": 31.486635858680938, \"C(hour_of_week)[128]\": 30.371496890619145, \"C(hour_of_week)[129]\": 24.542281415297484, \"C(hour_of_week)[130]\": 23.080358310115333, \"C(hour_of_week)[131]\": 22.12671249926624, \"C(hour_of_week)[132]\": 21.80261264168631, \"C(hour_of_week)[133]\": 19.124890248441016, \"C(hour_of_week)[134]\": 18.598897570415986, \"C(hour_of_week)[135]\": 18.980580306522366, \"C(hour_of_week)[136]\": 19.711871828479243, \"C(hour_of_week)[137]\": 21.54837878115354, \"C(hour_of_week)[138]\": 24.792448566548646, \"C(hour_of_week)[139]\": 30.411171823863764, \"C(hour_of_week)[140]\": 31.045448566610016, \"C(hour_of_week)[141]\": 30.926724616991372, \"C(hour_of_week)[142]\": 31.20207028937535, \"C(hour_of_week)[143]\": 31.67751224549073, \"C(hour_of_week)[144]\": 31.523676458295895, \"C(hour_of_week)[145]\": 31.630152913436977, \"C(hour_of_week)[146]\": 31.90576798845558, \"C(hour_of_week)[147]\": 31.954484843221444, \"C(hour_of_week)[148]\": 32.22954082342334, \"C(hour_of_week)[149]\": 32.76433080154116, \"C(hour_of_week)[150]\": 32.29874264361644, \"C(hour_of_week)[151]\": 32.31885870545857, \"C(hour_of_week)[152]\": 27.05358027346026, \"C(hour_of_week)[153]\": 24.92577860890868, \"C(hour_of_week)[154]\": 23.645576556682865, \"C(hour_of_week)[155]\": 23.009598389626998, \"C(hour_of_week)[156]\": 22.68232694640618, \"C(hour_of_week)[157]\": 22.071141798131883, \"C(hour_of_week)[158]\": 22.159714687068632, \"C(hour_of_week)[159]\": 22.62162735928998, \"C(hour_of_week)[160]\": 24.406460590973563, \"C(hour_of_week)[161]\": 26.316733814852977, \"C(hour_of_week)[162]\": 31.763118194197624, \"C(hour_of_week)[163]\": 31.853519702329205, \"C(hour_of_week)[164]\": 32.37267868638918, \"C(hour_of_week)[165]\": 32.56731989949366, \"C(hour_of_week)[166]\": 32.46366099353558, \"C(hour_of_week)[167]\": 32.43130473132385, \"bin_0_occupied\": -0.5397879922762127, \"bin_1_occupied\": -0.34397717717226745, \"bin_2_occupied\": -0.14714690337327535, \"bin_0_unoccupied\": -0.3728458101434492, \"bin_1_unoccupied\": -0.2544749189147975, \"bin_2_unoccupied\": 0.0067672015189005775, \"bin_3_unoccupied\": -0.09190119178221492}}, \"feb\": {\"segment_name\": \"jan-feb-mar-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 31.69619495313116, \"C(hour_of_week)[1]\": 31.883173988305636, \"C(hour_of_week)[2]\": 31.99141404639163, \"C(hour_of_week)[3]\": 32.09074739677064, \"C(hour_of_week)[4]\": 32.23600462548064, \"C(hour_of_week)[5]\": 32.26958780022159, \"C(hour_of_week)[6]\": 31.607163998402076, \"C(hour_of_week)[7]\": 25.224210851491552, \"C(hour_of_week)[8]\": 23.200868092940475, \"C(hour_of_week)[9]\": 21.756239708839956, \"C(hour_of_week)[10]\": 21.45053533307132, \"C(hour_of_week)[11]\": 20.974927752771226, \"C(hour_of_week)[12]\": 20.726945207164654, \"C(hour_of_week)[13]\": 19.956064449212096, \"C(hour_of_week)[14]\": 20.004053519252523, \"C(hour_of_week)[15]\": 20.039374644764944, \"C(hour_of_week)[16]\": 20.256460456181017, \"C(hour_of_week)[17]\": 21.926348984992135, \"C(hour_of_week)[18]\": 25.784389808153406, \"C(hour_of_week)[19]\": 29.844102116791973, \"C(hour_of_week)[20]\": 31.05749203415121, \"C(hour_of_week)[21]\": 31.43759830970828, \"C(hour_of_week)[22]\": 30.994696096981446, \"C(hour_of_week)[23]\": 31.148536871586423, \"C(hour_of_week)[24]\": 31.445733791551778, \"C(hour_of_week)[25]\": 31.656250562688555, \"C(hour_of_week)[26]\": 31.45673484407656, \"C(hour_of_week)[27]\": 31.058893471617367, \"C(hour_of_week)[28]\": 31.62665283708522, \"C(hour_of_week)[29]\": 32.055577625013214, \"C(hour_of_week)[30]\": 31.071109066481203, \"C(hour_of_week)[31]\": 24.923003177418057, \"C(hour_of_week)[32]\": 22.8536054467803, \"C(hour_of_week)[33]\": 21.565074912090754, \"C(hour_of_week)[34]\": 21.421183327212113, \"C(hour_of_week)[35]\": 21.286096590096317, \"C(hour_of_week)[36]\": 20.959338027433493, \"C(hour_of_week)[37]\": 20.358976799048513, \"C(hour_of_week)[38]\": 20.392517036677425, \"C(hour_of_week)[39]\": 20.22605769454511, \"C(hour_of_week)[40]\": 20.20912765270942, \"C(hour_of_week)[41]\": 22.545761199548352, \"C(hour_of_week)[42]\": 29.805129485793895, \"C(hour_of_week)[43]\": 30.397627823860567, \"C(hour_of_week)[44]\": 31.646015985238012, \"C(hour_of_week)[45]\": 31.76166077797324, \"C(hour_of_week)[46]\": 31.412894996543244, \"C(hour_of_week)[47]\": 31.747111834666857, \"C(hour_of_week)[48]\": 31.497813724873673, \"C(hour_of_week)[49]\": 31.971052517868568, \"C(hour_of_week)[50]\": 31.796321327068743, \"C(hour_of_week)[51]\": 32.31670125990251, \"C(hour_of_week)[52]\": 32.016875417911976, \"C(hour_of_week)[53]\": 32.26366972625077, \"C(hour_of_week)[54]\": 31.21080511896095, \"C(hour_of_week)[55]\": 25.02366667038599, \"C(hour_of_week)[56]\": 23.194660829499313, \"C(hour_of_week)[57]\": 21.724244935041582, \"C(hour_of_week)[58]\": 21.338611215217874, \"C(hour_of_week)[59]\": 21.196324884446863, \"C(hour_of_week)[60]\": 21.014778751151216, \"C(hour_of_week)[61]\": 20.239522385552984, \"C(hour_of_week)[62]\": 20.121881243936457, \"C(hour_of_week)[63]\": 19.816600446880404, \"C(hour_of_week)[64]\": 19.968190293020605, \"C(hour_of_week)[65]\": 21.89468657983282, \"C(hour_of_week)[66]\": 25.58955554272984, \"C(hour_of_week)[67]\": 26.275135234761965, \"C(hour_of_week)[68]\": 30.516714895153605, \"C(hour_of_week)[69]\": 30.68421335931127, \"C(hour_of_week)[70]\": 31.12011660306499, \"C(hour_of_week)[71]\": 31.056096798756233, \"C(hour_of_week)[72]\": 31.692996319309923, \"C(hour_of_week)[73]\": 31.644433823156373, \"C(hour_of_week)[74]\": 31.76206205137058, \"C(hour_of_week)[75]\": 31.597534349965404, \"C(hour_of_week)[76]\": 31.795408618870823, \"C(hour_of_week)[77]\": 32.003085745028, \"C(hour_of_week)[78]\": 31.41905159462924, \"C(hour_of_week)[79]\": 25.19410149086313, \"C(hour_of_week)[80]\": 23.280632189041466, \"C(hour_of_week)[81]\": 21.236633397225464, \"C(hour_of_week)[82]\": 20.80546521259806, \"C(hour_of_week)[83]\": 20.97298122325837, \"C(hour_of_week)[84]\": 20.693438105825326, \"C(hour_of_week)[85]\": 19.79730731816196, \"C(hour_of_week)[86]\": 19.64639621138425, \"C(hour_of_week)[87]\": 19.454786997318166, \"C(hour_of_week)[88]\": 19.984503658417623, \"C(hour_of_week)[89]\": 21.60309625790427, \"C(hour_of_week)[90]\": 25.250521323557383, \"C(hour_of_week)[91]\": 26.285118160160078, \"C(hour_of_week)[92]\": 27.009533579232546, \"C(hour_of_week)[93]\": 30.680938246152486, \"C(hour_of_week)[94]\": 30.408523133580893, \"C(hour_of_week)[95]\": 30.40163570350816, \"C(hour_of_week)[96]\": 30.702092253129234, \"C(hour_of_week)[97]\": 31.289194097096217, \"C(hour_of_week)[98]\": 31.675475965146518, \"C(hour_of_week)[99]\": 31.9019823441271, \"C(hour_of_week)[100]\": 32.194093951688025, \"C(hour_of_week)[101]\": 32.41128619621574, \"C(hour_of_week)[102]\": 31.403093694041896, \"C(hour_of_week)[103]\": 25.396670362001547, \"C(hour_of_week)[104]\": 23.574908903592085, \"C(hour_of_week)[105]\": 22.347264550064004, \"C(hour_of_week)[106]\": 21.783729974786027, \"C(hour_of_week)[107]\": 21.715308036471715, \"C(hour_of_week)[108]\": 21.371582721441662, \"C(hour_of_week)[109]\": 20.57920317267788, \"C(hour_of_week)[110]\": 20.162147035309946, \"C(hour_of_week)[111]\": 20.17328294898878, \"C(hour_of_week)[112]\": 20.267415557908443, \"C(hour_of_week)[113]\": 21.588898958557735, \"C(hour_of_week)[114]\": 25.68719547714796, \"C(hour_of_week)[115]\": 26.88784809866192, \"C(hour_of_week)[116]\": 30.384521093224844, \"C(hour_of_week)[117]\": 30.713591998043963, \"C(hour_of_week)[118]\": 30.70428179311557, \"C(hour_of_week)[119]\": 27.90456419487642, \"C(hour_of_week)[120]\": 31.342643904310705, \"C(hour_of_week)[121]\": 31.396692134779563, \"C(hour_of_week)[122]\": 31.720058588928353, \"C(hour_of_week)[123]\": 31.83784054000408, \"C(hour_of_week)[124]\": 31.969932838053104, \"C(hour_of_week)[125]\": 32.33787214691998, \"C(hour_of_week)[126]\": 32.87128911229978, \"C(hour_of_week)[127]\": 31.75711099419046, \"C(hour_of_week)[128]\": 27.304643595077042, \"C(hour_of_week)[129]\": 25.913003298340104, \"C(hour_of_week)[130]\": 24.7318936523136, \"C(hour_of_week)[131]\": 23.56012282272306, \"C(hour_of_week)[132]\": 23.027322175986097, \"C(hour_of_week)[133]\": 20.599415452585323, \"C(hour_of_week)[134]\": 20.140523234050118, \"C(hour_of_week)[135]\": 20.378163840883204, \"C(hour_of_week)[136]\": 20.612977365130988, \"C(hour_of_week)[137]\": 22.0241785149091, \"C(hour_of_week)[138]\": 25.15122864759263, \"C(hour_of_week)[139]\": 26.901887332692464, \"C(hour_of_week)[140]\": 31.08559919721048, \"C(hour_of_week)[141]\": 31.46762358423757, \"C(hour_of_week)[142]\": 31.578309248819625, \"C(hour_of_week)[143]\": 31.80370806875947, \"C(hour_of_week)[144]\": 31.871049252849307, \"C(hour_of_week)[145]\": 31.8902285482373, \"C(hour_of_week)[146]\": 32.377531002109585, \"C(hour_of_week)[147]\": 32.23360856527041, \"C(hour_of_week)[148]\": 32.39370227874754, \"C(hour_of_week)[149]\": 32.76910914776338, \"C(hour_of_week)[150]\": 32.406658984411145, \"C(hour_of_week)[151]\": 32.021657985064834, \"C(hour_of_week)[152]\": 27.302047873495646, \"C(hour_of_week)[153]\": 25.25129490626848, \"C(hour_of_week)[154]\": 23.927027824336275, \"C(hour_of_week)[155]\": 23.11330868447653, \"C(hour_of_week)[156]\": 23.092483732841984, \"C(hour_of_week)[157]\": 22.670839177305698, \"C(hour_of_week)[158]\": 22.34396350990764, \"C(hour_of_week)[159]\": 22.46316163196926, \"C(hour_of_week)[160]\": 23.24015708557393, \"C(hour_of_week)[161]\": 25.183133758437204, \"C(hour_of_week)[162]\": 29.852980325507737, \"C(hour_of_week)[163]\": 30.502918781669155, \"C(hour_of_week)[164]\": 31.014737451484837, \"C(hour_of_week)[165]\": 31.213454859392417, \"C(hour_of_week)[166]\": 31.406168995022067, \"C(hour_of_week)[167]\": 31.36338670892857, \"bin_0_occupied\": -0.527614366229834, \"bin_1_occupied\": -0.3671416187265185, \"bin_2_occupied\": -0.17678692012547065, \"bin_0_unoccupied\": -0.41436649971578454, \"bin_1_unoccupied\": -0.21027104555092474, \"bin_2_unoccupied\": -0.030931109147582472, \"bin_3_unoccupied\": 0.035954795964798086}}, \"mar\": {\"segment_name\": \"feb-mar-apr-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 29.606509743611365, \"C(hour_of_week)[1]\": 29.570577507170487, \"C(hour_of_week)[2]\": 29.894551074352773, \"C(hour_of_week)[3]\": 30.201982912437487, \"C(hour_of_week)[4]\": 30.458006803474902, \"C(hour_of_week)[5]\": 30.575538190141774, \"C(hour_of_week)[6]\": 29.891861064545132, \"C(hour_of_week)[7]\": 25.422202404675495, \"C(hour_of_week)[8]\": 23.681571762480907, \"C(hour_of_week)[9]\": 22.709993126856723, \"C(hour_of_week)[10]\": 22.813744673000112, \"C(hour_of_week)[11]\": 22.67661849098906, \"C(hour_of_week)[12]\": 22.243257315276175, \"C(hour_of_week)[13]\": 21.825394871803365, \"C(hour_of_week)[14]\": 21.76419112178408, \"C(hour_of_week)[15]\": 21.457409200447408, \"C(hour_of_week)[16]\": 21.25247198499195, \"C(hour_of_week)[17]\": 21.994671452675277, \"C(hour_of_week)[18]\": 24.385311979686048, \"C(hour_of_week)[19]\": 26.036060784572296, \"C(hour_of_week)[20]\": 29.98411167398938, \"C(hour_of_week)[21]\": 30.27470604919401, \"C(hour_of_week)[22]\": 30.12798018150804, \"C(hour_of_week)[23]\": 30.390619112519033, \"C(hour_of_week)[24]\": 30.567741367664933, \"C(hour_of_week)[25]\": 30.90686651216744, \"C(hour_of_week)[26]\": 30.759065158081107, \"C(hour_of_week)[27]\": 30.36244589356716, \"C(hour_of_week)[28]\": 30.81727375751529, \"C(hour_of_week)[29]\": 31.138325987190207, \"C(hour_of_week)[30]\": 30.46410231328193, \"C(hour_of_week)[31]\": 28.055484606844658, \"C(hour_of_week)[32]\": 23.895616718262502, \"C(hour_of_week)[33]\": 23.001850398079505, \"C(hour_of_week)[34]\": 22.96403235230715, \"C(hour_of_week)[35]\": 22.65598914478821, \"C(hour_of_week)[36]\": 22.313599898520728, \"C(hour_of_week)[37]\": 22.044729415280795, \"C(hour_of_week)[38]\": 22.16014095970541, \"C(hour_of_week)[39]\": 21.88045469380224, \"C(hour_of_week)[40]\": 21.606597790019624, \"C(hour_of_week)[41]\": 22.450565018541212, \"C(hour_of_week)[42]\": 24.65425520679909, \"C(hour_of_week)[43]\": 28.447471516088296, \"C(hour_of_week)[44]\": 29.433285504445916, \"C(hour_of_week)[45]\": 29.50420050378277, \"C(hour_of_week)[46]\": 29.438769586535024, \"C(hour_of_week)[47]\": 29.976369142033022, \"C(hour_of_week)[48]\": 30.09679618962316, \"C(hour_of_week)[49]\": 30.35913852561247, \"C(hour_of_week)[50]\": 30.38680007049313, \"C(hour_of_week)[51]\": 30.9213831969174, \"C(hour_of_week)[52]\": 30.98224325864481, \"C(hour_of_week)[53]\": 31.140436285525446, \"C(hour_of_week)[54]\": 30.1846466459844, \"C(hour_of_week)[55]\": 25.41643473636842, \"C(hour_of_week)[56]\": 23.77245108344277, \"C(hour_of_week)[57]\": 22.215789258220827, \"C(hour_of_week)[58]\": 22.373457987146136, \"C(hour_of_week)[59]\": 22.235275040920378, \"C(hour_of_week)[60]\": 22.057567555427134, \"C(hour_of_week)[61]\": 21.696944881008182, \"C(hour_of_week)[62]\": 21.38798420900042, \"C(hour_of_week)[63]\": 20.733100996044413, \"C(hour_of_week)[64]\": 20.347537247278407, \"C(hour_of_week)[65]\": 20.754600526000136, \"C(hour_of_week)[66]\": 22.85023465882854, \"C(hour_of_week)[67]\": 24.859119357188366, \"C(hour_of_week)[68]\": 28.031947550457993, \"C(hour_of_week)[69]\": 28.192189338992257, \"C(hour_of_week)[70]\": 28.87329726573205, \"C(hour_of_week)[71]\": 28.814073214555155, \"C(hour_of_week)[72]\": 29.272795891520655, \"C(hour_of_week)[73]\": 29.463350313690682, \"C(hour_of_week)[74]\": 29.818250997342826, \"C(hour_of_week)[75]\": 29.700215186437074, \"C(hour_of_week)[76]\": 29.82893239247302, \"C(hour_of_week)[77]\": 30.1884935938724, \"C(hour_of_week)[78]\": 29.746660687045047, \"C(hour_of_week)[79]\": 25.119547274272676, \"C(hour_of_week)[80]\": 23.20917927338823, \"C(hour_of_week)[81]\": 22.07010185351045, \"C(hour_of_week)[82]\": 22.163712233924254, \"C(hour_of_week)[83]\": 22.020099848496077, \"C(hour_of_week)[84]\": 21.686955369179568, \"C(hour_of_week)[85]\": 21.358676823474806, \"C(hour_of_week)[86]\": 21.270366535753688, \"C(hour_of_week)[87]\": 20.907198842526405, \"C(hour_of_week)[88]\": 20.79434626401162, \"C(hour_of_week)[89]\": 21.017544223045704, \"C(hour_of_week)[90]\": 22.715054789525517, \"C(hour_of_week)[91]\": 24.4523716369436, \"C(hour_of_week)[92]\": 25.7676288845483, \"C(hour_of_week)[93]\": 26.14382239030146, \"C(hour_of_week)[94]\": 28.192777181335057, \"C(hour_of_week)[95]\": 28.27534938058132, \"C(hour_of_week)[96]\": 28.62022804185484, \"C(hour_of_week)[97]\": 29.02635597931267, \"C(hour_of_week)[98]\": 29.391081966169565, \"C(hour_of_week)[99]\": 29.698164274642615, \"C(hour_of_week)[100]\": 29.9212153978975, \"C(hour_of_week)[101]\": 30.26373856579102, \"C(hour_of_week)[102]\": 29.398533923918382, \"C(hour_of_week)[103]\": 24.707224656623723, \"C(hour_of_week)[104]\": 22.91794334076898, \"C(hour_of_week)[105]\": 22.263957312824157, \"C(hour_of_week)[106]\": 22.167354715588576, \"C(hour_of_week)[107]\": 21.965734169873702, \"C(hour_of_week)[108]\": 21.68677649798916, \"C(hour_of_week)[109]\": 21.366649960986255, \"C(hour_of_week)[110]\": 21.143496695548734, \"C(hour_of_week)[111]\": 20.690958534781053, \"C(hour_of_week)[112]\": 20.20273317845596, \"C(hour_of_week)[113]\": 20.301971188011024, \"C(hour_of_week)[114]\": 23.294334859680777, \"C(hour_of_week)[115]\": 25.11836818746646, \"C(hour_of_week)[116]\": 25.98323265600618, \"C(hour_of_week)[117]\": 26.580701582516248, \"C(hour_of_week)[118]\": 28.296192086429787, \"C(hour_of_week)[119]\": 26.733974603799894, \"C(hour_of_week)[120]\": 29.060730011454314, \"C(hour_of_week)[121]\": 29.619631938151976, \"C(hour_of_week)[122]\": 29.873905566019566, \"C(hour_of_week)[123]\": 30.290500324699167, \"C(hour_of_week)[124]\": 30.618365325283833, \"C(hour_of_week)[125]\": 31.281371184110824, \"C(hour_of_week)[126]\": 31.460024032466265, \"C(hour_of_week)[127]\": 30.576885459225505, \"C(hour_of_week)[128]\": 27.133113875058292, \"C(hour_of_week)[129]\": 25.858501688544358, \"C(hour_of_week)[130]\": 24.771611361912026, \"C(hour_of_week)[131]\": 23.809754193915268, \"C(hour_of_week)[132]\": 23.2233510625287, \"C(hour_of_week)[133]\": 21.80368545587041, \"C(hour_of_week)[134]\": 21.1855224723839, \"C(hour_of_week)[135]\": 21.41209723327908, \"C(hour_of_week)[136]\": 21.403765914054894, \"C(hour_of_week)[137]\": 21.98517017076719, \"C(hour_of_week)[138]\": 23.801834448277816, \"C(hour_of_week)[139]\": 25.511299123655746, \"C(hour_of_week)[140]\": 26.948034713113525, \"C(hour_of_week)[141]\": 29.697698815170355, \"C(hour_of_week)[142]\": 29.663212816709894, \"C(hour_of_week)[143]\": 30.13592171301013, \"C(hour_of_week)[144]\": 30.202063592995245, \"C(hour_of_week)[145]\": 30.18749849014261, \"C(hour_of_week)[146]\": 30.6765439457621, \"C(hour_of_week)[147]\": 30.681133416998353, \"C(hour_of_week)[148]\": 30.72073215230051, \"C(hour_of_week)[149]\": 31.080416635810433, \"C(hour_of_week)[150]\": 31.01233766914123, \"C(hour_of_week)[151]\": 30.38993895255213, \"C(hour_of_week)[152]\": 26.346436403718744, \"C(hour_of_week)[153]\": 24.135369577851325, \"C(hour_of_week)[154]\": 22.762205243751108, \"C(hour_of_week)[155]\": 21.783985327221774, \"C(hour_of_week)[156]\": 21.643093126157503, \"C(hour_of_week)[157]\": 21.26894561921078, \"C(hour_of_week)[158]\": 20.69703175805654, \"C(hour_of_week)[159]\": 20.56203478834863, \"C(hour_of_week)[160]\": 20.98784975661244, \"C(hour_of_week)[161]\": 22.03310475347403, \"C(hour_of_week)[162]\": 23.263109542729204, \"C(hour_of_week)[163]\": 24.992151137728143, \"C(hour_of_week)[164]\": 27.938597960330135, \"C(hour_of_week)[165]\": 28.465518649621863, \"C(hour_of_week)[166]\": 28.997096494272682, \"C(hour_of_week)[167]\": 28.89222807673672, \"bin_0_occupied\": -0.4776947145908701, \"bin_1_occupied\": -0.41218741798488834, \"bin_2_occupied\": -0.26483554007848864, \"bin_0_unoccupied\": -0.427042266557898, \"bin_1_unoccupied\": -0.2725225469513356, \"bin_2_unoccupied\": -0.05816825424413172, \"bin_3_unoccupied\": 0.07191198678486313}}, \"apr\": {\"segment_name\": \"mar-apr-may-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 24.94532743686949, \"C(hour_of_week)[1]\": 27.95841406218917, \"C(hour_of_week)[2]\": 25.358484280629895, \"C(hour_of_week)[3]\": 25.744341840041947, \"C(hour_of_week)[4]\": 25.898107831755766, \"C(hour_of_week)[5]\": 26.0402711176253, \"C(hour_of_week)[6]\": 25.53820628391771, \"C(hour_of_week)[7]\": 24.536994186235095, \"C(hour_of_week)[8]\": 25.77249582845344, \"C(hour_of_week)[9]\": 25.561562466249136, \"C(hour_of_week)[10]\": 25.987951683731154, \"C(hour_of_week)[11]\": 25.866548120420333, \"C(hour_of_week)[12]\": 25.33571241229781, \"C(hour_of_week)[13]\": 25.232945477832104, \"C(hour_of_week)[14]\": 25.16874398222068, \"C(hour_of_week)[15]\": 24.664661646312002, \"C(hour_of_week)[16]\": 24.40152524499718, \"C(hour_of_week)[17]\": 24.40693726739642, \"C(hour_of_week)[18]\": 24.719060019965383, \"C(hour_of_week)[19]\": 25.968789610571033, \"C(hour_of_week)[20]\": 27.32933893696577, \"C(hour_of_week)[21]\": 25.600695134797192, \"C(hour_of_week)[22]\": 27.85776619426793, \"C(hour_of_week)[23]\": 25.896463300682992, \"C(hour_of_week)[24]\": 25.915253650294986, \"C(hour_of_week)[25]\": 26.203732139486092, \"C(hour_of_week)[26]\": 26.34948351949061, \"C(hour_of_week)[27]\": 26.321727961827598, \"C(hour_of_week)[28]\": 26.368111312033435, \"C(hour_of_week)[29]\": 26.529355875594163, \"C(hour_of_week)[30]\": 25.983318109039157, \"C(hour_of_week)[31]\": 24.83028753462647, \"C(hour_of_week)[32]\": 25.661910100866514, \"C(hour_of_week)[33]\": 25.822409614023723, \"C(hour_of_week)[34]\": 26.171142081867863, \"C(hour_of_week)[35]\": 26.00260542153902, \"C(hour_of_week)[36]\": 25.862119618955145, \"C(hour_of_week)[37]\": 25.755446859416747, \"C(hour_of_week)[38]\": 25.90749566238141, \"C(hour_of_week)[39]\": 25.423021839938183, \"C(hour_of_week)[40]\": 25.31274570446314, \"C(hour_of_week)[41]\": 25.010149282201883, \"C(hour_of_week)[42]\": 24.559993985482414, \"C(hour_of_week)[43]\": 25.698525382657543, \"C(hour_of_week)[44]\": 26.725472137746152, \"C(hour_of_week)[45]\": 26.995802186671888, \"C(hour_of_week)[46]\": 25.228583453292227, \"C(hour_of_week)[47]\": 25.129457200504834, \"C(hour_of_week)[48]\": 25.35101821558891, \"C(hour_of_week)[49]\": 27.905117654831173, \"C(hour_of_week)[50]\": 25.788363588066684, \"C(hour_of_week)[51]\": 26.244918075458266, \"C(hour_of_week)[52]\": 26.36863046590338, \"C(hour_of_week)[53]\": 26.42758133312134, \"C(hour_of_week)[54]\": 26.166572141874077, \"C(hour_of_week)[55]\": 26.5757019332486, \"C(hour_of_week)[56]\": 26.070053470062348, \"C(hour_of_week)[57]\": 25.45164738799054, \"C(hour_of_week)[58]\": 25.93485548847692, \"C(hour_of_week)[59]\": 25.853494741792083, \"C(hour_of_week)[60]\": 25.646634376429848, \"C(hour_of_week)[61]\": 25.57081855189757, \"C(hour_of_week)[62]\": 25.036061606190135, \"C(hour_of_week)[63]\": 24.23240089187477, \"C(hour_of_week)[64]\": 24.060044315867447, \"C(hour_of_week)[65]\": 23.343784128619163, \"C(hour_of_week)[66]\": 23.246268302642783, \"C(hour_of_week)[67]\": 24.89364375294519, \"C(hour_of_week)[68]\": 25.87887224008219, \"C(hour_of_week)[69]\": 26.076815003373525, \"C(hour_of_week)[70]\": 26.74100153206202, \"C(hour_of_week)[71]\": 26.52082619643934, \"C(hour_of_week)[72]\": 26.74890607326323, \"C(hour_of_week)[73]\": 27.036065481783783, \"C(hour_of_week)[74]\": 27.408118860844144, \"C(hour_of_week)[75]\": 27.35347283753901, \"C(hour_of_week)[76]\": 27.54697835118427, \"C(hour_of_week)[77]\": 27.839808685094784, \"C(hour_of_week)[78]\": 27.143559890759278, \"C(hour_of_week)[79]\": 25.862332250363707, \"C(hour_of_week)[80]\": 24.914850834891862, \"C(hour_of_week)[81]\": 25.353449812475823, \"C(hour_of_week)[82]\": 26.111755803366897, \"C(hour_of_week)[83]\": 26.060879965869432, \"C(hour_of_week)[84]\": 25.99794520982336, \"C(hour_of_week)[85]\": 26.018476296817866, \"C(hour_of_week)[86]\": 25.806988457851304, \"C(hour_of_week)[87]\": 25.304203075319034, \"C(hour_of_week)[88]\": 24.59493455741477, \"C(hour_of_week)[89]\": 23.618955868438267, \"C(hour_of_week)[90]\": 22.621459168417342, \"C(hour_of_week)[91]\": 24.088094289772066, \"C(hour_of_week)[92]\": 25.433137245328258, \"C(hour_of_week)[93]\": 25.750115804185434, \"C(hour_of_week)[94]\": 26.365038611702374, \"C(hour_of_week)[95]\": 26.447387093795466, \"C(hour_of_week)[96]\": 26.858935630674857, \"C(hour_of_week)[97]\": 24.6354735413625, \"C(hour_of_week)[98]\": 24.808826654636864, \"C(hour_of_week)[99]\": 25.09592383956292, \"C(hour_of_week)[100]\": 25.33086242752335, \"C(hour_of_week)[101]\": 25.4394246512882, \"C(hour_of_week)[102]\": 24.639157271900658, \"C(hour_of_week)[103]\": 25.70842121599432, \"C(hour_of_week)[104]\": 24.80871998929216, \"C(hour_of_week)[105]\": 25.181411878416725, \"C(hour_of_week)[106]\": 25.625782416090768, \"C(hour_of_week)[107]\": 25.41022670060498, \"C(hour_of_week)[108]\": 25.28441101786722, \"C(hour_of_week)[109]\": 25.289050239681718, \"C(hour_of_week)[110]\": 24.93680085388033, \"C(hour_of_week)[111]\": 24.333916541413082, \"C(hour_of_week)[112]\": 23.733820905231248, \"C(hour_of_week)[113]\": 22.970669537342157, \"C(hour_of_week)[114]\": 23.973514844608673, \"C(hour_of_week)[115]\": 25.46415435951515, \"C(hour_of_week)[116]\": 26.326992779380763, \"C(hour_of_week)[117]\": 26.806774372647318, \"C(hour_of_week)[118]\": 27.202866306910494, \"C(hour_of_week)[119]\": 26.929993309857025, \"C(hour_of_week)[120]\": 24.877083772691382, \"C(hour_of_week)[121]\": 28.226027307530742, \"C(hour_of_week)[122]\": 25.690835905562135, \"C(hour_of_week)[123]\": 25.94869204946375, \"C(hour_of_week)[124]\": 26.346371490621117, \"C(hour_of_week)[125]\": 26.84049651702878, \"C(hour_of_week)[126]\": 26.485671085004906, \"C(hour_of_week)[127]\": 25.698818666081177, \"C(hour_of_week)[128]\": 27.250226360530313, \"C(hour_of_week)[129]\": 26.173351005778787, \"C(hour_of_week)[130]\": 25.194607984116143, \"C(hour_of_week)[131]\": 24.645146626599946, \"C(hour_of_week)[132]\": 24.53951860892081, \"C(hour_of_week)[133]\": 24.298384295753422, \"C(hour_of_week)[134]\": 23.758008890121964, \"C(hour_of_week)[135]\": 23.961562275219332, \"C(hour_of_week)[136]\": 23.96891322745426, \"C(hour_of_week)[137]\": 24.01492467826856, \"C(hour_of_week)[138]\": 24.360551574327545, \"C(hour_of_week)[139]\": 25.29713523037705, \"C(hour_of_week)[140]\": 26.732067993600822, \"C(hour_of_week)[141]\": 27.758539725416902, \"C(hour_of_week)[142]\": 27.71157252664556, \"C(hour_of_week)[143]\": 28.48251411541564, \"C(hour_of_week)[144]\": 28.357869732278, \"C(hour_of_week)[145]\": 25.23293848669381, \"C(hour_of_week)[146]\": 25.601493800507523, \"C(hour_of_week)[147]\": 26.191385429824926, \"C(hour_of_week)[148]\": 26.213582233037545, \"C(hour_of_week)[149]\": 26.70499819557181, \"C(hour_of_week)[150]\": 26.348590431553188, \"C(hour_of_week)[151]\": 25.894865756458742, \"C(hour_of_week)[152]\": 27.087064092936618, \"C(hour_of_week)[153]\": 25.23317263387358, \"C(hour_of_week)[154]\": 24.203527728659093, \"C(hour_of_week)[155]\": 23.448851444628612, \"C(hour_of_week)[156]\": 23.119404192346185, \"C(hour_of_week)[157]\": 22.68992700465754, \"C(hour_of_week)[158]\": 22.185281587432044, \"C(hour_of_week)[159]\": 22.22954552331926, \"C(hour_of_week)[160]\": 22.629722255028625, \"C(hour_of_week)[161]\": 23.016870240921893, \"C(hour_of_week)[162]\": 23.697884545227616, \"C(hour_of_week)[163]\": 24.863912317428934, \"C(hour_of_week)[164]\": 26.34355973620864, \"C(hour_of_week)[165]\": 27.024162553658797, \"C(hour_of_week)[166]\": 24.543922117247234, \"C(hour_of_week)[167]\": 27.4523665173592, \"bin_0_occupied\": -0.31288354776816923, \"bin_1_occupied\": -0.5097332126768013, \"bin_2_occupied\": -0.4911792137768626, \"bin_3_occupied\": -0.1947890203086559, \"bin_4_occupied\": 0.02633850336936243, \"bin_0_unoccupied\": -0.4540651803675606, \"bin_1_unoccupied\": -0.417641000573493, \"bin_2_unoccupied\": -0.13743962287054115, \"bin_3_unoccupied\": -0.09455961013287216, \"bin_4_unoccupied\": 0.2557824771519784, \"bin_5_unoccupied\": 0.15473917176742324}}, \"may\": {\"segment_name\": \"apr-may-jun-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 26.809299112573363, \"C(hour_of_week)[1]\": 26.551498757039358, \"C(hour_of_week)[2]\": 26.938705209900018, \"C(hour_of_week)[3]\": 27.1313786780544, \"C(hour_of_week)[4]\": 27.172983140202216, \"C(hour_of_week)[5]\": 26.96831795372348, \"C(hour_of_week)[6]\": 26.95941952825705, \"C(hour_of_week)[7]\": 20.150308914511932, \"C(hour_of_week)[8]\": 20.1426273995468, \"C(hour_of_week)[9]\": 20.552273497778227, \"C(hour_of_week)[10]\": 21.13320000052441, \"C(hour_of_week)[11]\": 20.976549145620783, \"C(hour_of_week)[12]\": 20.533476789006887, \"C(hour_of_week)[13]\": 20.578323633546773, \"C(hour_of_week)[14]\": 20.408591335246566, \"C(hour_of_week)[15]\": 19.896114082385715, \"C(hour_of_week)[16]\": 19.68187172961265, \"C(hour_of_week)[17]\": 18.92996899552198, \"C(hour_of_week)[18]\": 25.48482171016112, \"C(hour_of_week)[19]\": 25.8856834256579, \"C(hour_of_week)[20]\": 26.954644400633313, \"C(hour_of_week)[21]\": 27.131123786141156, \"C(hour_of_week)[22]\": 27.195669758412798, \"C(hour_of_week)[23]\": 26.953286466349375, \"C(hour_of_week)[24]\": 26.9753761890457, \"C(hour_of_week)[25]\": 27.215801155041625, \"C(hour_of_week)[26]\": 27.377567748715165, \"C(hour_of_week)[27]\": 27.389100899643164, \"C(hour_of_week)[28]\": 27.295071140604858, \"C(hour_of_week)[29]\": 27.20397616446049, \"C(hour_of_week)[30]\": 26.745965386674204, \"C(hour_of_week)[31]\": 19.529863766550136, \"C(hour_of_week)[32]\": 19.484199658063467, \"C(hour_of_week)[33]\": 20.42284442324013, \"C(hour_of_week)[34]\": 21.0595118734399, \"C(hour_of_week)[35]\": 20.917092797436045, \"C(hour_of_week)[36]\": 20.946401250720996, \"C(hour_of_week)[37]\": 20.89880243654278, \"C(hour_of_week)[38]\": 20.766285880249484, \"C(hour_of_week)[39]\": 20.286564103531216, \"C(hour_of_week)[40]\": 20.108868345615566, \"C(hour_of_week)[41]\": 19.020782560636103, \"C(hour_of_week)[42]\": 25.186836271766055, \"C(hour_of_week)[43]\": 25.231605568467, \"C(hour_of_week)[44]\": 25.68068654886593, \"C(hour_of_week)[45]\": 26.132197428360264, \"C(hour_of_week)[46]\": 26.37799859851469, \"C(hour_of_week)[47]\": 25.85296071253909, \"C(hour_of_week)[48]\": 25.839234609335143, \"C(hour_of_week)[49]\": 26.002942227535875, \"C(hour_of_week)[50]\": 26.153274305211607, \"C(hour_of_week)[51]\": 26.410947829373985, \"C(hour_of_week)[52]\": 26.503519454689403, \"C(hour_of_week)[53]\": 26.39823420438387, \"C(hour_of_week)[54]\": 26.654285619701792, \"C(hour_of_week)[55]\": 27.267488161727904, \"C(hour_of_week)[56]\": 19.702287002441388, \"C(hour_of_week)[57]\": 20.439633668254416, \"C(hour_of_week)[58]\": 20.952164530259584, \"C(hour_of_week)[59]\": 20.90481054773177, \"C(hour_of_week)[60]\": 20.82284900132815, \"C(hour_of_week)[61]\": 20.703873695015975, \"C(hour_of_week)[62]\": 20.373740143900303, \"C(hour_of_week)[63]\": 19.87481878149373, \"C(hour_of_week)[64]\": 19.777626529359075, \"C(hour_of_week)[65]\": 27.079096581432772, \"C(hour_of_week)[66]\": 25.304126482436605, \"C(hour_of_week)[67]\": 25.3931014640617, \"C(hour_of_week)[68]\": 25.75805379462641, \"C(hour_of_week)[69]\": 26.00883536393592, \"C(hour_of_week)[70]\": 26.145793213753475, \"C(hour_of_week)[71]\": 25.624694317513363, \"C(hour_of_week)[72]\": 25.641119677886422, \"C(hour_of_week)[73]\": 25.89502225124429, \"C(hour_of_week)[74]\": 26.256414332705667, \"C(hour_of_week)[75]\": 26.156061266557703, \"C(hour_of_week)[76]\": 26.485210179001562, \"C(hour_of_week)[77]\": 26.31274749660708, \"C(hour_of_week)[78]\": 25.69186355711933, \"C(hour_of_week)[79]\": 26.807108835197273, \"C(hour_of_week)[80]\": 19.161329966185352, \"C(hour_of_week)[81]\": 20.112123497154972, \"C(hour_of_week)[82]\": 20.920786571207856, \"C(hour_of_week)[83]\": 21.090813186650017, \"C(hour_of_week)[84]\": 21.070696747648434, \"C(hour_of_week)[85]\": 20.94206994010004, \"C(hour_of_week)[86]\": 20.831455299383503, \"C(hour_of_week)[87]\": 20.432770153818296, \"C(hour_of_week)[88]\": 19.837208215885322, \"C(hour_of_week)[89]\": 18.720700060722002, \"C(hour_of_week)[90]\": 24.51941679164439, \"C(hour_of_week)[91]\": 24.785106689742236, \"C(hour_of_week)[92]\": 25.22361610777215, \"C(hour_of_week)[93]\": 25.5718042399158, \"C(hour_of_week)[94]\": 25.93375658716609, \"C(hour_of_week)[95]\": 26.004916011817755, \"C(hour_of_week)[96]\": 26.117458375109976, \"C(hour_of_week)[97]\": 26.390760060647345, \"C(hour_of_week)[98]\": 26.583149354123055, \"C(hour_of_week)[99]\": 26.674709875082932, \"C(hour_of_week)[100]\": 26.796303063862418, \"C(hour_of_week)[101]\": 26.589072385184664, \"C(hour_of_week)[102]\": 26.01109473869544, \"C(hour_of_week)[103]\": 26.55934039477706, \"C(hour_of_week)[104]\": 26.97720851109976, \"C(hour_of_week)[105]\": 20.472139921322494, \"C(hour_of_week)[106]\": 21.085520239088996, \"C(hour_of_week)[107]\": 20.95821062000301, \"C(hour_of_week)[108]\": 20.91044990325138, \"C(hour_of_week)[109]\": 20.93548230212962, \"C(hour_of_week)[110]\": 20.702062830802152, \"C(hour_of_week)[111]\": 20.49225181314433, \"C(hour_of_week)[112]\": 20.077691804450016, \"C(hour_of_week)[113]\": 19.08718431225351, \"C(hour_of_week)[114]\": 25.123592460839312, \"C(hour_of_week)[115]\": 25.47596709685693, \"C(hour_of_week)[116]\": 25.86232217086545, \"C(hour_of_week)[117]\": 26.550615497284202, \"C(hour_of_week)[118]\": 26.625727065370388, \"C(hour_of_week)[119]\": 26.18060374869981, \"C(hour_of_week)[120]\": 26.608854584293685, \"C(hour_of_week)[121]\": 26.99762174039028, \"C(hour_of_week)[122]\": 27.231821143439362, \"C(hour_of_week)[123]\": 27.265092129919413, \"C(hour_of_week)[124]\": 27.547731360871442, \"C(hour_of_week)[125]\": 20.840440740529317, \"C(hour_of_week)[126]\": 26.839521214450155, \"C(hour_of_week)[127]\": 26.164765689817372, \"C(hour_of_week)[128]\": 25.773710108487254, \"C(hour_of_week)[129]\": 25.346179887862448, \"C(hour_of_week)[130]\": 24.8040600073316, \"C(hour_of_week)[131]\": 24.641922542738467, \"C(hour_of_week)[132]\": 25.54459541735725, \"C(hour_of_week)[133]\": 26.372081181953043, \"C(hour_of_week)[134]\": 26.195537179065585, \"C(hour_of_week)[135]\": 26.30351765717446, \"C(hour_of_week)[136]\": 26.391721538050447, \"C(hour_of_week)[137]\": 26.451718244242794, \"C(hour_of_week)[138]\": 25.71952640630832, \"C(hour_of_week)[139]\": 25.30660033113384, \"C(hour_of_week)[140]\": 26.024688670754642, \"C(hour_of_week)[141]\": 26.66582880037223, \"C(hour_of_week)[142]\": 26.37878321156504, \"C(hour_of_week)[143]\": 26.751227952496425, \"C(hour_of_week)[144]\": 26.628604281963653, \"C(hour_of_week)[145]\": 26.792090771626146, \"C(hour_of_week)[146]\": 26.99211375215611, \"C(hour_of_week)[147]\": 27.498165622667354, \"C(hour_of_week)[148]\": 27.319734054294454, \"C(hour_of_week)[149]\": 27.739273338832692, \"C(hour_of_week)[150]\": 26.905485165889754, \"C(hour_of_week)[151]\": 26.648602916498753, \"C(hour_of_week)[152]\": 25.979104063954694, \"C(hour_of_week)[153]\": 25.13385773425032, \"C(hour_of_week)[154]\": 24.725603503812223, \"C(hour_of_week)[155]\": 24.278149962880693, \"C(hour_of_week)[156]\": 24.099580828346195, \"C(hour_of_week)[157]\": 23.84554253500937, \"C(hour_of_week)[158]\": 23.59797133693031, \"C(hour_of_week)[159]\": 23.933947926820576, \"C(hour_of_week)[160]\": 24.06488323890634, \"C(hour_of_week)[161]\": 24.00375930205891, \"C(hour_of_week)[162]\": 24.263083681926314, \"C(hour_of_week)[163]\": 24.277690665334394, \"C(hour_of_week)[164]\": 25.56488867200746, \"C(hour_of_week)[165]\": 26.04189754518007, \"C(hour_of_week)[166]\": 26.212838622203574, \"C(hour_of_week)[167]\": 26.326027800924063, \"bin_0_occupied\": -0.33037572797740017, \"bin_1_occupied\": 0.016895543781330024, \"bin_2_occupied\": 0.03901815834378147, \"bin_3_occupied\": 0.10537627117913564, \"bin_4_occupied\": 0.15222368750805618, \"bin_0_unoccupied\": -0.3768106512847225, \"bin_1_unoccupied\": -0.5165553933295964, \"bin_2_unoccupied\": -0.45010531099154466, \"bin_3_unoccupied\": -0.09471166401732745, \"bin_4_unoccupied\": 0.10978579316320011, \"bin_5_unoccupied\": 0.22803938324709278}}, \"jun\": {\"segment_name\": \"may-jun-jul-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 60.67650222249238, \"C(hour_of_week)[1]\": 60.41615726825565, \"C(hour_of_week)[2]\": 60.56919699780234, \"C(hour_of_week)[3]\": 60.54327696233125, \"C(hour_of_week)[4]\": 60.51813496364416, \"C(hour_of_week)[5]\": 60.07959813480906, \"C(hour_of_week)[6]\": 61.01089807260293, \"C(hour_of_week)[7]\": 10.496281621340083, \"C(hour_of_week)[8]\": 11.551539968320114, \"C(hour_of_week)[9]\": 12.574809200315517, \"C(hour_of_week)[10]\": 13.439553601055259, \"C(hour_of_week)[11]\": 13.447531109359979, \"C(hour_of_week)[12]\": 13.038852679153836, \"C(hour_of_week)[13]\": 13.128031287525578, \"C(hour_of_week)[14]\": 12.848683863330688, \"C(hour_of_week)[15]\": 12.420163460708771, \"C(hour_of_week)[16]\": 12.093431686083136, \"C(hour_of_week)[17]\": 10.781624419804842, \"C(hour_of_week)[18]\": 61.62419468469082, \"C(hour_of_week)[19]\": 61.15215404192815, \"C(hour_of_week)[20]\": 61.311892828157035, \"C(hour_of_week)[21]\": 61.65154481923449, \"C(hour_of_week)[22]\": 61.17931221167856, \"C(hour_of_week)[23]\": 60.605960822988116, \"C(hour_of_week)[24]\": 60.48749973691623, \"C(hour_of_week)[25]\": 60.676057139002516, \"C(hour_of_week)[26]\": 60.73149943255651, \"C(hour_of_week)[27]\": 60.67629561739463, \"C(hour_of_week)[28]\": 60.632034036986056, \"C(hour_of_week)[29]\": 60.29463149375192, \"C(hour_of_week)[30]\": 60.54177823206151, \"C(hour_of_week)[31]\": 10.24029358342246, \"C(hour_of_week)[32]\": 11.113915065162196, \"C(hour_of_week)[33]\": 12.378758825065303, \"C(hour_of_week)[34]\": 13.021778477471052, \"C(hour_of_week)[35]\": 12.985859972067773, \"C(hour_of_week)[36]\": 13.01805799988418, \"C(hour_of_week)[37]\": 12.911494400238302, \"C(hour_of_week)[38]\": 12.602396034555971, \"C(hour_of_week)[39]\": 12.445214297663174, \"C(hour_of_week)[40]\": 11.990414758971365, \"C(hour_of_week)[41]\": 10.650295582228507, \"C(hour_of_week)[42]\": 61.01257826063294, \"C(hour_of_week)[43]\": 60.244329700533974, \"C(hour_of_week)[44]\": 60.26222933752997, \"C(hour_of_week)[45]\": 60.797013370521974, \"C(hour_of_week)[46]\": 60.35302542264533, \"C(hour_of_week)[47]\": 59.84931082920367, \"C(hour_of_week)[48]\": 59.61755666468906, \"C(hour_of_week)[49]\": 59.73499480137057, \"C(hour_of_week)[50]\": 59.677079103681095, \"C(hour_of_week)[51]\": 59.659700958645736, \"C(hour_of_week)[52]\": 59.87401940592316, \"C(hour_of_week)[53]\": 59.59385906744005, \"C(hour_of_week)[54]\": 60.161990676624306, \"C(hour_of_week)[55]\": 9.714265287676835, \"C(hour_of_week)[56]\": 11.021608092015974, \"C(hour_of_week)[57]\": 12.203021668587295, \"C(hour_of_week)[58]\": 12.851261566082647, \"C(hour_of_week)[59]\": 12.979001889910284, \"C(hour_of_week)[60]\": 12.833224194181913, \"C(hour_of_week)[61]\": 12.729352201221701, \"C(hour_of_week)[62]\": 12.586692498361575, \"C(hour_of_week)[63]\": 12.26526532363907, \"C(hour_of_week)[64]\": 11.954647735080119, \"C(hour_of_week)[65]\": 10.664518656870976, \"C(hour_of_week)[66]\": 61.815708840699735, \"C(hour_of_week)[67]\": 60.83634628040101, \"C(hour_of_week)[68]\": 60.706176783278, \"C(hour_of_week)[69]\": 60.95048601153016, \"C(hour_of_week)[70]\": 60.48605555362514, \"C(hour_of_week)[71]\": 59.91614041366424, \"C(hour_of_week)[72]\": 59.761095472551794, \"C(hour_of_week)[73]\": 60.013429340400016, \"C(hour_of_week)[74]\": 60.17746460940701, \"C(hour_of_week)[75]\": 60.119328864710525, \"C(hour_of_week)[76]\": 60.293503577681186, \"C(hour_of_week)[77]\": 59.8617298049054, \"C(hour_of_week)[78]\": 59.924317236326885, \"C(hour_of_week)[79]\": 9.644211301030907, \"C(hour_of_week)[80]\": 10.88969749468703, \"C(hour_of_week)[81]\": 12.022132530450477, \"C(hour_of_week)[82]\": 12.909880926342245, \"C(hour_of_week)[83]\": 13.014643723287275, \"C(hour_of_week)[84]\": 12.979356370349485, \"C(hour_of_week)[85]\": 12.780678240210719, \"C(hour_of_week)[86]\": 12.55777040936484, \"C(hour_of_week)[87]\": 12.317913551915105, \"C(hour_of_week)[88]\": 11.67382348519997, \"C(hour_of_week)[89]\": 10.455729135607472, \"C(hour_of_week)[90]\": 60.89462573265589, \"C(hour_of_week)[91]\": 60.33096947760878, \"C(hour_of_week)[92]\": 60.355212048621155, \"C(hour_of_week)[93]\": 60.71708260521646, \"C(hour_of_week)[94]\": 60.451938458612325, \"C(hour_of_week)[95]\": 60.17023122914937, \"C(hour_of_week)[96]\": 60.07810941290483, \"C(hour_of_week)[97]\": 60.27023867521617, \"C(hour_of_week)[98]\": 60.44132937090376, \"C(hour_of_week)[99]\": 60.46628591715299, \"C(hour_of_week)[100]\": 60.51481253527568, \"C(hour_of_week)[101]\": 60.17622852633339, \"C(hour_of_week)[102]\": 60.12964569904602, \"C(hour_of_week)[103]\": 10.161771732146853, \"C(hour_of_week)[104]\": 11.093660184718221, \"C(hour_of_week)[105]\": 12.153239391511189, \"C(hour_of_week)[106]\": 12.980735545584574, \"C(hour_of_week)[107]\": 13.022951715784332, \"C(hour_of_week)[108]\": 12.779569569958396, \"C(hour_of_week)[109]\": 12.776138039460646, \"C(hour_of_week)[110]\": 12.695124961533525, \"C(hour_of_week)[111]\": 12.528975044967314, \"C(hour_of_week)[112]\": 12.039864316190881, \"C(hour_of_week)[113]\": 10.766999815109605, \"C(hour_of_week)[114]\": 60.859929293281525, \"C(hour_of_week)[115]\": 60.352244284844915, \"C(hour_of_week)[116]\": 60.42675831148539, \"C(hour_of_week)[117]\": 60.92752400844525, \"C(hour_of_week)[118]\": 60.45373728918682, \"C(hour_of_week)[119]\": 60.017984775612106, \"C(hour_of_week)[120]\": 60.38956488956445, \"C(hour_of_week)[121]\": 60.759128252267296, \"C(hour_of_week)[122]\": 60.91559325372759, \"C(hour_of_week)[123]\": 60.99469863862249, \"C(hour_of_week)[124]\": 61.106248471587875, \"C(hour_of_week)[125]\": 60.73010081187387, \"C(hour_of_week)[126]\": 60.234494618160554, \"C(hour_of_week)[127]\": 59.78096948038826, \"C(hour_of_week)[128]\": 59.55201658438834, \"C(hour_of_week)[129]\": 59.547432298702205, \"C(hour_of_week)[130]\": 59.49759622221076, \"C(hour_of_week)[131]\": 59.5688094670692, \"C(hour_of_week)[132]\": 61.2698132148873, \"C(hour_of_week)[133]\": 9.948181034481177, \"C(hour_of_week)[134]\": 62.57421636805126, \"C(hour_of_week)[135]\": 62.63883118498022, \"C(hour_of_week)[136]\": 62.573012502350636, \"C(hour_of_week)[137]\": 10.00797977879207, \"C(hour_of_week)[138]\": 61.4387669039969, \"C(hour_of_week)[139]\": 60.46405171344144, \"C(hour_of_week)[140]\": 60.401760456019225, \"C(hour_of_week)[141]\": 60.77425931700524, \"C(hour_of_week)[142]\": 60.361412119049774, \"C(hour_of_week)[143]\": 60.51066993619704, \"C(hour_of_week)[144]\": 60.4202778156332, \"C(hour_of_week)[145]\": 60.47420148645551, \"C(hour_of_week)[146]\": 60.523397305160515, \"C(hour_of_week)[147]\": 60.76512466692259, \"C(hour_of_week)[148]\": 60.49488052823777, \"C(hour_of_week)[149]\": 60.51731427220481, \"C(hour_of_week)[150]\": 59.715315344683056, \"C(hour_of_week)[151]\": 59.864318387509385, \"C(hour_of_week)[152]\": 59.97744725484236, \"C(hour_of_week)[153]\": 60.00379045198551, \"C(hour_of_week)[154]\": 59.98791256467151, \"C(hour_of_week)[155]\": 59.68118744783514, \"C(hour_of_week)[156]\": 59.59927338810505, \"C(hour_of_week)[157]\": 59.549460201486916, \"C(hour_of_week)[158]\": 59.67809756872401, \"C(hour_of_week)[159]\": 60.02988638432881, \"C(hour_of_week)[160]\": 60.05005397345205, \"C(hour_of_week)[161]\": 59.78137420724008, \"C(hour_of_week)[162]\": 59.84918607726996, \"C(hour_of_week)[163]\": 59.61925297319014, \"C(hour_of_week)[164]\": 60.103651498209295, \"C(hour_of_week)[165]\": 60.26664980110008, \"C(hour_of_week)[166]\": 60.23539797374825, \"C(hour_of_week)[167]\": 60.43285637595931, \"bin_0_occupied\": -0.1180231413287284, \"bin_1_occupied\": 0.035710448406035544, \"bin_2_occupied\": 0.13256090210357388, \"bin_3_occupied\": 0.15161162776715645, \"bin_0_unoccupied\": -1.1879322949037743, \"bin_1_unoccupied\": -0.4572954874548413, \"bin_2_unoccupied\": -0.07465341460632846, \"bin_3_unoccupied\": 0.05387622395435813, \"bin_4_unoccupied\": 0.20783973236877415}}, \"jul\": {\"segment_name\": \"jun-jul-aug-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 9.690608528702803, \"C(hour_of_week)[1]\": 9.486031769326488, \"C(hour_of_week)[2]\": 9.476208657812649, \"C(hour_of_week)[3]\": 9.469355697109204, \"C(hour_of_week)[4]\": 9.48020848789061, \"C(hour_of_week)[5]\": 9.138654537755862, \"C(hour_of_week)[6]\": 10.50139521103516, \"C(hour_of_week)[7]\": 3.026473487878018, \"C(hour_of_week)[8]\": 4.164834483329621, \"C(hour_of_week)[9]\": 5.159964341155153, \"C(hour_of_week)[10]\": 6.120641568612342, \"C(hour_of_week)[11]\": 6.189967044658065, \"C(hour_of_week)[12]\": 5.841975234502378, \"C(hour_of_week)[13]\": 5.963738810878539, \"C(hour_of_week)[14]\": 5.606135409137146, \"C(hour_of_week)[15]\": 5.259871821794395, \"C(hour_of_week)[16]\": 4.890277886188841, \"C(hour_of_week)[17]\": 3.4018480143955045, \"C(hour_of_week)[18]\": 12.453251669378911, \"C(hour_of_week)[19]\": 11.41754419845419, \"C(hour_of_week)[20]\": 11.047692201357775, \"C(hour_of_week)[21]\": 11.262842984575432, \"C(hour_of_week)[22]\": 10.223110212847795, \"C(hour_of_week)[23]\": 9.599073921600011, \"C(hour_of_week)[24]\": 9.34731468560463, \"C(hour_of_week)[25]\": 9.492907687062704, \"C(hour_of_week)[26]\": 9.494584418508275, \"C(hour_of_week)[27]\": 9.44207223436857, \"C(hour_of_week)[28]\": 9.380749478287083, \"C(hour_of_week)[29]\": 9.133539510588065, \"C(hour_of_week)[30]\": 10.176607609148979, \"C(hour_of_week)[31]\": 2.7125564560188877, \"C(hour_of_week)[32]\": 3.856894574221677, \"C(hour_of_week)[33]\": 5.167560741511714, \"C(hour_of_week)[34]\": 5.669139661098907, \"C(hour_of_week)[35]\": 5.84305365463517, \"C(hour_of_week)[36]\": 5.853302020012606, \"C(hour_of_week)[37]\": 5.554538909985226, \"C(hour_of_week)[38]\": 5.266390895866113, \"C(hour_of_week)[39]\": 5.345703784077667, \"C(hour_of_week)[40]\": 4.509612428006058, \"C(hour_of_week)[41]\": 3.2221254766347123, \"C(hour_of_week)[42]\": 11.857196362604089, \"C(hour_of_week)[43]\": 10.575819332825622, \"C(hour_of_week)[44]\": 10.723007104193108, \"C(hour_of_week)[45]\": 11.085197494664634, \"C(hour_of_week)[46]\": 9.985435461042863, \"C(hour_of_week)[47]\": 9.590830858726525, \"C(hour_of_week)[48]\": 9.383640686105913, \"C(hour_of_week)[49]\": 9.51452972507969, \"C(hour_of_week)[50]\": 9.669717249702693, \"C(hour_of_week)[51]\": 9.539717582598705, \"C(hour_of_week)[52]\": 9.495832425852477, \"C(hour_of_week)[53]\": 9.164968818982377, \"C(hour_of_week)[54]\": 9.636261136588782, \"C(hour_of_week)[55]\": 2.4733069168080632, \"C(hour_of_week)[56]\": 3.952557754659086, \"C(hour_of_week)[57]\": 4.824373653422509, \"C(hour_of_week)[58]\": 5.681078364080193, \"C(hour_of_week)[59]\": 5.956278599405682, \"C(hour_of_week)[60]\": 5.599689469151592, \"C(hour_of_week)[61]\": 5.585886044991774, \"C(hour_of_week)[62]\": 5.60941065121418, \"C(hour_of_week)[63]\": 5.23083254754291, \"C(hour_of_week)[64]\": 4.750825129653364, \"C(hour_of_week)[65]\": 3.501635328022452, \"C(hour_of_week)[66]\": 12.195487038116822, \"C(hour_of_week)[67]\": 10.99388884199698, \"C(hour_of_week)[68]\": 10.861581140504214, \"C(hour_of_week)[69]\": 10.922154358626887, \"C(hour_of_week)[70]\": 10.135150867103194, \"C(hour_of_week)[71]\": 9.500744327032876, \"C(hour_of_week)[72]\": 9.24702844055628, \"C(hour_of_week)[73]\": 9.412008798620079, \"C(hour_of_week)[74]\": 9.459768313031066, \"C(hour_of_week)[75]\": 9.391993800952102, \"C(hour_of_week)[76]\": 9.400803430986365, \"C(hour_of_week)[77]\": 9.112216401776855, \"C(hour_of_week)[78]\": 9.807952604036341, \"C(hour_of_week)[79]\": 2.430110382512389, \"C(hour_of_week)[80]\": 3.5931153924017565, \"C(hour_of_week)[81]\": 4.861792653484315, \"C(hour_of_week)[82]\": 5.680447383180336, \"C(hour_of_week)[83]\": 5.644504188007382, \"C(hour_of_week)[84]\": 5.732866384921891, \"C(hour_of_week)[85]\": 5.526078557123668, \"C(hour_of_week)[86]\": 5.178891601192831, \"C(hour_of_week)[87]\": 5.256626728770961, \"C(hour_of_week)[88]\": 4.594650921690995, \"C(hour_of_week)[89]\": 3.077187801022447, \"C(hour_of_week)[90]\": 11.781757562222172, \"C(hour_of_week)[91]\": 10.667844821366698, \"C(hour_of_week)[92]\": 10.64264271431917, \"C(hour_of_week)[93]\": 11.007679754798186, \"C(hour_of_week)[94]\": 9.96951247026564, \"C(hour_of_week)[95]\": 9.548436243251839, \"C(hour_of_week)[96]\": 9.339139776692813, \"C(hour_of_week)[97]\": 9.506651461945825, \"C(hour_of_week)[98]\": 9.576879124520953, \"C(hour_of_week)[99]\": 9.581288894559504, \"C(hour_of_week)[100]\": 9.684318698467631, \"C(hour_of_week)[101]\": 9.321861675995034, \"C(hour_of_week)[102]\": 9.788217124642216, \"C(hour_of_week)[103]\": 2.631414963633457, \"C(hour_of_week)[104]\": 3.9609344622914726, \"C(hour_of_week)[105]\": 4.872482505078288, \"C(hour_of_week)[106]\": 5.762052469888751, \"C(hour_of_week)[107]\": 5.897379504296278, \"C(hour_of_week)[108]\": 5.535621431254118, \"C(hour_of_week)[109]\": 5.511556778330792, \"C(hour_of_week)[110]\": 5.6140072708471465, \"C(hour_of_week)[111]\": 5.210474108377973, \"C(hour_of_week)[112]\": 4.708922146791419, \"C(hour_of_week)[113]\": 3.405571187308798, \"C(hour_of_week)[114]\": 11.553166435928183, \"C(hour_of_week)[115]\": 10.686937636858392, \"C(hour_of_week)[116]\": 10.654737928003918, \"C(hour_of_week)[117]\": 10.734639091209036, \"C(hour_of_week)[118]\": 10.077109386907573, \"C(hour_of_week)[119]\": 9.564076499523331, \"C(hour_of_week)[120]\": 9.945349984751845, \"C(hour_of_week)[121]\": 10.313001000789182, \"C(hour_of_week)[122]\": 10.447956489496498, \"C(hour_of_week)[123]\": 10.496011912851976, \"C(hour_of_week)[124]\": 10.47768107691954, \"C(hour_of_week)[125]\": 10.144246176063962, \"C(hour_of_week)[126]\": 9.544628575078887, \"C(hour_of_week)[127]\": 9.196818787202142, \"C(hour_of_week)[128]\": 8.98860014225951, \"C(hour_of_week)[129]\": 9.027471205179914, \"C(hour_of_week)[130]\": 9.195754180851015, \"C(hour_of_week)[131]\": 9.34070051864865, \"C(hour_of_week)[132]\": 11.529954506448561, \"C(hour_of_week)[133]\": 2.6665388362130926, \"C(hour_of_week)[134]\": 12.826694733110468, \"C(hour_of_week)[135]\": 12.884832097550618, \"C(hour_of_week)[136]\": 12.75847742148832, \"C(hour_of_week)[137]\": 12.734913480618829, \"C(hour_of_week)[138]\": 11.533113534084674, \"C(hour_of_week)[139]\": 10.596478174191892, \"C(hour_of_week)[140]\": 10.331590514517611, \"C(hour_of_week)[141]\": 10.334936189018215, \"C(hour_of_week)[142]\": 9.948445833852459, \"C(hour_of_week)[143]\": 10.035821696596692, \"C(hour_of_week)[144]\": 9.52134694732889, \"C(hour_of_week)[145]\": 9.402793377863794, \"C(hour_of_week)[146]\": 9.421615039160972, \"C(hour_of_week)[147]\": 9.443651353701934, \"C(hour_of_week)[148]\": 9.464382403437227, \"C(hour_of_week)[149]\": 9.24388023993942, \"C(hour_of_week)[150]\": 8.660613700100452, \"C(hour_of_week)[151]\": 9.153641335414093, \"C(hour_of_week)[152]\": 9.4746885190151, \"C(hour_of_week)[153]\": 9.763445103133677, \"C(hour_of_week)[154]\": 9.82295234528213, \"C(hour_of_week)[155]\": 9.57141777282848, \"C(hour_of_week)[156]\": 9.514246878104686, \"C(hour_of_week)[157]\": 9.558029637806825, \"C(hour_of_week)[158]\": 9.849898383891663, \"C(hour_of_week)[159]\": 10.21045202267707, \"C(hour_of_week)[160]\": 10.3377426165394, \"C(hour_of_week)[161]\": 10.039732129291064, \"C(hour_of_week)[162]\": 9.802109129494273, \"C(hour_of_week)[163]\": 9.381721622331458, \"C(hour_of_week)[164]\": 9.519178632178523, \"C(hour_of_week)[165]\": 9.484588701932717, \"C(hour_of_week)[166]\": 9.459396395565031, \"C(hour_of_week)[167]\": 9.549925393707973, \"bin_0_occupied\": 0.01905918165387921, \"bin_1_occupied\": 0.1519842620276844, \"bin_2_occupied\": 0.13554187373401674, \"bin_0_unoccupied\": -0.14052662682892825, \"bin_1_unoccupied\": -0.053106211540812034, \"bin_2_unoccupied\": 0.04475248775873241, \"bin_3_unoccupied\": 0.18459889342587046}}, \"aug\": {\"segment_name\": \"jul-aug-sep-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 16.37987014202073, \"C(hour_of_week)[1]\": 16.15764192634435, \"C(hour_of_week)[2]\": 16.12726627366887, \"C(hour_of_week)[3]\": 16.064562550298902, \"C(hour_of_week)[4]\": 16.013834877663324, \"C(hour_of_week)[5]\": 15.901576717792164, \"C(hour_of_week)[6]\": 17.48766220063024, \"C(hour_of_week)[7]\": -0.13081474027286788, \"C(hour_of_week)[8]\": 1.004395911684945, \"C(hour_of_week)[9]\": 1.9961886062205245, \"C(hour_of_week)[10]\": 3.043108028885957, \"C(hour_of_week)[11]\": 3.0922399814005566, \"C(hour_of_week)[12]\": 2.7990836914463415, \"C(hour_of_week)[13]\": 2.8837186611263625, \"C(hour_of_week)[14]\": 2.508957090378118, \"C(hour_of_week)[15]\": 2.1464256202268928, \"C(hour_of_week)[16]\": 1.7406396648914217, \"C(hour_of_week)[17]\": 0.22128525924161835, \"C(hour_of_week)[18]\": 19.248786605754013, \"C(hour_of_week)[19]\": 18.260987822605692, \"C(hour_of_week)[20]\": 17.875533498326714, \"C(hour_of_week)[21]\": 18.091760004835194, \"C(hour_of_week)[22]\": 16.811670771140125, \"C(hour_of_week)[23]\": 16.220060938367475, \"C(hour_of_week)[24]\": 15.961164004862113, \"C(hour_of_week)[25]\": 16.10762216587754, \"C(hour_of_week)[26]\": 16.090760983479587, \"C(hour_of_week)[27]\": 16.023280380147362, \"C(hour_of_week)[28]\": 15.99108464334176, \"C(hour_of_week)[29]\": 15.961821871054825, \"C(hour_of_week)[30]\": 17.23352302430612, \"C(hour_of_week)[31]\": -0.5919856540579111, \"C(hour_of_week)[32]\": 0.7011756614081719, \"C(hour_of_week)[33]\": 2.068518903698477, \"C(hour_of_week)[34]\": 2.4827684332781335, \"C(hour_of_week)[35]\": 2.7640131395403174, \"C(hour_of_week)[36]\": 2.8207280302667748, \"C(hour_of_week)[37]\": 2.394020635230076, \"C(hour_of_week)[38]\": 2.183827743566116, \"C(hour_of_week)[39]\": 2.2707045773430146, \"C(hour_of_week)[40]\": 1.1946955537737693, \"C(hour_of_week)[41]\": -0.014568046172136917, \"C(hour_of_week)[42]\": 18.754923767801387, \"C(hour_of_week)[43]\": 17.47360992820215, \"C(hour_of_week)[44]\": 17.813560117824238, \"C(hour_of_week)[45]\": 17.970261421720185, \"C(hour_of_week)[46]\": 16.683780707287454, \"C(hour_of_week)[47]\": 16.378119631701445, \"C(hour_of_week)[48]\": 16.130373570719552, \"C(hour_of_week)[49]\": 16.236576845979748, \"C(hour_of_week)[50]\": 16.454077893270988, \"C(hour_of_week)[51]\": 16.20954024917702, \"C(hour_of_week)[52]\": 16.066420683976137, \"C(hour_of_week)[53]\": 15.999480782509423, \"C(hour_of_week)[54]\": 16.473691179637882, \"C(hour_of_week)[55]\": -0.6171564986047997, \"C(hour_of_week)[56]\": 0.8947708503884328, \"C(hour_of_week)[57]\": 1.6625695639954898, \"C(hour_of_week)[58]\": 2.746240452513886, \"C(hour_of_week)[59]\": 3.062912892188155, \"C(hour_of_week)[60]\": 2.5726640186117855, \"C(hour_of_week)[61]\": 2.679118204376252, \"C(hour_of_week)[62]\": 2.729948829056415, \"C(hour_of_week)[63]\": 2.186606025240854, \"C(hour_of_week)[64]\": 1.7024696928862078, \"C(hour_of_week)[65]\": 0.43636043454694473, \"C(hour_of_week)[66]\": 18.574063896354424, \"C(hour_of_week)[67]\": 17.808542492328648, \"C(hour_of_week)[68]\": 17.647560513747493, \"C(hour_of_week)[69]\": 17.55100604028615, \"C(hour_of_week)[70]\": 16.950629114685103, \"C(hour_of_week)[71]\": 16.284554491329025, \"C(hour_of_week)[72]\": 16.020217756499363, \"C(hour_of_week)[73]\": 16.124770033721557, \"C(hour_of_week)[74]\": 16.171047837860062, \"C(hour_of_week)[75]\": 16.115435776125786, \"C(hour_of_week)[76]\": 16.1064462353893, \"C(hour_of_week)[77]\": 16.061127203232967, \"C(hour_of_week)[78]\": 16.732758652040886, \"C(hour_of_week)[79]\": -0.5203886476926733, \"C(hour_of_week)[80]\": 0.491438291421737, \"C(hour_of_week)[81]\": 1.9797083199230254, \"C(hour_of_week)[82]\": 2.7170688960673406, \"C(hour_of_week)[83]\": 2.646991473368331, \"C(hour_of_week)[84]\": 2.8484108596531197, \"C(hour_of_week)[85]\": 2.6157573207441587, \"C(hour_of_week)[86]\": 2.2572898901360787, \"C(hour_of_week)[87]\": 2.498241241512, \"C(hour_of_week)[88]\": 1.8015187435402495, \"C(hour_of_week)[89]\": 0.17147395475086924, \"C(hour_of_week)[90]\": 18.816353587395916, \"C(hour_of_week)[91]\": 17.551063425252515, \"C(hour_of_week)[92]\": 17.513325537024336, \"C(hour_of_week)[93]\": 17.87293153159593, \"C(hour_of_week)[94]\": 16.633980896368417, \"C(hour_of_week)[95]\": 16.231094897405754, \"C(hour_of_week)[96]\": 15.991416471422452, \"C(hour_of_week)[97]\": 16.148055221140936, \"C(hour_of_week)[98]\": 16.18178321954306, \"C(hour_of_week)[99]\": 16.203751447286248, \"C(hour_of_week)[100]\": 16.361800161960623, \"C(hour_of_week)[101]\": 16.18228194835862, \"C(hour_of_week)[102]\": 16.655491189016164, \"C(hour_of_week)[103]\": -0.6845188122180161, \"C(hour_of_week)[104]\": 0.8939794221595996, \"C(hour_of_week)[105]\": 1.7180627304854674, \"C(hour_of_week)[106]\": 2.58178715314196, \"C(hour_of_week)[107]\": 2.890769840879912, \"C(hour_of_week)[108]\": 2.5116141516958024, \"C(hour_of_week)[109]\": 2.4863708981695005, \"C(hour_of_week)[110]\": 2.668022407407344, \"C(hour_of_week)[111]\": 2.0900268199597996, \"C(hour_of_week)[112]\": 1.6097795944659303, \"C(hour_of_week)[113]\": 0.4025451529956001, \"C(hour_of_week)[114]\": 18.239615137820984, \"C(hour_of_week)[115]\": 17.609531347877727, \"C(hour_of_week)[116]\": 17.656124662762153, \"C(hour_of_week)[117]\": 17.53032523263551, \"C(hour_of_week)[118]\": 17.043088783790804, \"C(hour_of_week)[119]\": 16.38809692611024, \"C(hour_of_week)[120]\": 16.751514139735786, \"C(hour_of_week)[121]\": 17.063956655200236, \"C(hour_of_week)[122]\": 17.156723849965427, \"C(hour_of_week)[123]\": 17.174265343121164, \"C(hour_of_week)[124]\": 17.12951471221455, \"C(hour_of_week)[125]\": 17.013621609500305, \"C(hour_of_week)[126]\": 16.49184996069778, \"C(hour_of_week)[127]\": 15.962112249943978, \"C(hour_of_week)[128]\": 15.773385487046838, \"C(hour_of_week)[129]\": 15.809415384485, \"C(hour_of_week)[130]\": 15.978388533351259, \"C(hour_of_week)[131]\": 16.120786115957387, \"C(hour_of_week)[132]\": 18.459854578151116, \"C(hour_of_week)[133]\": -0.27356795117660226, \"C(hour_of_week)[134]\": 19.742481605522784, \"C(hour_of_week)[135]\": 19.819979501329364, \"C(hour_of_week)[136]\": -0.4838250483234745, \"C(hour_of_week)[137]\": -0.501422995518741, \"C(hour_of_week)[138]\": 18.270624396075306, \"C(hour_of_week)[139]\": 17.414458684125712, \"C(hour_of_week)[140]\": 17.27213356377706, \"C(hour_of_week)[141]\": 17.114853467043247, \"C(hour_of_week)[142]\": 16.77671715991275, \"C(hour_of_week)[143]\": 16.923653477932447, \"C(hour_of_week)[144]\": 16.326815993756632, \"C(hour_of_week)[145]\": 16.191744581853445, \"C(hour_of_week)[146]\": 16.24423960549348, \"C(hour_of_week)[147]\": 16.258538664060307, \"C(hour_of_week)[148]\": 16.256429067407865, \"C(hour_of_week)[149]\": 16.28708006163922, \"C(hour_of_week)[150]\": 15.861247441115868, \"C(hour_of_week)[151]\": 16.11186787694335, \"C(hour_of_week)[152]\": 16.424848588050054, \"C(hour_of_week)[153]\": 16.779465695717743, \"C(hour_of_week)[154]\": 16.883991709663725, \"C(hour_of_week)[155]\": 16.512503805152676, \"C(hour_of_week)[156]\": 16.520205329414228, \"C(hour_of_week)[157]\": 16.683993703898096, \"C(hour_of_week)[158]\": 16.958591661056033, \"C(hour_of_week)[159]\": 17.339246404469037, \"C(hour_of_week)[160]\": 17.52856290654105, \"C(hour_of_week)[161]\": 17.13510162457645, \"C(hour_of_week)[162]\": 16.651636375902026, \"C(hour_of_week)[163]\": 16.29417887903526, \"C(hour_of_week)[164]\": 16.42422981928218, \"C(hour_of_week)[165]\": 16.292387262759448, \"C(hour_of_week)[166]\": 16.277765043246944, \"C(hour_of_week)[167]\": 16.40872258714465, \"bin_0_occupied\": 0.07126384596382582, \"bin_1_occupied\": 0.11357546592288134, \"bin_2_occupied\": 0.14392311090663848, \"bin_0_unoccupied\": -0.26741671237889175, \"bin_1_unoccupied\": -0.030856837708973746, \"bin_2_unoccupied\": 0.04298660213586147, \"bin_3_unoccupied\": 0.15488918492927964}}, \"sep\": {\"segment_name\": \"aug-sep-oct-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 23.366228293275242, \"C(hour_of_week)[1]\": 23.126631268722353, \"C(hour_of_week)[2]\": 23.137403860760774, \"C(hour_of_week)[3]\": 23.165974714283937, \"C(hour_of_week)[4]\": 23.08227079699936, \"C(hour_of_week)[5]\": 23.138793799434218, \"C(hour_of_week)[6]\": 24.078518807637604, \"C(hour_of_week)[7]\": 19.997699082425566, \"C(hour_of_week)[8]\": 20.64825286007556, \"C(hour_of_week)[9]\": 21.4061968627101, \"C(hour_of_week)[10]\": 22.365204160978223, \"C(hour_of_week)[11]\": 22.496519535913652, \"C(hour_of_week)[12]\": 22.247956481034148, \"C(hour_of_week)[13]\": 22.26691004404814, \"C(hour_of_week)[14]\": 21.939908553645104, \"C(hour_of_week)[15]\": 21.559295953397573, \"C(hour_of_week)[16]\": 21.10824955244883, \"C(hour_of_week)[17]\": 19.792698792480092, \"C(hour_of_week)[18]\": 23.843299457775665, \"C(hour_of_week)[19]\": 23.671220781913753, \"C(hour_of_week)[20]\": 23.555826683662726, \"C(hour_of_week)[21]\": 23.76861539603402, \"C(hour_of_week)[22]\": 23.038193241361732, \"C(hour_of_week)[23]\": 22.543532454496148, \"C(hour_of_week)[24]\": 22.436125147460935, \"C(hour_of_week)[25]\": 22.563301236518075, \"C(hour_of_week)[26]\": 22.706756539937757, \"C(hour_of_week)[27]\": 22.64776529522092, \"C(hour_of_week)[28]\": 22.79576009298976, \"C(hour_of_week)[29]\": 22.798621601789474, \"C(hour_of_week)[30]\": 23.759531067092404, \"C(hour_of_week)[31]\": 19.578815150082495, \"C(hour_of_week)[32]\": 20.34328926168453, \"C(hour_of_week)[33]\": 21.671107476899984, \"C(hour_of_week)[34]\": 22.136276173830208, \"C(hour_of_week)[35]\": 22.25993172253686, \"C(hour_of_week)[36]\": 22.385479302045113, \"C(hour_of_week)[37]\": 22.04615817756068, \"C(hour_of_week)[38]\": 21.789008273992984, \"C(hour_of_week)[39]\": 21.81246323120298, \"C(hour_of_week)[40]\": 20.934330680923445, \"C(hour_of_week)[41]\": 19.649757291585132, \"C(hour_of_week)[42]\": 23.693368945227437, \"C(hour_of_week)[43]\": 23.26527656972177, \"C(hour_of_week)[44]\": 23.466816819879458, \"C(hour_of_week)[45]\": 23.738100976256458, \"C(hour_of_week)[46]\": 22.85380183331455, \"C(hour_of_week)[47]\": 22.57121324821127, \"C(hour_of_week)[48]\": 22.373550883513865, \"C(hour_of_week)[49]\": 22.525320088152892, \"C(hour_of_week)[50]\": 22.70354441009877, \"C(hour_of_week)[51]\": 22.584109663702822, \"C(hour_of_week)[52]\": 22.408504432061985, \"C(hour_of_week)[53]\": 22.50648692079134, \"C(hour_of_week)[54]\": 23.053580563451213, \"C(hour_of_week)[55]\": 19.395344070122395, \"C(hour_of_week)[56]\": 20.50767232650263, \"C(hour_of_week)[57]\": 21.390893809357898, \"C(hour_of_week)[58]\": 22.337513318057688, \"C(hour_of_week)[59]\": 22.671428110517375, \"C(hour_of_week)[60]\": 22.34735053037412, \"C(hour_of_week)[61]\": 22.324814190468274, \"C(hour_of_week)[62]\": 22.308778824903694, \"C(hour_of_week)[63]\": 21.845271306466003, \"C(hour_of_week)[64]\": 21.271974297464563, \"C(hour_of_week)[65]\": 20.037911259492038, \"C(hour_of_week)[66]\": 23.285417650528196, \"C(hour_of_week)[67]\": 23.117321998169334, \"C(hour_of_week)[68]\": 23.087453269906838, \"C(hour_of_week)[69]\": 22.932025007536932, \"C(hour_of_week)[70]\": 22.615011182257565, \"C(hour_of_week)[71]\": 22.180455464271454, \"C(hour_of_week)[72]\": 22.02084741700281, \"C(hour_of_week)[73]\": 22.09019331079449, \"C(hour_of_week)[74]\": 22.23804053508062, \"C(hour_of_week)[75]\": 22.283739054279398, \"C(hour_of_week)[76]\": 22.377485055619672, \"C(hour_of_week)[77]\": 22.397023911607242, \"C(hour_of_week)[78]\": 22.817982954753308, \"C(hour_of_week)[79]\": 19.445324011424148, \"C(hour_of_week)[80]\": 20.317719731703214, \"C(hour_of_week)[81]\": 21.70535522869095, \"C(hour_of_week)[82]\": 22.46693399681773, \"C(hour_of_week)[83]\": 22.453911300466473, \"C(hour_of_week)[84]\": 22.590473529516593, \"C(hour_of_week)[85]\": 22.411492574056282, \"C(hour_of_week)[86]\": 22.146187472929263, \"C(hour_of_week)[87]\": 22.257622772353802, \"C(hour_of_week)[88]\": 21.627142144122697, \"C(hour_of_week)[89]\": 20.131901100687365, \"C(hour_of_week)[90]\": 23.677701747336684, \"C(hour_of_week)[91]\": 23.104613038752127, \"C(hour_of_week)[92]\": 23.120972916159886, \"C(hour_of_week)[93]\": 23.625370792377968, \"C(hour_of_week)[94]\": 22.76497057117175, \"C(hour_of_week)[95]\": 22.475112534683138, \"C(hour_of_week)[96]\": 22.293204707702692, \"C(hour_of_week)[97]\": 22.344914760894284, \"C(hour_of_week)[98]\": 22.488163891056093, \"C(hour_of_week)[99]\": 22.48719593671759, \"C(hour_of_week)[100]\": 22.682255958049502, \"C(hour_of_week)[101]\": 22.73401065535415, \"C(hour_of_week)[102]\": 23.31244498627201, \"C(hour_of_week)[103]\": 19.457787304536307, \"C(hour_of_week)[104]\": 20.669129901510274, \"C(hour_of_week)[105]\": 21.43322750682772, \"C(hour_of_week)[106]\": 22.094811251150926, \"C(hour_of_week)[107]\": 22.39772577896682, \"C(hour_of_week)[108]\": 22.166268296316723, \"C(hour_of_week)[109]\": 22.004791052954296, \"C(hour_of_week)[110]\": 22.072956468214905, \"C(hour_of_week)[111]\": 21.59098856499101, \"C(hour_of_week)[112]\": 21.062013255429367, \"C(hour_of_week)[113]\": 20.04864166954059, \"C(hour_of_week)[114]\": 23.485980160154487, \"C(hour_of_week)[115]\": 23.658839837892096, \"C(hour_of_week)[116]\": 23.644627063495122, \"C(hour_of_week)[117]\": 23.671555812076583, \"C(hour_of_week)[118]\": 23.38352595492042, \"C(hour_of_week)[119]\": 22.79801169539237, \"C(hour_of_week)[120]\": 23.067655871312628, \"C(hour_of_week)[121]\": 23.257319960573646, \"C(hour_of_week)[122]\": 23.409957880552685, \"C(hour_of_week)[123]\": 23.418717573128202, \"C(hour_of_week)[124]\": 23.468376166715775, \"C(hour_of_week)[125]\": 23.374299447062583, \"C(hour_of_week)[126]\": 23.32991905647293, \"C(hour_of_week)[127]\": 22.49463812058477, \"C(hour_of_week)[128]\": 22.069501344754325, \"C(hour_of_week)[129]\": 21.890855378259552, \"C(hour_of_week)[130]\": 21.882696903399193, \"C(hour_of_week)[131]\": 21.859353675824575, \"C(hour_of_week)[132]\": 23.579002408433993, \"C(hour_of_week)[133]\": 24.91633840318825, \"C(hour_of_week)[134]\": 24.812682658938254, \"C(hour_of_week)[135]\": 24.809965239246104, \"C(hour_of_week)[136]\": 24.513850810240825, \"C(hour_of_week)[137]\": 24.52143349997702, \"C(hour_of_week)[138]\": 23.512910470980273, \"C(hour_of_week)[139]\": 23.231248042317077, \"C(hour_of_week)[140]\": 23.167370809322215, \"C(hour_of_week)[141]\": 23.150189758729876, \"C(hour_of_week)[142]\": 22.983914062386347, \"C(hour_of_week)[143]\": 23.263984469654766, \"C(hour_of_week)[144]\": 22.91983866137546, \"C(hour_of_week)[145]\": 22.892051824183554, \"C(hour_of_week)[146]\": 23.050354369105634, \"C(hour_of_week)[147]\": 23.198817774144405, \"C(hour_of_week)[148]\": 23.091426238698755, \"C(hour_of_week)[149]\": 23.373358274777175, \"C(hour_of_week)[150]\": 23.325183390544865, \"C(hour_of_week)[151]\": 23.12427393655535, \"C(hour_of_week)[152]\": 23.006103549808486, \"C(hour_of_week)[153]\": 22.97247763537626, \"C(hour_of_week)[154]\": 22.98033032109599, \"C(hour_of_week)[155]\": 22.4778608414167, \"C(hour_of_week)[156]\": 22.583536893177044, \"C(hour_of_week)[157]\": 22.77022032363438, \"C(hour_of_week)[158]\": 22.95215902263051, \"C(hour_of_week)[159]\": 23.238031595501177, \"C(hour_of_week)[160]\": 23.382534276603256, \"C(hour_of_week)[161]\": 23.054326830274718, \"C(hour_of_week)[162]\": 22.851021002648523, \"C(hour_of_week)[163]\": 22.995882154598554, \"C(hour_of_week)[164]\": 23.1360361473538, \"C(hour_of_week)[165]\": 23.012436403964674, \"C(hour_of_week)[166]\": 23.166174086057786, \"C(hour_of_week)[167]\": 23.304412856322813, \"bin_0_occupied\": -0.3592009167601687, \"bin_1_occupied\": 0.015247893689167305, \"bin_2_occupied\": 0.09272818647205205, \"bin_3_occupied\": 0.1043730755194128, \"bin_4_occupied\": 0.17052095527619673, \"bin_0_unoccupied\": -0.3896117828283877, \"bin_1_unoccupied\": -0.30657109823613604, \"bin_2_unoccupied\": -0.06977012927096878, \"bin_3_unoccupied\": 0.06573835148384805, \"bin_4_unoccupied\": 0.15641711821939408}}, \"oct\": {\"segment_name\": \"sep-oct-nov-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_4_occupied + bin_5_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 30.27655536591546, \"C(hour_of_week)[1]\": 30.03114746932107, \"C(hour_of_week)[2]\": 30.070224938555917, \"C(hour_of_week)[3]\": 18.013508017689915, \"C(hour_of_week)[4]\": 29.719558659615082, \"C(hour_of_week)[5]\": 18.17224589346977, \"C(hour_of_week)[6]\": 18.2195703249951, \"C(hour_of_week)[7]\": 17.202488530767376, \"C(hour_of_week)[8]\": 16.5955873635177, \"C(hour_of_week)[9]\": 16.686275063106656, \"C(hour_of_week)[10]\": 17.249109355395756, \"C(hour_of_week)[11]\": 17.394977053076268, \"C(hour_of_week)[12]\": 17.286415968664176, \"C(hour_of_week)[13]\": 17.2266315061564, \"C(hour_of_week)[14]\": 16.999919673822223, \"C(hour_of_week)[15]\": 16.65317481812397, \"C(hour_of_week)[16]\": 29.588993122020852, \"C(hour_of_week)[17]\": 28.843400457823133, \"C(hour_of_week)[18]\": 27.880979065065908, \"C(hour_of_week)[19]\": 28.416283714819205, \"C(hour_of_week)[20]\": 28.58434451026054, \"C(hour_of_week)[21]\": 28.684047592985518, \"C(hour_of_week)[22]\": 28.644384242052233, \"C(hour_of_week)[23]\": 28.313349245476946, \"C(hour_of_week)[24]\": 28.435373783284202, \"C(hour_of_week)[25]\": 28.513939145893833, \"C(hour_of_week)[26]\": 28.81639852444017, \"C(hour_of_week)[27]\": 28.825740652131365, \"C(hour_of_week)[28]\": 29.02116571110834, \"C(hour_of_week)[29]\": 29.05734887425925, \"C(hour_of_week)[30]\": 29.33734373267347, \"C(hour_of_week)[31]\": 28.81328001893316, \"C(hour_of_week)[32]\": 28.67415645460644, \"C(hour_of_week)[33]\": 16.774359128174748, \"C(hour_of_week)[34]\": 17.217244164784464, \"C(hour_of_week)[35]\": 17.267745934847316, \"C(hour_of_week)[36]\": 17.369064205040253, \"C(hour_of_week)[37]\": 17.208596335319232, \"C(hour_of_week)[38]\": 16.914263639909066, \"C(hour_of_week)[39]\": 16.718677976730323, \"C(hour_of_week)[40]\": 16.56863984786057, \"C(hour_of_week)[41]\": 28.691015596396145, \"C(hour_of_week)[42]\": 27.6964157759401, \"C(hour_of_week)[43]\": 28.09177937460716, \"C(hour_of_week)[44]\": 28.23332435851736, \"C(hour_of_week)[45]\": 28.51037887444903, \"C(hour_of_week)[46]\": 28.192625141874014, \"C(hour_of_week)[47]\": 27.806730758975487, \"C(hour_of_week)[48]\": 27.76574453718197, \"C(hour_of_week)[49]\": 28.08822390459864, \"C(hour_of_week)[50]\": 28.248134392489053, \"C(hour_of_week)[51]\": 28.298167754357223, \"C(hour_of_week)[52]\": 28.333259929904287, \"C(hour_of_week)[53]\": 28.54490347183069, \"C(hour_of_week)[54]\": 28.68555420298292, \"C(hour_of_week)[55]\": 28.245642338027803, \"C(hour_of_week)[56]\": 28.256144861565385, \"C(hour_of_week)[57]\": 29.094972054297397, \"C(hour_of_week)[58]\": 29.681588187748133, \"C(hour_of_week)[59]\": 17.096686983232782, \"C(hour_of_week)[60]\": 17.20069174576422, \"C(hour_of_week)[61]\": 17.065520749278306, \"C(hour_of_week)[62]\": 16.91369536938222, \"C(hour_of_week)[63]\": 16.848944516674283, \"C(hour_of_week)[64]\": 29.228962844611072, \"C(hour_of_week)[65]\": 28.374592225348962, \"C(hour_of_week)[66]\": 27.1944216304326, \"C(hour_of_week)[67]\": 27.478666441618742, \"C(hour_of_week)[68]\": 27.870418637309083, \"C(hour_of_week)[69]\": 27.74366871749255, \"C(hour_of_week)[70]\": 27.755174188925196, \"C(hour_of_week)[71]\": 27.52426559548981, \"C(hour_of_week)[72]\": 27.657891570347285, \"C(hour_of_week)[73]\": 27.845126341180297, \"C(hour_of_week)[74]\": 28.180205086128993, \"C(hour_of_week)[75]\": 28.219615303617996, \"C(hour_of_week)[76]\": 28.413623419085557, \"C(hour_of_week)[77]\": 28.523721472638364, \"C(hour_of_week)[78]\": 28.563663066835815, \"C(hour_of_week)[79]\": 27.765365020032966, \"C(hour_of_week)[80]\": 28.349612438542366, \"C(hour_of_week)[81]\": 29.385053168189362, \"C(hour_of_week)[82]\": 29.961246184545562, \"C(hour_of_week)[83]\": 30.130836685899624, \"C(hour_of_week)[84]\": 17.62424539121027, \"C(hour_of_week)[85]\": 17.447889491576607, \"C(hour_of_week)[86]\": 17.32779975363923, \"C(hour_of_week)[87]\": 17.241891424389607, \"C(hour_of_week)[88]\": 29.61747224880832, \"C(hour_of_week)[89]\": 28.75788160267457, \"C(hour_of_week)[90]\": 28.168973818595447, \"C(hour_of_week)[91]\": 28.68634439289838, \"C(hour_of_week)[92]\": 28.739875510482506, \"C(hour_of_week)[93]\": 29.32012657867501, \"C(hour_of_week)[94]\": 28.98407446183369, \"C(hour_of_week)[95]\": 28.83332230742362, \"C(hour_of_week)[96]\": 28.80162125138472, \"C(hour_of_week)[97]\": 28.703192017233302, \"C(hour_of_week)[98]\": 29.05036481878761, \"C(hour_of_week)[99]\": 28.94193564775637, \"C(hour_of_week)[100]\": 29.061663551904847, \"C(hour_of_week)[101]\": 29.198365928242637, \"C(hour_of_week)[102]\": 29.475954756150553, \"C(hour_of_week)[103]\": 28.62435007820763, \"C(hour_of_week)[104]\": 28.856768028142188, \"C(hour_of_week)[105]\": 16.84114452231205, \"C(hour_of_week)[106]\": 29.652497929918685, \"C(hour_of_week)[107]\": 29.660743985225377, \"C(hour_of_week)[108]\": 17.015124309849213, \"C(hour_of_week)[109]\": 29.425955585154625, \"C(hour_of_week)[110]\": 29.2979274861928, \"C(hour_of_week)[111]\": 28.986901778435904, \"C(hour_of_week)[112]\": 28.69931897764228, \"C(hour_of_week)[113]\": 28.317752754659228, \"C(hour_of_week)[114]\": 28.578287651597606, \"C(hour_of_week)[115]\": 29.35308630597045, \"C(hour_of_week)[116]\": 29.278985541109158, \"C(hour_of_week)[117]\": 29.570633238241516, \"C(hour_of_week)[118]\": 29.378746486105914, \"C(hour_of_week)[119]\": 29.043916300555836, \"C(hour_of_week)[120]\": 29.089565072331908, \"C(hour_of_week)[121]\": 29.19842939621716, \"C(hour_of_week)[122]\": 29.370787671995405, \"C(hour_of_week)[123]\": 29.390076457206977, \"C(hour_of_week)[124]\": 29.409357344317634, \"C(hour_of_week)[125]\": 29.387390216798305, \"C(hour_of_week)[126]\": 29.78858391058893, \"C(hour_of_week)[127]\": 28.701909050802886, \"C(hour_of_week)[128]\": 27.848367581018696, \"C(hour_of_week)[129]\": 27.215599880456303, \"C(hour_of_week)[130]\": 26.869053563354772, \"C(hour_of_week)[131]\": 26.5816237139736, \"C(hour_of_week)[132]\": 27.336512199186238, \"C(hour_of_week)[133]\": 27.75639384005387, \"C(hour_of_week)[134]\": 27.611365361544706, \"C(hour_of_week)[135]\": 27.634559939237306, \"C(hour_of_week)[136]\": 27.623253145124878, \"C(hour_of_week)[137]\": 27.792912161025374, \"C(hour_of_week)[138]\": 28.279168128312477, \"C(hour_of_week)[139]\": 28.82991709294854, \"C(hour_of_week)[140]\": 28.780850180801107, \"C(hour_of_week)[141]\": 28.927924317111117, \"C(hour_of_week)[142]\": 28.891599078362983, \"C(hour_of_week)[143]\": 29.299036924743625, \"C(hour_of_week)[144]\": 29.173137679248235, \"C(hour_of_week)[145]\": 29.367837384619037, \"C(hour_of_week)[146]\": 29.46570455082897, \"C(hour_of_week)[147]\": 18.04904238713644, \"C(hour_of_week)[148]\": 29.55662046136834, \"C(hour_of_week)[149]\": 29.877935735565107, \"C(hour_of_week)[150]\": 18.488922116633127, \"C(hour_of_week)[151]\": 17.799438428178348, \"C(hour_of_week)[152]\": 29.035601680404454, \"C(hour_of_week)[153]\": 28.562183729581506, \"C(hour_of_week)[154]\": 28.175658966221523, \"C(hour_of_week)[155]\": 27.725853387860056, \"C(hour_of_week)[156]\": 27.713950130290225, \"C(hour_of_week)[157]\": 27.75660665907822, \"C(hour_of_week)[158]\": 27.876736367707498, \"C(hour_of_week)[159]\": 27.93842248465982, \"C(hour_of_week)[160]\": 28.295917857649506, \"C(hour_of_week)[161]\": 28.42222397873188, \"C(hour_of_week)[162]\": 28.657549741357236, \"C(hour_of_week)[163]\": 29.204111439074815, \"C(hour_of_week)[164]\": 29.399040338061916, \"C(hour_of_week)[165]\": 29.366573385300626, \"C(hour_of_week)[166]\": 29.837953804415797, \"C(hour_of_week)[167]\": 29.967505146319947, \"bin_0_occupied\": -0.2381895696019812, \"bin_1_occupied\": -0.23838868703383748, \"bin_2_occupied\": -0.10001059504365054, \"bin_3_occupied\": -0.014031048613680153, \"bin_4_occupied\": 0.24976180351099614, \"bin_5_occupied\": 0.15885793312588484, \"bin_0_unoccupied\": -0.5246241633383874, \"bin_1_unoccupied\": -0.4801754794862052, \"bin_2_unoccupied\": -0.24284495015951646, \"bin_3_unoccupied\": -0.08148064561400563, \"bin_4_unoccupied\": 0.13700209114734171, \"bin_5_unoccupied\": 0.13877956899084032}}, \"nov\": {\"segment_name\": \"oct-nov-dec-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_3_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied + bin_3_unoccupied + bin_4_unoccupied + bin_5_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 28.711758124766707, \"C(hour_of_week)[1]\": 28.543747182696446, \"C(hour_of_week)[2]\": 28.46600046780161, \"C(hour_of_week)[3]\": 28.011829409448303, \"C(hour_of_week)[4]\": 27.924853191459892, \"C(hour_of_week)[5]\": 28.147669206784137, \"C(hour_of_week)[6]\": 27.50619703368132, \"C(hour_of_week)[7]\": 21.97994191538849, \"C(hour_of_week)[8]\": 20.35600634017883, \"C(hour_of_week)[9]\": 19.539863947055753, \"C(hour_of_week)[10]\": 19.732536239702, \"C(hour_of_week)[11]\": 19.84605667720476, \"C(hour_of_week)[12]\": 19.868371614170368, \"C(hour_of_week)[13]\": 19.71864511933228, \"C(hour_of_week)[14]\": 19.582735391963908, \"C(hour_of_week)[15]\": 19.13061578991631, \"C(hour_of_week)[16]\": 19.66878653736818, \"C(hour_of_week)[17]\": 20.365461409173086, \"C(hour_of_week)[18]\": 21.83477701486796, \"C(hour_of_week)[19]\": 22.611114071815493, \"C(hour_of_week)[20]\": 22.984280520056362, \"C(hour_of_week)[21]\": 22.901056855225104, \"C(hour_of_week)[22]\": 23.335528561956405, \"C(hour_of_week)[23]\": 23.19910690719484, \"C(hour_of_week)[24]\": 23.50970523381244, \"C(hour_of_week)[25]\": 23.53627605313309, \"C(hour_of_week)[26]\": 23.691844336806557, \"C(hour_of_week)[27]\": 27.48942141866559, \"C(hour_of_week)[28]\": 23.94440679899692, \"C(hour_of_week)[29]\": 27.74531030712705, \"C(hour_of_week)[30]\": 27.48182521796086, \"C(hour_of_week)[31]\": 21.738270455562112, \"C(hour_of_week)[32]\": 20.215803818488066, \"C(hour_of_week)[33]\": 19.694389713135184, \"C(hour_of_week)[34]\": 19.81967047848799, \"C(hour_of_week)[35]\": 19.822284453914744, \"C(hour_of_week)[36]\": 19.776278027690644, \"C(hour_of_week)[37]\": 19.694825079056933, \"C(hour_of_week)[38]\": 19.408221732252468, \"C(hour_of_week)[39]\": 18.92051477083984, \"C(hour_of_week)[40]\": 19.517269179176107, \"C(hour_of_week)[41]\": 20.395990522386548, \"C(hour_of_week)[42]\": 21.784374240647264, \"C(hour_of_week)[43]\": 22.55727541887872, \"C(hour_of_week)[44]\": 22.873695445909725, \"C(hour_of_week)[45]\": 22.929428932684026, \"C(hour_of_week)[46]\": 23.199360272641226, \"C(hour_of_week)[47]\": 22.858799583581373, \"C(hour_of_week)[48]\": 26.60780687190149, \"C(hour_of_week)[49]\": 26.879366544137454, \"C(hour_of_week)[50]\": 27.05915720837217, \"C(hour_of_week)[51]\": 27.265655417419843, \"C(hour_of_week)[52]\": 27.3947153147074, \"C(hour_of_week)[53]\": 27.498859729420516, \"C(hour_of_week)[54]\": 27.073208347633283, \"C(hour_of_week)[55]\": 21.638403815991992, \"C(hour_of_week)[56]\": 20.163678253942116, \"C(hour_of_week)[57]\": 20.032556719062278, \"C(hour_of_week)[58]\": 19.809305084996367, \"C(hour_of_week)[59]\": 19.683714459680132, \"C(hour_of_week)[60]\": 19.67198156946672, \"C(hour_of_week)[61]\": 19.365226164464737, \"C(hour_of_week)[62]\": 19.04211196454203, \"C(hour_of_week)[63]\": 19.190344224819395, \"C(hour_of_week)[64]\": 19.7875150090469, \"C(hour_of_week)[65]\": 20.3950968508413, \"C(hour_of_week)[66]\": 21.853665459727868, \"C(hour_of_week)[67]\": 22.49243035452334, \"C(hour_of_week)[68]\": 22.828849503428135, \"C(hour_of_week)[69]\": 22.740067984439328, \"C(hour_of_week)[70]\": 23.27491452645309, \"C(hour_of_week)[71]\": 22.92127300189337, \"C(hour_of_week)[72]\": 22.90294205498723, \"C(hour_of_week)[73]\": 23.219510149328087, \"C(hour_of_week)[74]\": 23.401476519298214, \"C(hour_of_week)[75]\": 23.337547169743164, \"C(hour_of_week)[76]\": 27.070296174790624, \"C(hour_of_week)[77]\": 27.26201004625137, \"C(hour_of_week)[78]\": 27.105407618012393, \"C(hour_of_week)[79]\": 21.49169658942448, \"C(hour_of_week)[80]\": 20.688419374088703, \"C(hour_of_week)[81]\": 20.755034948243786, \"C(hour_of_week)[82]\": 20.60683923935888, \"C(hour_of_week)[83]\": 20.51724533070212, \"C(hour_of_week)[84]\": 20.515155042197303, \"C(hour_of_week)[85]\": 20.103637248009225, \"C(hour_of_week)[86]\": 19.973472789160326, \"C(hour_of_week)[87]\": 19.775099397662352, \"C(hour_of_week)[88]\": 20.311631994164735, \"C(hour_of_week)[89]\": 21.00014449782292, \"C(hour_of_week)[90]\": 22.84918281719662, \"C(hour_of_week)[91]\": 27.4229437027431, \"C(hour_of_week)[92]\": 27.479429255615763, \"C(hour_of_week)[93]\": 27.875679283651827, \"C(hour_of_week)[94]\": 27.838312417991638, \"C(hour_of_week)[95]\": 27.65335267988237, \"C(hour_of_week)[96]\": 27.62390677452466, \"C(hour_of_week)[97]\": 27.460122865434585, \"C(hour_of_week)[98]\": 27.81401610991429, \"C(hour_of_week)[99]\": 27.695077903272878, \"C(hour_of_week)[100]\": 27.697185757914042, \"C(hour_of_week)[101]\": 27.917711146550275, \"C(hour_of_week)[102]\": 27.532015564383965, \"C(hour_of_week)[103]\": 21.574590291881414, \"C(hour_of_week)[104]\": 20.43011925498432, \"C(hour_of_week)[105]\": 20.398545637271425, \"C(hour_of_week)[106]\": 20.396046454456954, \"C(hour_of_week)[107]\": 20.236635867001844, \"C(hour_of_week)[108]\": 20.060713325700846, \"C(hour_of_week)[109]\": 19.898317822136008, \"C(hour_of_week)[110]\": 19.68901192972416, \"C(hour_of_week)[111]\": 19.13389181520952, \"C(hour_of_week)[112]\": 19.649912689807095, \"C(hour_of_week)[113]\": 20.450742652160443, \"C(hour_of_week)[114]\": 22.482725042970507, \"C(hour_of_week)[115]\": 27.136968896690252, \"C(hour_of_week)[116]\": 27.224259631477786, \"C(hour_of_week)[117]\": 27.488775870197646, \"C(hour_of_week)[118]\": 27.447035417080137, \"C(hour_of_week)[119]\": 27.26581744699056, \"C(hour_of_week)[120]\": 27.55041238927323, \"C(hour_of_week)[121]\": 27.601136725954227, \"C(hour_of_week)[122]\": 27.770025625731176, \"C(hour_of_week)[123]\": 27.746262171024128, \"C(hour_of_week)[124]\": 27.775673980584884, \"C(hour_of_week)[125]\": 27.764390711034117, \"C(hour_of_week)[126]\": 28.36438574592641, \"C(hour_of_week)[127]\": 27.147849702518698, \"C(hour_of_week)[128]\": 22.505866212956757, \"C(hour_of_week)[129]\": 21.140897452220003, \"C(hour_of_week)[130]\": 20.212813663047548, \"C(hour_of_week)[131]\": 19.803255686941583, \"C(hour_of_week)[132]\": 19.70892147617806, \"C(hour_of_week)[133]\": 18.564094990065296, \"C(hour_of_week)[134]\": 18.37956481223923, \"C(hour_of_week)[135]\": 18.641870890631402, \"C(hour_of_week)[136]\": 19.345146958929508, \"C(hour_of_week)[137]\": 20.272256560409037, \"C(hour_of_week)[138]\": 22.167351126194585, \"C(hour_of_week)[139]\": 26.91977962567692, \"C(hour_of_week)[140]\": 23.28751993771277, \"C(hour_of_week)[141]\": 27.159861286410653, \"C(hour_of_week)[142]\": 23.77051199193381, \"C(hour_of_week)[143]\": 27.981430717282695, \"C(hour_of_week)[144]\": 27.736689076274537, \"C(hour_of_week)[145]\": 27.8156148967581, \"C(hour_of_week)[146]\": 27.853742724016445, \"C(hour_of_week)[147]\": 28.06406086704453, \"C(hour_of_week)[148]\": 28.117208972141047, \"C(hour_of_week)[149]\": 28.38021480503514, \"C(hour_of_week)[150]\": 28.414240855227042, \"C(hour_of_week)[151]\": 28.011683734458025, \"C(hour_of_week)[152]\": 27.02827364805027, \"C(hour_of_week)[153]\": 22.1131538707186, \"C(hour_of_week)[154]\": 21.122288039582962, \"C(hour_of_week)[155]\": 20.469431442899538, \"C(hour_of_week)[156]\": 20.056073818276772, \"C(hour_of_week)[157]\": 19.66258801493068, \"C(hour_of_week)[158]\": 19.685490561903904, \"C(hour_of_week)[159]\": 19.837166232963945, \"C(hour_of_week)[160]\": 21.21940315782187, \"C(hour_of_week)[161]\": 22.327445128865026, \"C(hour_of_week)[162]\": 22.894332562020114, \"C(hour_of_week)[163]\": 23.430844650897, \"C(hour_of_week)[164]\": 27.746343175858478, \"C(hour_of_week)[165]\": 27.941731739356324, \"C(hour_of_week)[166]\": 28.384048196985592, \"C(hour_of_week)[167]\": 28.564249380533447, \"bin_0_occupied\": -0.4054337293818377, \"bin_1_occupied\": -0.49667653919236643, \"bin_2_occupied\": -0.35049457752621377, \"bin_3_occupied\": -0.18622454544907033, \"bin_0_unoccupied\": -0.3138042860374506, \"bin_1_unoccupied\": -0.36298571227470716, \"bin_2_unoccupied\": -0.11349222561511674, \"bin_3_unoccupied\": -0.1742483194082457, \"bin_4_unoccupied\": 0.09967681123790502, \"bin_5_unoccupied\": 0.44943678761599093}}, \"dec\": {\"segment_name\": \"nov-dec-jan-weighted\", \"formula\": \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied + bin_2_occupied + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied\", \"warnings\": [], \"model_params\": {\"C(hour_of_week)[0]\": 31.98895156419638, \"C(hour_of_week)[1]\": 32.177718390904325, \"C(hour_of_week)[2]\": 32.13650550124076, \"C(hour_of_week)[3]\": 31.972267771709618, \"C(hour_of_week)[4]\": 31.87244908586085, \"C(hour_of_week)[5]\": 32.072619277460625, \"C(hour_of_week)[6]\": 31.13663143880229, \"C(hour_of_week)[7]\": 24.0173751665062, \"C(hour_of_week)[8]\": 21.906662080924583, \"C(hour_of_week)[9]\": 20.312815159181294, \"C(hour_of_week)[10]\": 20.197503732511386, \"C(hour_of_week)[11]\": 19.86108787779591, \"C(hour_of_week)[12]\": 19.691211986807268, \"C(hour_of_week)[13]\": 19.313576948745368, \"C(hour_of_week)[14]\": 19.415385265358886, \"C(hour_of_week)[15]\": 19.10945981062134, \"C(hour_of_week)[16]\": 19.879018358551324, \"C(hour_of_week)[17]\": 21.574035723326787, \"C(hour_of_week)[18]\": 29.67577783070654, \"C(hour_of_week)[19]\": 30.194334501643784, \"C(hour_of_week)[20]\": 30.691152391161104, \"C(hour_of_week)[21]\": 30.550405911005885, \"C(hour_of_week)[22]\": 30.55935385728337, \"C(hour_of_week)[23]\": 30.48088742255164, \"C(hour_of_week)[24]\": 30.946357164168752, \"C(hour_of_week)[25]\": 30.830990713030708, \"C(hour_of_week)[26]\": 30.809610265687194, \"C(hour_of_week)[27]\": 31.163595380035012, \"C(hour_of_week)[28]\": 31.186084518954342, \"C(hour_of_week)[29]\": 31.37117440755988, \"C(hour_of_week)[30]\": 30.75043606830743, \"C(hour_of_week)[31]\": 23.791485688430672, \"C(hour_of_week)[32]\": 21.51502245381142, \"C(hour_of_week)[33]\": 20.18767777946643, \"C(hour_of_week)[34]\": 19.993754743267548, \"C(hour_of_week)[35]\": 19.943524469151942, \"C(hour_of_week)[36]\": 19.767670332463076, \"C(hour_of_week)[37]\": 19.492542795312353, \"C(hour_of_week)[38]\": 19.222997795186174, \"C(hour_of_week)[39]\": 18.827245932600206, \"C(hour_of_week)[40]\": 19.77963941817016, \"C(hour_of_week)[41]\": 22.215420595175914, \"C(hour_of_week)[42]\": 25.33428561663138, \"C(hour_of_week)[43]\": 30.684160826684998, \"C(hour_of_week)[44]\": 31.521461786923695, \"C(hour_of_week)[45]\": 31.654396331038296, \"C(hour_of_week)[46]\": 31.955295151456657, \"C(hour_of_week)[47]\": 31.67434889930864, \"C(hour_of_week)[48]\": 31.429445532855063, \"C(hour_of_week)[49]\": 31.381890931592917, \"C(hour_of_week)[50]\": 31.582692074929547, \"C(hour_of_week)[51]\": 32.01983279064792, \"C(hour_of_week)[52]\": 31.9800347390195, \"C(hour_of_week)[53]\": 31.89823568553781, \"C(hour_of_week)[54]\": 31.3644344789662, \"C(hour_of_week)[55]\": 24.021529464439787, \"C(hour_of_week)[56]\": 22.192138717807254, \"C(hour_of_week)[57]\": 21.291898631928785, \"C(hour_of_week)[58]\": 20.60087262244678, \"C(hour_of_week)[59]\": 20.419218632522405, \"C(hour_of_week)[60]\": 20.157527139235384, \"C(hour_of_week)[61]\": 19.50034265183374, \"C(hour_of_week)[62]\": 19.31033546799707, \"C(hour_of_week)[63]\": 19.508913507524273, \"C(hour_of_week)[64]\": 21.05861051539218, \"C(hour_of_week)[65]\": 22.9647490695246, \"C(hour_of_week)[66]\": 31.316166247649573, \"C(hour_of_week)[67]\": 31.812781326074116, \"C(hour_of_week)[68]\": 31.987593219613018, \"C(hour_of_week)[69]\": 31.79329340105898, \"C(hour_of_week)[70]\": 32.53687874757611, \"C(hour_of_week)[71]\": 32.11744926642573, \"C(hour_of_week)[72]\": 31.62735126696792, \"C(hour_of_week)[73]\": 31.801792944564497, \"C(hour_of_week)[74]\": 31.61809007959986, \"C(hour_of_week)[75]\": 31.5776290767317, \"C(hour_of_week)[76]\": 31.449158818117574, \"C(hour_of_week)[77]\": 31.6499965864865, \"C(hour_of_week)[78]\": 31.404268664799837, \"C(hour_of_week)[79]\": 23.8826997466251, \"C(hour_of_week)[80]\": 22.20025175292171, \"C(hour_of_week)[81]\": 21.529494913805937, \"C(hour_of_week)[82]\": 20.920631415677228, \"C(hour_of_week)[83]\": 20.86370404763257, \"C(hour_of_week)[84]\": 20.858969510416287, \"C(hour_of_week)[85]\": 20.215418652600107, \"C(hour_of_week)[86]\": 19.895104029458835, \"C(hour_of_week)[87]\": 19.367329386574454, \"C(hour_of_week)[88]\": 20.479148131878897, \"C(hour_of_week)[89]\": 21.998138977954515, \"C(hour_of_week)[90]\": 30.35598094555071, \"C(hour_of_week)[91]\": 30.88531024059744, \"C(hour_of_week)[92]\": 31.120715223258415, \"C(hour_of_week)[93]\": 31.320675963350794, \"C(hour_of_week)[94]\": 31.44124326371025, \"C(hour_of_week)[95]\": 31.002164316261904, \"C(hour_of_week)[96]\": 31.159272121237514, \"C(hour_of_week)[97]\": 31.376372872870935, \"C(hour_of_week)[98]\": 31.733431732116035, \"C(hour_of_week)[99]\": 31.738754555126842, \"C(hour_of_week)[100]\": 31.971499977998924, \"C(hour_of_week)[101]\": 32.35598835623367, \"C(hour_of_week)[102]\": 31.53264445230679, \"C(hour_of_week)[103]\": 23.923007451110177, \"C(hour_of_week)[104]\": 22.018717272660435, \"C(hour_of_week)[105]\": 21.53234906007758, \"C(hour_of_week)[106]\": 21.399276212032262, \"C(hour_of_week)[107]\": 21.359956111472723, \"C(hour_of_week)[108]\": 20.98771271292019, \"C(hour_of_week)[109]\": 20.733733104920457, \"C(hour_of_week)[110]\": 20.393872098375663, \"C(hour_of_week)[111]\": 19.68460540453138, \"C(hour_of_week)[112]\": 20.872074486016746, \"C(hour_of_week)[113]\": 22.55507802970553, \"C(hour_of_week)[114]\": 30.47942507432029, \"C(hour_of_week)[115]\": 31.176134560328833, \"C(hour_of_week)[116]\": 31.56560749989059, \"C(hour_of_week)[117]\": 31.216751101795136, \"C(hour_of_week)[118]\": 31.369058223861504, \"C(hour_of_week)[119]\": 31.381759687116293, \"C(hour_of_week)[120]\": 31.605880204931914, \"C(hour_of_week)[121]\": 31.63841885317086, \"C(hour_of_week)[122]\": 31.848061307062533, \"C(hour_of_week)[123]\": 31.75055262295338, \"C(hour_of_week)[124]\": 31.68288193895297, \"C(hour_of_week)[125]\": 31.401826594676454, \"C(hour_of_week)[126]\": 31.980824951175777, \"C(hour_of_week)[127]\": 31.065046627173786, \"C(hour_of_week)[128]\": 30.18669344712927, \"C(hour_of_week)[129]\": 23.30491185109104, \"C(hour_of_week)[130]\": 21.982363096925276, \"C(hour_of_week)[131]\": 21.377087255865863, \"C(hour_of_week)[132]\": 21.0937158575841, \"C(hour_of_week)[133]\": 18.927393416227037, \"C(hour_of_week)[134]\": 18.45406871126409, \"C(hour_of_week)[135]\": 18.905333610864087, \"C(hour_of_week)[136]\": 19.965486757483877, \"C(hour_of_week)[137]\": 21.53048885355686, \"C(hour_of_week)[138]\": 24.033842495318847, \"C(hour_of_week)[139]\": 25.325060291572, \"C(hour_of_week)[140]\": 30.580505579163212, \"C(hour_of_week)[141]\": 30.628318948201525, \"C(hour_of_week)[142]\": 30.91143573147886, \"C(hour_of_week)[143]\": 31.630568248300623, \"C(hour_of_week)[144]\": 31.310654966802474, \"C(hour_of_week)[145]\": 31.197222682556298, \"C(hour_of_week)[146]\": 31.323665824442518, \"C(hour_of_week)[147]\": 31.444790781459737, \"C(hour_of_week)[148]\": 31.66163487642701, \"C(hour_of_week)[149]\": 32.04025959822834, \"C(hour_of_week)[150]\": 31.835273298465854, \"C(hour_of_week)[151]\": 31.723241959450164, \"C(hour_of_week)[152]\": 30.695900219021834, \"C(hour_of_week)[153]\": 23.707660768793176, \"C(hour_of_week)[154]\": 22.37528352061392, \"C(hour_of_week)[155]\": 21.661029652499398, \"C(hour_of_week)[156]\": 21.111808171535273, \"C(hour_of_week)[157]\": 20.414434336519477, \"C(hour_of_week)[158]\": 20.51293423997956, \"C(hour_of_week)[159]\": 20.968217984122212, \"C(hour_of_week)[160]\": 23.155601971005225, \"C(hour_of_week)[161]\": 30.271929584170444, \"C(hour_of_week)[162]\": 30.72315239117088, \"C(hour_of_week)[163]\": 31.033755412575744, \"C(hour_of_week)[164]\": 31.536330175519105, \"C(hour_of_week)[165]\": 31.867192158805633, \"C(hour_of_week)[166]\": 31.94706953117463, \"C(hour_of_week)[167]\": 32.10651980575815, \"bin_0_occupied\": -0.5377872114998198, \"bin_1_occupied\": -0.35838551642613065, \"bin_2_occupied\": -0.09763135622759472, \"bin_0_unoccupied\": -0.35895440770368875, \"bin_1_unoccupied\": -0.2545363019920993, \"bin_2_unoccupied\": -0.041317096752884204}}}, \"prediction_segment_type\": \"one_month\", \"prediction_segment_name_mapping\": {\"jan\": \"dec-jan-feb-weighted\", \"feb\": \"jan-feb-mar-weighted\", \"mar\": \"feb-mar-apr-weighted\", \"apr\": \"mar-apr-may-weighted\", \"may\": \"apr-may-jun-weighted\", \"jun\": \"may-jun-jul-weighted\", \"jul\": \"jun-jul-aug-weighted\", \"aug\": \"jul-aug-sep-weighted\", \"sep\": \"aug-sep-oct-weighted\", \"oct\": \"sep-oct-nov-weighted\", \"nov\": \"oct-nov-dec-weighted\", \"dec\": \"nov-dec-jan-weighted\"}, \"prediction_feature_processor\": \"caltrack_hourly_prediction_feature_processor\", \"occupancy_lookup\": \"{\\\"columns\\\":[\\\"dec-jan-feb-weighted\\\",\\\"jan-feb-mar-weighted\\\",\\\"feb-mar-apr-weighted\\\",\\\"mar-apr-may-weighted\\\",\\\"apr-may-jun-weighted\\\",\\\"may-jun-jul-weighted\\\",\\\"jun-jul-aug-weighted\\\",\\\"jul-aug-sep-weighted\\\",\\\"aug-sep-oct-weighted\\\",\\\"sep-oct-nov-weighted\\\",\\\"oct-nov-dec-weighted\\\",\\\"nov-dec-jan-weighted\\\"],\\\"index\\\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167],\\\"data\\\":[[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[false,false,false,true,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,true],[false,true,false,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[false,false,true,true,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,true,false,false,false,false,false,false,false,false,false,false],[false,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,false,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[false,false,false,false,false,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,false,true,true,true,true,false,false,false],[true,false,false,false,false,false,false,false,false,false,false,true],[true,false,false,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[false,false,false,false,false,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[true,false,false,false,false,false,false,false,false,false,false,true],[true,false,false,false,false,false,false,false,false,false,true,true],[true,false,false,false,false,false,false,false,false,false,true,true],[true,true,false,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[false,false,false,false,false,true,true,true,true,false,false,false],[false,false,false,false,false,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[false,false,false,false,true,true,true,true,true,false,false,false],[true,false,false,false,false,false,false,false,false,false,false,true],[true,false,false,false,false,false,false,false,false,false,true,true],[true,true,false,false,false,false,false,false,false,false,true,true],[true,true,false,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,false,false,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,true,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,false,false,false,false,false,false,false,false,false,false,true],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,true,true,true,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,true,false,false,false,false],[false,false,false,false,false,true,false,true,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[true,false,false,false,false,false,false,false,false,false,true,false],[true,true,false,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[true,true,true,true,false,false,false,false,false,true,true,true],[false,false,false,false,false,false,false,false,false,false,true,true],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,false],[false,false,false,false,false,false,false,false,false,false,false,true],[true,true,false,false,false,false,false,false,false,false,false,true],[true,true,false,false,false,false,false,false,false,false,false,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true],[true,true,true,true,false,false,false,false,false,false,true,true],[true,true,true,false,false,false,false,false,false,false,true,true]]}\", \"occupied_temperature_bins\": \"{\\\"columns\\\":[\\\"dec-jan-feb-weighted\\\",\\\"jan-feb-mar-weighted\\\",\\\"feb-mar-apr-weighted\\\",\\\"mar-apr-may-weighted\\\",\\\"apr-may-jun-weighted\\\",\\\"may-jun-jul-weighted\\\",\\\"jun-jul-aug-weighted\\\",\\\"jul-aug-sep-weighted\\\",\\\"aug-sep-oct-weighted\\\",\\\"sep-oct-nov-weighted\\\",\\\"oct-nov-dec-weighted\\\",\\\"nov-dec-jan-weighted\\\"],\\\"index\\\":[30,45,55,65,75,90],\\\"data\\\":[[true,true,true,true,false,false,false,false,false,true,true,true],[true,true,true,true,true,false,false,false,true,true,true,true],[false,false,false,true,true,true,false,false,true,true,true,false],[false,false,false,true,true,true,true,true,true,true,false,false],[false,false,false,false,true,true,true,true,true,true,false,false],[false,false,false,false,false,false,false,false,false,false,false,false]]}\", \"unoccupied_temperature_bins\": \"{\\\"columns\\\":[\\\"dec-jan-feb-weighted\\\",\\\"jan-feb-mar-weighted\\\",\\\"feb-mar-apr-weighted\\\",\\\"mar-apr-may-weighted\\\",\\\"apr-may-jun-weighted\\\",\\\"may-jun-jul-weighted\\\",\\\"jun-jul-aug-weighted\\\",\\\"jul-aug-sep-weighted\\\",\\\"aug-sep-oct-weighted\\\",\\\"sep-oct-nov-weighted\\\",\\\"oct-nov-dec-weighted\\\",\\\"nov-dec-jan-weighted\\\"],\\\"index\\\":[30,45,55,65,75,90],\\\"data\\\":[[true,true,true,true,true,false,false,false,false,true,true,true],[true,true,true,true,true,true,false,false,true,true,true,true],[true,true,true,true,true,true,true,true,true,true,true,false],[false,false,false,true,true,true,true,true,true,true,true,false],[false,false,false,true,true,true,true,true,true,true,true,false],[false,false,false,false,false,false,false,false,false,false,false,false]]}\", \"segment_type\": \"three_month_weighted\"}, \"warnings\": [], \"metadata\": {}, \"settings\": {}, \"totals_metrics\": {\"dec-jan-feb-weighted\": {\"observed_length\": 744.0, \"predicted_length\": 744.0, \"merged_length\": 744.0, \"num_parameters\": 175.0, \"observed_mean\": 16.975577813969302, \"predicted_mean\": 16.58706266632562, \"observed_variance\": 65.22169844339591, \"predicted_variance\": 56.686542235019076, \"observed_skew\": 0.6740964259461145, \"predicted_skew\": 0.683989411717173, \"observed_kurtosis\": -0.24110122998761918, \"predicted_kurtosis\": -0.16221917570260835, \"observed_cvstd\": 0.47606203337157116, \"predicted_cvstd\": 0.45421613638552577, \"r_squared\": 0.9223031131338872, \"r_squared_adj\": 0.8983648117226729, \"rmse\": 2.29563389994378, \"rmse_adj\": 2.6250220994677376, \"cvrmse\": 0.13523156178252085, \"cvrmse_adj\": 0.15463521349521261, \"mape\": 0.1127714638738227, \"mape_no_zeros\": 0.1127714638738227, \"num_meter_zeros\": 0.0, \"nmae\": 0.09540866875574809, \"nmbe\": -0.02288671124490181, \"autocorr_resid\": 0.909328820646372, \"confidence_level\": 0.9, \"n_prime\": 35.33145087930007, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -140.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"jan-feb-mar-weighted\": {\"observed_length\": 672.0, \"predicted_length\": 672.0, \"merged_length\": 672.0, \"num_parameters\": 175.0, \"observed_mean\": 15.262569550469774, \"predicted_mean\": 15.243265090681446, \"observed_variance\": 48.44331721300837, \"predicted_variance\": 46.89142527156544, \"observed_skew\": 0.4450696439474107, \"predicted_skew\": 0.5517814347258485, \"observed_kurtosis\": -0.652410136224447, \"predicted_kurtosis\": -0.3750912828138073, \"observed_cvstd\": 0.45636534457897304, \"predicted_cvstd\": 0.449564583896351, \"r_squared\": 0.9128953305898703, \"r_squared_adj\": 0.8821628363423446, \"rmse\": 2.0637525232039304, \"rmse_adj\": 2.399739275142423, \"cvrmse\": 0.13521658436212722, \"cvrmse_adj\": 0.15723035804731583, \"mape\": 0.13547948981858599, \"mape_no_zeros\": 0.13547948981858599, \"num_meter_zeros\": 0.0, \"nmae\": 0.10375864668886092, \"nmbe\": -0.0012648237064206537, \"autocorr_resid\": 0.8977903369559663, \"confidence_level\": 0.9, \"n_prime\": 36.19203461419263, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -139.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"feb-mar-apr-weighted\": {\"observed_length\": 743.0, \"predicted_length\": 743.0, \"merged_length\": 743.0, \"num_parameters\": 175.0, \"observed_mean\": 11.425157101955858, \"predicted_mean\": 11.640093325286806, \"observed_variance\": 21.97041644540524, \"predicted_variance\": 19.772084833039766, \"observed_skew\": 0.10217620503977408, \"predicted_skew\": 0.16964124156863408, \"observed_kurtosis\": -1.0130347341895818, \"predicted_kurtosis\": -0.9794024233486502, \"observed_cvstd\": 0.41053427167889, \"predicted_cvstd\": 0.3822629653909161, \"r_squared\": 0.8462941120259951, \"r_squared_adj\": 0.7988540231451294, \"rmse\": 1.855071225373516, \"rmse_adj\": 2.121684619027993, \"cvrmse\": 0.16236724001422692, \"cvrmse_adj\": 0.18570288356601983, \"mape\": 0.15078183723884633, \"mape_no_zeros\": 0.15078183723884633, \"num_meter_zeros\": 0.0, \"nmae\": 0.12029002515387002, \"nmbe\": 0.01881253985506701, \"autocorr_resid\": 0.8925826378847774, \"confidence_level\": 0.9, \"n_prime\": 42.17047036889776, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -133.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"mar-apr-may-weighted\": {\"observed_length\": 720.0, \"predicted_length\": 720.0, \"merged_length\": 720.0, \"num_parameters\": 179.0, \"observed_mean\": 10.314028599105063, \"predicted_mean\": 10.155674241143528, \"observed_variance\": 28.117755072479223, \"predicted_variance\": 20.20400141556595, \"observed_skew\": 0.404803927635106, \"predicted_skew\": 0.2516231419564327, \"observed_kurtosis\": -0.732538514378525, \"predicted_kurtosis\": -0.6973530907346053, \"observed_cvstd\": 0.5144744286185965, \"predicted_cvstd\": 0.4429061761559904, \"r_squared\": 0.7209086306447058, \"r_squared_adj\": 0.6283950100621176, \"rmse\": 2.8058070535766837, \"rmse_adj\": 3.2368711986559657, \"cvrmse\": 0.2720379361581507, \"cvrmse_adj\": 0.3138319006539138, \"mape\": 0.2750692657522693, \"mape_no_zeros\": 0.2750692657522693, \"num_meter_zeros\": 0.0, \"nmae\": 0.20514305011100478, \"nmbe\": -0.01535329831985089, \"autocorr_resid\": 0.9325856187580183, \"confidence_level\": 0.9, \"n_prime\": 25.11575892064237, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -154.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"apr-may-jun-weighted\": {\"observed_length\": 744.0, \"predicted_length\": 744.0, \"merged_length\": 744.0, \"num_parameters\": 179.0, \"observed_mean\": 4.839189352415977, \"predicted_mean\": 4.685786435437655, \"observed_variance\": 8.074138964674917, \"predicted_variance\": 5.738877262513476, \"observed_skew\": 0.474641811292158, \"predicted_skew\": 0.14842757111222937, \"observed_kurtosis\": -0.5561919082668467, \"predicted_kurtosis\": -1.0637666069418186, \"observed_cvstd\": 0.5875807440997417, \"predicted_cvstd\": 0.5115911680589924, \"r_squared\": 0.7835228662937936, \"r_squared_adj\": 0.7148182440714338, \"rmse\": 1.3367531148947842, \"rmse_adj\": 1.5339579290042593, \"cvrmse\": 0.2762349264608559, \"cvrmse_adj\": 0.31698654821978134, \"mape\": 0.3084077147273663, \"mape_no_zeros\": 0.3084077147273663, \"num_meter_zeros\": 0.0, \"nmae\": 0.20380075079159696, \"nmbe\": -0.03250012658728302, \"autocorr_resid\": 0.8356025957472101, \"confidence_level\": 0.9, \"n_prime\": 66.63297875447103, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -112.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"may-jun-jul-weighted\": {\"observed_length\": 720.0, \"predicted_length\": 720.0, \"merged_length\": 720.0, \"num_parameters\": 177.0, \"observed_mean\": 3.7194735800900776, \"predicted_mean\": 3.8167122592242158, \"observed_variance\": 6.169859741764847, \"predicted_variance\": 5.665560346692743, \"observed_skew\": 0.7657673047617335, \"predicted_skew\": 0.7788271983252987, \"observed_kurtosis\": -0.40516783269100554, \"predicted_kurtosis\": -0.48727788080107537, \"observed_cvstd\": 0.6682792407528718, \"predicted_cvstd\": 0.6240707359962647, \"r_squared\": 0.8856383330505185, \"r_squared_adj\": 0.8482914418142488, \"rmse\": 0.8466825220578726, \"rmse_adj\": 0.9749602920317336, \"cvrmse\": 0.22763504131070286, \"cvrmse_adj\": 0.26212319325255756, \"mape\": 0.27876288424571843, \"mape_no_zeros\": 0.27876288424571843, \"num_meter_zeros\": 0.0, \"nmae\": 0.16504216167987287, \"nmbe\": 0.02614312940805535, \"autocorr_resid\": 0.8143780968985377, \"confidence_level\": 0.9, \"n_prime\": 73.66037457215106, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -103.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"jun-jul-aug-weighted\": {\"observed_length\": 744.0, \"predicted_length\": 744.0, \"merged_length\": 744.0, \"num_parameters\": 175.0, \"observed_mean\": 4.169038803534541, \"predicted_mean\": 4.114697777476804, \"observed_variance\": 9.41630455952094, \"predicted_variance\": 8.767539554515215, \"observed_skew\": 0.6173355853287117, \"predicted_skew\": 0.7044536571183456, \"observed_kurtosis\": -1.1451418934682764, \"predicted_kurtosis\": -1.0291075107032448, \"observed_cvstd\": 0.7365400584480349, \"predicted_cvstd\": 0.7201002900010288, \"r_squared\": 0.9774928183202175, \"r_squared_adj\": 0.9705583873449324, \"rmse\": 0.46925161123485004, \"rmse_adj\": 0.5365820088876067, \"cvrmse\": 0.11255630694466437, \"cvrmse_adj\": 0.12870640792133875, \"mape\": 0.13323654301163465, \"mape_no_zeros\": 0.13323654301163465, \"num_meter_zeros\": 0.0, \"nmae\": 0.08286881867092781, \"nmbe\": -0.013034425587899601, \"autocorr_resid\": 0.7138709722080767, \"confidence_level\": 0.9, \"n_prime\": 124.21004855629569, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -51.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"jul-aug-sep-weighted\": {\"observed_length\": 744.0, \"predicted_length\": 744.0, \"merged_length\": 744.0, \"num_parameters\": 175.0, \"observed_mean\": 4.1709005456022945, \"predicted_mean\": 4.182022797943457, \"observed_variance\": 8.223727028655944, \"predicted_variance\": 8.170941911609846, \"observed_skew\": 0.6039917635516562, \"predicted_skew\": 0.6008191353533886, \"observed_kurtosis\": -1.104567091685139, \"predicted_kurtosis\": -1.1394320666710778, \"observed_cvstd\": 0.6880128818843937, \"predicted_cvstd\": 0.683977359717002, \"r_squared\": 0.9768439917184871, \"r_squared_adj\": 0.9697096581810492, \"rmse\": 0.43719217592265724, \"rmse_adj\": 0.49992253709943374, \"cvrmse\": 0.10481961177032212, \"cvrmse_adj\": 0.11985961583920793, \"mape\": 0.11037269338761324, \"mape_no_zeros\": 0.11037269338761324, \"num_meter_zeros\": 0.0, \"nmae\": 0.07509987042751545, \"nmbe\": 0.002666630915688038, \"autocorr_resid\": 0.7036858033725844, \"confidence_level\": 0.9, \"n_prime\": 129.40048091871353, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -46.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"aug-sep-oct-weighted\": {\"observed_length\": 720.0, \"predicted_length\": 720.0, \"merged_length\": 720.0, \"num_parameters\": 178.0, \"observed_mean\": 3.8111630212843957, \"predicted_mean\": 3.8969274480912026, \"observed_variance\": 7.03023022443011, \"predicted_variance\": 5.917042140062216, \"observed_skew\": 0.8063588101055821, \"predicted_skew\": 0.8177362265470347, \"observed_kurtosis\": -0.6446318158043067, \"predicted_kurtosis\": -0.5796888619043243, \"observed_cvstd\": 0.6961920357476105, \"predicted_cvstd\": 0.624642908517579, \"r_squared\": 0.9072187136953531, \"r_squared_adj\": 0.8766917840054693, \"rmse\": 0.8174788946373461, \"rmse_adj\": 0.942200117303961, \"cvrmse\": 0.21449591373340113, \"cvrmse_adj\": 0.2472211532390528, \"mape\": 0.24399823591817513, \"mape_no_zeros\": 0.24399823591817513, \"num_meter_zeros\": 0.0, \"nmae\": 0.15924315379129975, \"nmbe\": 0.02250347894535967, \"autocorr_resid\": 0.8516493854889932, \"confidence_level\": 0.9, \"n_prime\": 57.685025731649155, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -120.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"sep-oct-nov-weighted\": {\"observed_length\": 744.0, \"predicted_length\": 744.0, \"merged_length\": 744.0, \"num_parameters\": 180.0, \"observed_mean\": 5.662585289703436, \"predicted_mean\": 5.892028987524305, \"observed_variance\": 6.8229944941965295, \"predicted_variance\": 7.105253187619346, \"observed_skew\": 0.562617890807956, \"predicted_skew\": 0.9000700726856201, \"observed_kurtosis\": 0.3556557563358336, \"predicted_kurtosis\": 0.5236348516581333, \"observed_cvstd\": 0.4615989528124706, \"predicted_cvstd\": 0.4527067449282713, \"r_squared\": 0.7217697816996256, \"r_squared_adj\": 0.6328151825982625, \"rmse\": 1.4663887103544158, \"rmse_adj\": 1.6842091564848167, \"cvrmse\": 0.25896099313873894, \"cvrmse_adj\": 0.2974276007016263, \"mape\": 0.28006307936585645, \"mape_no_zeros\": 0.28006307936585645, \"num_meter_zeros\": 0.0, \"nmae\": 0.20985654879720786, \"nmbe\": 0.040519248025822775, \"autocorr_resid\": 0.8278570430190151, \"confidence_level\": 0.9, \"n_prime\": 70.0680397753187, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -110.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"oct-nov-dec-weighted\": {\"observed_length\": 721.0, \"predicted_length\": 721.0, \"merged_length\": 721.0, \"num_parameters\": 178.0, \"observed_mean\": 11.77842601381678, \"predicted_mean\": 11.585596535210962, \"observed_variance\": 19.87516548206451, \"predicted_variance\": 15.6749924086204, \"observed_skew\": 0.30299107664967073, \"predicted_skew\": 0.1608893890962298, \"observed_kurtosis\": -0.726876852225411, \"predicted_kurtosis\": -0.47466054841293204, \"observed_cvstd\": 0.37876470485629116, \"predicted_cvstd\": 0.3419689331306722, \"r_squared\": 0.813719584267142, \"r_squared_adj\": 0.752542621166683, \"rmse\": 1.9347941195771303, \"rmse_adj\": 2.229474330264044, \"cvrmse\": 0.1642659313992807, \"cvrmse_adj\": 0.18928457228909368, \"mape\": 0.15328836796491105, \"mape_no_zeros\": 0.15328836796491105, \"num_meter_zeros\": 0.0, \"nmae\": 0.12845616084972555, \"nmbe\": -0.01637141315653005, \"autocorr_resid\": 0.9123704443590211, \"confidence_level\": 0.9, \"n_prime\": 33.038007779043255, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -145.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}, \"nov-dec-jan-weighted\": {\"observed_length\": 743.0, \"predicted_length\": 743.0, \"merged_length\": 743.0, \"num_parameters\": 174.0, \"observed_mean\": 12.938116860935468, \"predicted_mean\": 12.991986751643429, \"observed_variance\": 17.952318231780367, \"predicted_variance\": 19.502650561511743, \"observed_skew\": 0.08156247119700205, \"predicted_skew\": 0.3593926307458516, \"observed_kurtosis\": -0.5247576326379408, \"predicted_kurtosis\": -0.42405170880311305, \"observed_cvstd\": 0.3277039331086056, \"predicted_cvstd\": 0.3401446972937604, \"r_squared\": 0.8545401657942181, \"r_squared_adj\": 0.8099802870058271, \"rmse\": 1.6922376139098074, \"rmse_adj\": 1.933746843092955, \"cvrmse\": 0.13079473868559982, \"cvrmse_adj\": 0.14946122869948625, \"mape\": 0.11734654429743072, \"mape_no_zeros\": 0.11734654429743072, \"num_meter_zeros\": 0.0, \"nmae\": 0.10413020519481771, \"nmbe\": 0.004163657763102415, \"autocorr_resid\": 0.9163823729027268, \"confidence_level\": 0.9, \"n_prime\": 32.41936359452597, \"single_tailed_confidence_level\": 0.95, \"degrees_of_freedom\": -142.0, \"t_stat\": null, \"cvrmse_auto_corr_correction\": null, \"approx_factor_auto_corr_correction\": null, \"fsu_base_term\": null}}, \"avgs_metrics\": null}"
  },
  {
    "path": "tests/snapshots/__init__.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n"
  },
  {
    "path": "tests/snapshots/snap_test_features.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n# snapshottest: v1 - https://goo.gl/zC4yUc\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\nfrom __future__ import unicode_literals\n\nfrom snapshottest import Snapshot\n\n\nsnapshots = Snapshot()\n\nsnapshots[\"test_compute_temperature_features_hourly_hourly_degree_days values\"] = [\n    5.25,\n    5.72,\n    4.73,\n    4.33,\n    1.0,\n    0.0,\n]\n\nsnapshots[\n    \"test_compute_temperature_features_hourly_hourly_degree_days_use_mean_false values\"\n] = [0.22, 0.24, 0.2, 0.18, 1.0, 0.0]\n\nsnapshots[\"test_compute_temperature_features_daily_daily_degree_days values\"] = [\n    11.05,\n    11.61,\n    3.61,\n    3.25,\n    1.0,\n    0.0,\n]\n\nsnapshots[\n    \"test_compute_temperature_features_daily_daily_degree_days_use_mean_false values\"\n] = [11.05, 11.61, 3.61, 3.25, 1.0, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_monthly_daily_degree_days values\"\n] = [10.83, 11.39, 3.68, 3.31, 30.38, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_monthly_daily_degree_days_use_mean_false values\"\n] = [324.38, 341.38, 112.59, 101.33, 30.38, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_bimonthly_daily_degree_days values\"\n] = [10.94, 11.51, 3.65, 3.28, 61.62, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_daily_hourly_degree_days_use_mean_false values\"\n] = [11.43, 12.01, 4.05, 3.7, 23.99, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_monthly_hourly_degree_days values\"\n] = [11.22, 11.79, 4.11, 3.76, 729.23, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_monthly_hourly_degree_days_use_mean_false values\"\n] = [336.54, 353.64, 125.96, 115.09, 729.23, 0.0]\n\nsnapshots[\n    \"test_compute_temperature_features_billing_bimonthly_hourly_degree_days values\"\n] = [11.33, 11.9, 4.07, 3.72, 1478.77, 0.0]\n\nsnapshots[\"test_compute_temperature_features_daily_hourly_degree_days values\"] = [\n    11.44,\n    12.02,\n    4.05,\n    3.7,\n    23.99,\n    0.0,\n]\n"
  },
  {
    "path": "tests/test_caltrack_design_matrices.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport pytest\n\nfrom opendsm.eemeter.models.hourly_caltrack.design_matrices import (\n    create_caltrack_hourly_preliminary_design_matrix,\n    create_caltrack_hourly_segmented_design_matrices,\n    create_caltrack_daily_design_matrix,\n    create_caltrack_billing_design_matrix,\n)\nfrom opendsm.eemeter.common.features import (\n    estimate_hour_of_week_occupancy,\n    fit_temperature_bins,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import segment_time_series\n\n\ndef test_create_caltrack_hourly_preliminary_design_matrix(\n    il_electricity_cdd_hdd_hourly,\n):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    design_matrix = create_caltrack_hourly_preliminary_design_matrix(\n        meter_data[:1000], temperature_data\n    )\n    assert design_matrix.shape == (1000, 7)\n    assert sorted(design_matrix.columns) == [\n        \"cdd_65\",\n        \"hdd_50\",\n        \"hour_of_week\",\n        \"meter_value\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n        \"temperature_mean\",\n    ]\n    # In newer pandas, categorical columns (like hour_of_week) arent included in sum\n    design_matrix.hour_of_week = design_matrix.hour_of_week.astype(float)\n    assert round(design_matrix.sum().sum(), 2) == 136352.61\n\n\ndef test_create_caltrack_daily_design_matrix(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    design_matrix = create_caltrack_daily_design_matrix(\n        meter_data[:100], temperature_data\n    )\n    assert design_matrix.shape == (100, 6)\n    assert sorted(design_matrix.columns) == [\n        \"meter_value\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(design_matrix.sum().sum(), 2) == 9267.06\n\n\ndef test_create_caltrack_billing_design_matrix(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    design_matrix = create_caltrack_billing_design_matrix(\n        meter_data[:10], temperature_data\n    )\n    assert design_matrix.shape == (275, 6)\n    assert sorted(design_matrix.columns) == [\n        \"meter_value\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(design_matrix.sum().sum(), 2) == 29925.27\n\n\n@pytest.fixture\ndef preliminary_hourly_design_matrix(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    return create_caltrack_hourly_preliminary_design_matrix(\n        meter_data[:1000], temperature_data\n    )\n\n\n@pytest.fixture\ndef segmentation(preliminary_hourly_design_matrix):\n    return segment_time_series(\n        preliminary_hourly_design_matrix.index, \"three_month_weighted\"\n    )\n\n\n@pytest.fixture\ndef occupancy_lookup(preliminary_hourly_design_matrix, segmentation):\n    return estimate_hour_of_week_occupancy(\n        preliminary_hourly_design_matrix, segmentation=segmentation\n    )\n\n\n@pytest.fixture\ndef temperature_bins(preliminary_hourly_design_matrix, segmentation, occupancy_lookup):\n    return fit_temperature_bins(\n        preliminary_hourly_design_matrix,\n        segmentation=segmentation,\n        occupancy_lookup=occupancy_lookup,\n    )\n\n\ndef test_create_caltrack_hourly_segmented_design_matrices(\n    preliminary_hourly_design_matrix, segmentation, occupancy_lookup, temperature_bins\n):\n    occupied_temperature_bins, unoccupied_temperature_bins = temperature_bins\n    design_matrices = create_caltrack_hourly_segmented_design_matrices(\n        preliminary_hourly_design_matrix,\n        segmentation,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n\n    design_matrix = design_matrices[\"dec-jan-feb-weighted\"]\n    assert design_matrix.shape == (1000, 8)\n    assert sorted(design_matrix.columns) == [\n        \"bin_0_occupied\",\n        \"bin_0_unoccupied\",\n        \"bin_1_unoccupied\",\n        \"bin_2_unoccupied\",\n        \"bin_3_unoccupied\",\n        \"hour_of_week\",\n        \"meter_value\",\n        \"weight\",\n    ]\n    design_matrix.hour_of_week = design_matrix.hour_of_week.astype(float)\n    assert round(design_matrix.sum().sum(), 2) == 126210.07\n\n    design_matrix = design_matrices[\"mar-apr-may-weighted\"]\n    assert design_matrix.shape == (1000, 5)\n    assert sorted(design_matrix.columns) == [\n        \"bin_0_occupied\",\n        \"bin_0_unoccupied\",\n        \"hour_of_week\",\n        \"meter_value\",\n        \"weight\",\n    ]\n    design_matrix.hour_of_week = design_matrix.hour_of_week.astype(float)\n    assert round(design_matrix.sum().sum(), 2) == 167659.28\n\n\ndef test_create_caltrack_billing_design_matrix_empty_temp(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"][:0]\n    with pytest.raises(ValueError):\n        design_matrix = create_caltrack_billing_design_matrix(\n            meter_data[:10], temperature_data\n        )\n\n\ndef test_create_caltrack_billing_design_matrix_partial_empty_temp(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"][:200]\n    design_matrix = create_caltrack_billing_design_matrix(\n        meter_data[:10], temperature_data\n    )\n    assert \"n_days_kept\" in design_matrix.columns\n"
  },
  {
    "path": "tests/test_caltrack_hourly.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.eemeter.models.hourly_caltrack.model import (\n    caltrack_hourly_fit_feature_processor,\n    caltrack_hourly_prediction_feature_processor,\n    fit_caltrack_hourly_model_segment,\n    fit_caltrack_hourly_model,\n)\nfrom opendsm.eemeter.common.features import (\n    compute_time_features,\n)\n\n\n@pytest.fixture\ndef segmented_data():\n    index = pd.date_range(start=\"2017-01-01\", periods=24, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index)\n    segmented_data = pd.DataFrame(\n        {\n            \"hour_of_week\": time_features.hour_of_week,\n            \"temperature_mean\": np.linspace(0, 100, 24),\n            \"meter_value\": np.linspace(10, 70, 24),\n            \"weight\": np.ones((24,)),\n        },\n        index=index,\n    )\n    return segmented_data\n\n\n@pytest.fixture\ndef occupancy_lookup():\n    index = pd.Categorical(range(168))\n    occupancy = pd.Series([i % 2 == 0 for i in range(168)], index=index)\n    return pd.DataFrame(\n        {\"dec-jan-feb-weighted\": occupancy, \"jan-feb-mar-weighted\": occupancy}\n    )\n\n\n@pytest.fixture\ndef occupied_temperature_bins():\n    bins = pd.Series([True, True, True], index=[30, 60, 90])\n    return pd.DataFrame({\"dec-jan-feb-weighted\": bins, \"jan-feb-mar-weighted\": bins})\n\n\n@pytest.fixture\ndef unoccupied_temperature_bins():\n    bins = pd.Series([False, True, True], index=[30, 60, 90])\n    return pd.DataFrame({\"dec-jan-feb-weighted\": bins, \"jan-feb-mar-weighted\": bins})\n\n\ndef test_caltrack_hourly_fit_feature_processor(\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    result = caltrack_hourly_fit_feature_processor(\n        \"dec-jan-feb-weighted\",\n        segmented_data,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n    assert list(result.columns) == [\n        \"meter_value\",\n        \"hour_of_week\",\n        \"bin_0_occupied\",\n        \"bin_1_occupied\",\n        \"bin_2_occupied\",\n        \"bin_3_occupied\",\n        \"bin_0_unoccupied\",\n        \"bin_1_unoccupied\",\n        \"bin_2_unoccupied\",\n        \"weight\",\n    ]\n    assert result.shape == (24, 10)\n    result.hour_of_week = result.hour_of_week.astype(float)\n    assert round(result.sum().sum(), 2) == 5916.0\n\n\ndef test_caltrack_hourly_prediction_feature_processor(\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    result = caltrack_hourly_prediction_feature_processor(\n        \"dec-jan-feb-weighted\",\n        segmented_data,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n    assert list(result.columns) == [\n        \"hour_of_week\",\n        \"bin_0_occupied\",\n        \"bin_1_occupied\",\n        \"bin_2_occupied\",\n        \"bin_3_occupied\",\n        \"bin_0_unoccupied\",\n        \"bin_1_unoccupied\",\n        \"bin_2_unoccupied\",\n        \"weight\",\n    ]\n    assert result.shape == (24, 9)\n    result.hour_of_week = result.hour_of_week.astype(float)\n    assert round(result.sum().sum(), 2) == 4956.0\n\n\n@pytest.fixture\ndef segmented_design_matrices(\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    return {\n        \"dec-jan-feb-weighted\": caltrack_hourly_fit_feature_processor(\n            \"dec-jan-feb-weighted\",\n            segmented_data,\n            occupancy_lookup,\n            occupied_temperature_bins,\n            unoccupied_temperature_bins,\n        )\n    }\n\n\ndef test_fit_caltrack_hourly_model_segment(segmented_design_matrices):\n    segment_name = \"dec-jan-feb-weighted\"\n    segment_data = segmented_design_matrices[segment_name]\n    segment_model = fit_caltrack_hourly_model_segment(segment_name, segment_data)\n    assert segment_model.formula == (\n        \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied\"\n        \" + bin_1_occupied + bin_2_occupied + bin_3_occupied\"\n        \" + bin_0_unoccupied + bin_1_unoccupied + bin_2_unoccupied\"\n    )\n    assert segment_model.segment_name == \"dec-jan-feb-weighted\"\n    assert len(segment_model.model_params.keys()) == 31\n    assert segment_model.model is not None\n    assert segment_model.warnings is not None\n    prediction = segment_model.predict(segment_data)\n    assert round(prediction.sum(), 2) == 960.0\n\n\n@pytest.fixture\ndef temps():\n    index = pd.date_range(start=\"2017-01-01\", periods=24, freq=\"h\", tz=\"UTC\")\n    temps = pd.Series(np.linspace(0, 100, 24), index=index)\n    return temps\n\n\n@pytest.mark.parametrize(\"segment_type\", [\"three_month_weighted\"])\ndef test_fit_caltrack_hourly_model(\n    segmented_design_matrices,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n    temps,\n    segment_type,\n):\n    segmented_model_results = fit_caltrack_hourly_model(\n        segmented_design_matrices,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=segment_type,\n    )\n\n    assert segmented_model_results.model.segment_models is not None\n    assert str(segmented_model_results).startswith(\"CalTRACKHourlyModelResults\")\n    prediction = segmented_model_results.predict(temps.index, temps).result\n\n\n@pytest.mark.parametrize(\"segment_type\", [\"single\", \"three_month_weighted\"])\ndef test_serialize_caltrack_hourly_model(\n    segmented_design_matrices,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n    temps,\n    segment_type,\n):\n    segmented_model = fit_caltrack_hourly_model(\n        segmented_design_matrices,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=segment_type,\n    )\n    assert json.dumps(segmented_model.json())\n\n\n@pytest.fixture\ndef segmented_data_nans():\n    num_periods = 200\n    index = pd.date_range(start=\"2017-01-01\", periods=num_periods, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index)\n    segmented_data = pd.DataFrame(\n        {\n            \"hour_of_week\": time_features.hour_of_week,\n            \"temperature_mean\": np.linspace(0, 100, num_periods),\n            \"meter_value\": np.linspace(10, 70, num_periods),\n            \"weight\": np.ones((num_periods,)),\n        },\n        index=index,\n    )\n    return segmented_data\n\n\n@pytest.fixture\ndef occupancy_lookup_nans():\n    index = pd.Categorical(range(168))\n    occupancy = pd.Series([i % 2 == 0 for i in range(168)], index=index)\n    occupancy_nans = pd.Series([np.nan for i in range(168)], index=index)\n    return pd.DataFrame(\n        {\n            \"dec-jan-feb-weighted\": occupancy,\n            \"jan-feb-mar-weighted\": occupancy,\n            \"apr-may-jun-weighted\": occupancy_nans,\n        }\n    )\n\n\n@pytest.fixture\ndef temperature_bins_nans():\n    bins = pd.Series([True, True, True], index=[30, 60, 90])\n    bins_nans = pd.Series([False, False, False], index=[30, 60, 90])\n    return pd.DataFrame(\n        {\n            \"dec-jan-feb-weighted\": bins,\n            \"jan-feb-mar-weighted\": bins,\n            \"apr-may-jun-weighted\": bins_nans,\n        }\n    )\n\n\n@pytest.fixture\ndef segmented_design_matrices_nans(\n    segmented_data_nans, occupancy_lookup_nans, temperature_bins_nans\n):\n    return {\n        \"dec-jan-feb-weighted\": caltrack_hourly_fit_feature_processor(\n            \"dec-jan-feb-weighted\",\n            segmented_data_nans,\n            occupancy_lookup_nans,\n            temperature_bins_nans,\n            temperature_bins_nans,\n        ),\n        \"apr-may-jun-weighted\": caltrack_hourly_fit_feature_processor(\n            \"apr-may-jun-weighted\",\n            segmented_data_nans,\n            occupancy_lookup_nans,\n            temperature_bins_nans,\n            temperature_bins_nans,\n        ),\n    }\n\n\n@pytest.mark.parametrize(\"segment_type\", [\"three_month_weighted\"])\ndef test_fit_caltrack_hourly_model_nans_less_than_week_predict(\n    segmented_design_matrices_nans,\n    occupancy_lookup_nans,\n    temperature_bins_nans,\n    temps_extended,\n    temps,\n    segment_type,\n):\n    segmented_model_results = fit_caltrack_hourly_model(\n        segmented_design_matrices_nans,\n        occupancy_lookup_nans,\n        temperature_bins_nans,\n        temperature_bins_nans,\n        segment_type=segment_type,\n    )\n\n    assert segmented_model_results.model.segment_models is not None\n    assert segmented_model_results.model.model_lookup[\"jan\"].model is not None\n    assert segmented_model_results.model.model_lookup[\"may\"].model is not None\n    assert segmented_model_results.model.model_lookup[\"may\"].warnings == []\n    prediction = segmented_model_results.predict(temps.index, temps).result\n    assert prediction.shape[0] == 24\n    assert prediction[\"predicted_usage\"].sum().round() == 955.0\n\n\n@pytest.fixture\ndef segmented_data_nans_less_than_week():\n    num_periods = 4\n    index = pd.date_range(start=\"2017-01-01\", periods=num_periods, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index)\n    segmented_data = pd.DataFrame(\n        {\n            \"hour_of_week\": time_features.hour_of_week,\n            \"temperature_mean\": np.linspace(0, 100, num_periods),\n            \"meter_value\": np.linspace(10, 70, num_periods),\n            \"weight\": np.ones((num_periods,)),\n        },\n        index=index,\n    )\n    return segmented_data\n\n\n@pytest.fixture\ndef occupancy_lookup_nans_less_than_week():\n    index = pd.Categorical(range(168))\n    occupancy = pd.Series([i % 2 == 0 for i in range(168)], index=index)\n    occupancy_nans_less_than_week = pd.Series([np.nan for i in range(168)], index=index)\n    return pd.DataFrame(\n        {\n            \"dec-jan-feb-weighted\": occupancy,\n            \"jan-feb-mar-weighted\": occupancy,\n            \"apr-may-jun-weighted\": occupancy_nans_less_than_week,\n        }\n    )\n\n\n@pytest.fixture\ndef temperature_bins_nans_less_than_week():\n    bins = pd.Series([True, True, True], index=[30, 60, 90])\n    bins_nans_less_than_week = pd.Series([False, False, False], index=[30, 60, 90])\n    return pd.DataFrame(\n        {\n            \"dec-jan-feb-weighted\": bins,\n            \"jan-feb-mar-weighted\": bins,\n            \"apr-may-jun-weighted\": bins_nans_less_than_week,\n        }\n    )\n\n\n@pytest.fixture\ndef segmented_design_matrices_nans_less_than_week(\n    segmented_data_nans_less_than_week,\n    occupancy_lookup_nans_less_than_week,\n    temperature_bins_nans_less_than_week,\n):\n    return {\n        \"dec-jan-feb-weighted\": caltrack_hourly_fit_feature_processor(\n            \"dec-jan-feb-weighted\",\n            segmented_data_nans_less_than_week,\n            occupancy_lookup_nans_less_than_week,\n            temperature_bins_nans_less_than_week,\n            temperature_bins_nans_less_than_week,\n        ),\n        \"apr-may-jun-weighted\": caltrack_hourly_fit_feature_processor(\n            \"apr-may-jun-weighted\",\n            segmented_data_nans_less_than_week,\n            occupancy_lookup_nans_less_than_week,\n            temperature_bins_nans_less_than_week,\n            temperature_bins_nans_less_than_week,\n        ),\n    }\n\n\n@pytest.fixture\ndef temps_extended():\n    index = pd.date_range(start=\"2017-01-01\", periods=168, freq=\"h\", tz=\"UTC\")\n    temps = pd.Series(1, index=index)\n    return temps\n\n\n@pytest.mark.parametrize(\"segment_type\", [\"three_month_weighted\"])\ndef test_fit_caltrack_hourly_model_nans_less_than_week_fit(\n    segmented_design_matrices_nans_less_than_week,\n    occupancy_lookup_nans_less_than_week,\n    temperature_bins_nans_less_than_week,\n    temps_extended,\n    segment_type,\n):\n    segmented_model_results = fit_caltrack_hourly_model(\n        segmented_design_matrices_nans_less_than_week,\n        occupancy_lookup_nans_less_than_week,\n        temperature_bins_nans_less_than_week,\n        temperature_bins_nans_less_than_week,\n        segment_type=segment_type,\n    )\n\n    assert segmented_model_results.model.segment_models is not None\n    prediction = segmented_model_results.predict(\n        temps_extended.index, temps_extended\n    ).result\n    assert prediction.shape[0] == 168\n    assert prediction.dropna().shape[0] == 4\n\n\n@pytest.fixture\ndef segmented_design_matrices_empty_models(\n    segmented_data,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    return {\n        \"dec-jan-feb-weighted\": caltrack_hourly_fit_feature_processor(\n            \"dec-jan-feb-weighted\",\n            segmented_data[:0],\n            occupancy_lookup,\n            occupied_temperature_bins,\n            unoccupied_temperature_bins,\n        )\n    }\n\n\n@pytest.mark.parametrize(\"segment_type\", [\"three_month_weighted\"])\ndef test_predict_caltrack_hourly_model_empty_models(\n    temps,\n    segmented_design_matrices_empty_models,\n    occupancy_lookup,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n    segment_type,\n):\n    segmented_model_results = fit_caltrack_hourly_model(\n        segmented_design_matrices_empty_models,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=segment_type,\n    )\n\n    assert segmented_model_results.model.segment_models is not None\n    prediction = segmented_model_results.predict(temps.index, temps).result\n    assert prediction.shape[0] == 24\n    assert prediction.dropna().shape[0] == 0\n\n\n@pytest.fixture\ndef occupancy_lookup_zeroes():\n    index = pd.Categorical(range(168))\n    occupancy = pd.Series([False] * 168, index=index)\n    return pd.DataFrame(\n        {\"dec-jan-feb-weighted\": occupancy, \"jan-feb-mar-weighted\": occupancy}\n    )\n\n\n@pytest.fixture\ndef segmented_design_matrices_single_mode(\n    segmented_data,\n    occupancy_lookup_zeroes,\n    occupied_temperature_bins,\n    unoccupied_temperature_bins,\n):\n    return {\n        \"dec-jan-feb-weighted\": caltrack_hourly_fit_feature_processor(\n            \"dec-jan-feb-weighted\",\n            segmented_data,\n            occupancy_lookup_zeroes,\n            occupied_temperature_bins,\n            unoccupied_temperature_bins,\n        )\n    }\n\n\ndef test_fit_caltrack_hourly_model_segment_single_mode(\n    segmented_design_matrices_single_mode,\n):\n    segment_name = \"dec-jan-feb-weighted\"\n    segment_data = segmented_design_matrices_single_mode[segment_name]\n    segment_model = fit_caltrack_hourly_model_segment(segment_name, segment_data)\n    assert segment_model.formula == (\n        \"meter_value ~ C(hour_of_week) - 1 + bin_0_occupied + bin_1_occupied\"\n        \" + bin_2_occupied + bin_3_occupied + bin_0_unoccupied + bin_1_unoccupied\"\n        \" + bin_2_unoccupied\"\n    )\n    assert segment_model.segment_name == \"dec-jan-feb-weighted\"\n    assert len(segment_model.model_params.keys()) == 31\n    assert segment_model.model is not None\n    assert segment_model.warnings is not None\n    prediction = segment_model.predict(segment_data)\n    assert round(prediction.sum(), 2) == 960.0\n"
  },
  {
    "path": "tests/test_derivatives.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.eemeter.models.hourly_caltrack.design_matrices import (\n    create_caltrack_billing_design_matrix,\n    create_caltrack_hourly_preliminary_design_matrix,\n    create_caltrack_hourly_segmented_design_matrices,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.model import fit_caltrack_hourly_model\nfrom opendsm.eemeter.models.hourly_caltrack.derivatives import (\n    metered_savings,\n    modeled_savings,\n)\nfrom opendsm.eemeter.common.features import (\n    estimate_hour_of_week_occupancy,\n    fit_temperature_bins,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import segment_time_series\nfrom opendsm.eemeter.common.transform import get_baseline_data, get_reporting_data\nfrom opendsm.eemeter.models.daily.model import DailyModel\nfrom opendsm.eemeter.models.daily.data import DailyBaselineData, DailyReportingData\nfrom opendsm.eemeter.models.billing.model import BillingModel\nfrom opendsm.eemeter.models.billing.data import (\n    BillingBaselineData,\n    BillingReportingData,\n)\n\n\n@pytest.fixture\ndef baseline_data_daily(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_daily[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    baseline_data = DailyBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n\n    return baseline_data\n\n\n@pytest.fixture\ndef baseline_model_daily(baseline_data_daily):\n    model_results = DailyModel().fit(baseline_data_daily, ignore_disqualification=True)\n    return model_results\n\n\n@pytest.fixture\ndef reporting_data_daily(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    blackout_end_date = il_electricity_cdd_hdd_daily[\"blackout_end_date\"]\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date\n    )\n    reporting_data = DailyBaselineData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n    return reporting_data\n\n\n@pytest.fixture\ndef reporting_model_daily(reporting_data_daily):\n    model_results = DailyModel().fit(reporting_data_daily, ignore_disqualification=True)\n    return model_results\n\n\n@pytest.fixture\ndef reporting_meter_data_daily():\n    index = pd.date_range(\"2011-01-01\", freq=\"D\", periods=60, tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\n@pytest.fixture\ndef reporting_temperature_data():\n    index = pd.date_range(\"2011-01-01\", freq=\"D\", periods=60, tz=\"UTC\")\n    return pd.Series(np.arange(30.0, 90.0), index=index).asfreq(\"h\").ffill()\n\n\ndef test_metered_savings_cdd_hdd_daily(\n    baseline_model_daily, reporting_meter_data_daily, reporting_temperature_data\n):\n    reporting_data = DailyReportingData.from_series(\n        reporting_meter_data_daily, reporting_temperature_data, is_electricity_data=True\n    )\n    results = baseline_model_daily.predict(reporting_data)\n    metered_savings = results[\"predicted\"] - results[\"observed\"]\n\n    # platform difference on Windows requires bigger tolerance here\n    assert np.isclose(metered_savings.sum(), 1630, rtol=1e-2)\n\n\n@pytest.fixture\ndef baseline_model_billing(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_billing_monthly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    baseline_data = BillingBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n    model_results = BillingModel().fit(baseline_data, ignore_disqualification=True)\n    return model_results\n\n\n@pytest.fixture\ndef reporting_model_billing(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    meter_data.value = meter_data.value - 50\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_billing_monthly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    baseline_data = BillingBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n    model_results = BillingModel().fit(baseline_data, ignore_disqualification=True)\n    return model_results\n\n\n@pytest.fixture\ndef reporting_meter_data_billing():\n    index = pd.date_range(\"2011-01-01\", freq=\"MS\", periods=13, tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\ndef test_metered_savings_cdd_hdd_billing(\n    baseline_model_billing, reporting_meter_data_billing, reporting_temperature_data\n):\n    reporting_data = BillingReportingData.from_series(\n        reporting_meter_data_billing,\n        reporting_temperature_data,\n        is_electricity_data=True,\n    )\n    results = baseline_model_billing.predict(reporting_data)\n    metered_savings = (results[\"predicted\"] - results[\"observed\"]).sum()\n    assert np.isclose(metered_savings, 1605.14, rtol=1e-3)\n\n\ndef test_metered_savings_cdd_hdd_billing_no_reporting_data(\n    baseline_model_billing, reporting_meter_data_billing, reporting_temperature_data\n):\n    # TODO test makes less sense without the use of derivatives functions. can just be merged with other predict() tests\n    results = baseline_model_billing.predict(\n        BillingReportingData.from_series(\n            None, reporting_temperature_data, is_electricity_data=True\n        )\n    )\n    assert list(results.columns) == [\n        \"season\",\n        \"day_of_week\",\n        \"weekday_weekend\",\n        \"temperature\",\n        \"predicted\",\n        \"predicted_unc\",\n        \"heating_load\",\n        \"cooling_load\",\n        \"model_split\",\n        \"model_type\",\n    ]\n    predicted_sum = results.predicted.sum()\n    assert np.isclose(predicted_sum, 1607.1, rtol=1e-3)\n\n\ndef test_metered_savings_cdd_hdd_billing_single_record_reporting_data(\n    baseline_model_billing, reporting_meter_data_billing, reporting_temperature_data\n):\n    # results, error_bands = metered_savings(\n    #     baseline_model_billing,\n    #     reporting_meter_data_billing[:1],\n    #     reporting_temperature_data,\n    #     billing_data=True,\n    # )\n    results = baseline_model_billing.predict(\n        BillingReportingData.from_series(\n            reporting_meter_data_billing[:1],\n            reporting_temperature_data,\n            is_electricity_data=True,\n        )\n    )\n    assert list(results.columns) == [\n        \"season\",\n        \"day_of_week\",\n        \"weekday_weekend\",\n        \"temperature\",\n        \"predicted\",\n        \"predicted_unc\",\n        \"heating_load\",\n        \"cooling_load\",\n        \"model_split\",\n        \"model_type\",\n    ]\n    assert round(results.predicted.sum(), 2) == 0.0\n\n\n@pytest.fixture\ndef baseline_model_billing_single_record_baseline_data(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_billing_monthly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    baseline_data = create_caltrack_billing_design_matrix(\n        baseline_meter_data, temperature_data\n    ).rename(columns={\"meter_value\": \"observed\", \"temperature_mean\": \"temperature\"})\n    baseline_data = baseline_data[:60]\n\n    model_results = BillingModel().fit(\n        BillingBaselineData(baseline_data, is_electricity_data=True),\n        ignore_disqualification=True,\n    )\n    return model_results\n\n\ndef test_metered_savings_cdd_hdd_billing_single_record_baseline_data(\n    baseline_model_billing_single_record_baseline_data,\n    reporting_meter_data_billing,\n    reporting_temperature_data,\n):\n    # results, error_bands = metered_savings(\n    #     baseline_model_billing_single_record_baseline_data,\n    #     reporting_meter_data_billing,\n    #     reporting_temperature_data,\n    #     billing_data=True,\n    # )\n    results = baseline_model_billing_single_record_baseline_data.predict(\n        BillingReportingData.from_series(\n            reporting_meter_data_billing,\n            reporting_temperature_data,\n            is_electricity_data=True,\n        ),\n        ignore_disqualification=True,\n    )\n    assert list(results.columns) == [\n        \"season\",\n        \"day_of_week\",\n        \"weekday_weekend\",\n        \"temperature\",\n        \"observed\",\n        \"predicted\",\n        \"predicted_unc\",\n        \"heating_load\",\n        \"cooling_load\",\n        \"model_split\",\n        \"model_type\",\n    ]\n    metered_savings = (results.predicted - results.observed).sum()\n    assert np.isclose(metered_savings, 1785.8, rtol=1e-2)\n\n\n@pytest.fixture\ndef reporting_meter_data_billing_wrong_timestamp():\n    index = pd.date_range(\"2003-01-01\", freq=\"MS\", periods=13, tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\ndef test_metered_savings_cdd_hdd_billing_reporting_data_wrong_timestamp(\n    reporting_meter_data_billing_wrong_timestamp,\n    reporting_temperature_data,\n):\n    with pytest.raises(ValueError):\n        BillingReportingData.from_series(\n            reporting_meter_data_billing_wrong_timestamp,\n            reporting_temperature_data,\n            is_electricity_data=True,\n        )\n\n\ndef test_modeled_savings_cdd_hdd_daily(\n    baseline_model_daily,\n    reporting_model_daily,\n    reporting_meter_data_daily,\n    reporting_temperature_data,\n):\n    reporting_data = DailyReportingData.from_series(\n        reporting_meter_data_daily, reporting_temperature_data, is_electricity_data=True\n    )\n    baseline_model_result = baseline_model_daily.predict(reporting_data)\n    reporting_model_result = reporting_model_daily.predict(reporting_data)\n    modeled_savings = (\n        baseline_model_result[\"predicted\"] - reporting_model_result[\"predicted\"]\n    )\n    assert np.isclose(modeled_savings.sum(), 177.02, rtol=0.1)\n\n\n# TODO move to dataclass testing\ndef test_modeled_savings_daily_empty_temperature_data(\n    baseline_model_daily, reporting_model_daily\n):\n    index = pd.DatetimeIndex([], tz=\"UTC\", name=\"dt\", freq=\"h\")\n    temperature_data = pd.Series([], index=index).to_frame()\n\n    with pytest.raises(ValueError):\n        reporting = DailyReportingData(temperature_data, True)\n\n\n@pytest.fixture\ndef baseline_model_hourly(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    preliminary_hourly_design_matrix = create_caltrack_hourly_preliminary_design_matrix(\n        baseline_meter_data, temperature_data\n    )\n    segmentation = segment_time_series(\n        preliminary_hourly_design_matrix.index, \"three_month_weighted\"\n    )\n    occupancy_lookup = estimate_hour_of_week_occupancy(\n        preliminary_hourly_design_matrix, segmentation=segmentation\n    )\n    occupied_temperature_bins, unoccupied_temperature_bins = fit_temperature_bins(\n        preliminary_hourly_design_matrix,\n        segmentation=segmentation,\n        occupancy_lookup=occupancy_lookup,\n    )\n    design_matrices = create_caltrack_hourly_segmented_design_matrices(\n        preliminary_hourly_design_matrix,\n        segmentation,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n    segmented_model = fit_caltrack_hourly_model(\n        design_matrices,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=\"three_month_weighted\",\n    )\n    return segmented_model\n\n\n@pytest.fixture\ndef reporting_model_hourly(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    blackout_end_date = il_electricity_cdd_hdd_hourly[\"blackout_end_date\"]\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date\n    )\n    preliminary_hourly_design_matrix = create_caltrack_hourly_preliminary_design_matrix(\n        reporting_meter_data, temperature_data\n    )\n    segmentation = segment_time_series(\n        preliminary_hourly_design_matrix.index, \"three_month_weighted\"\n    )\n    occupancy_lookup = estimate_hour_of_week_occupancy(\n        preliminary_hourly_design_matrix, segmentation=segmentation\n    )\n    occupied_temperature_bins, unoccupied_temperature_bins = fit_temperature_bins(\n        preliminary_hourly_design_matrix,\n        segmentation=segmentation,\n        occupancy_lookup=occupancy_lookup,\n    )\n    design_matrices = create_caltrack_hourly_segmented_design_matrices(\n        preliminary_hourly_design_matrix,\n        segmentation,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n    segmented_model = fit_caltrack_hourly_model(\n        design_matrices,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=\"three_month_weighted\",\n    )\n    return segmented_model\n\n\n@pytest.fixture\ndef reporting_meter_data_hourly():\n    index = pd.date_range(\"2011-01-01\", freq=\"D\", periods=60, tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index).asfreq(\"h\").ffill()\n\n\ndef test_metered_savings_cdd_hdd_hourly(\n    baseline_model_hourly, reporting_meter_data_hourly, reporting_temperature_data\n):\n    results, error_bands = metered_savings(\n        baseline_model_hourly, reporting_meter_data_hourly, reporting_temperature_data\n    )\n    assert list(results.columns) == [\n        \"reporting_observed\",\n        \"counterfactual_usage\",\n        \"metered_savings\",\n    ]\n    assert round(results.metered_savings.sum(), 2) == -403.7\n    assert error_bands is None\n\n\ndef test_modeled_savings_cdd_hdd_hourly(\n    baseline_model_hourly,\n    reporting_model_hourly,\n    reporting_meter_data_hourly,\n    reporting_temperature_data,\n):\n    # using reporting data for convenience, but intention is to use normal data\n    results, error_bands = modeled_savings(\n        baseline_model_hourly,\n        reporting_model_hourly,\n        reporting_meter_data_hourly.index,\n        reporting_temperature_data,\n    )\n    assert list(results.columns) == [\n        \"modeled_baseline_usage\",\n        \"modeled_reporting_usage\",\n        \"modeled_savings\",\n    ]\n    assert round(results.modeled_savings.sum(), 1) == 55.3\n    assert error_bands is None\n\n\n@pytest.fixture\ndef normal_year_temperature_data():\n    index = pd.date_range(\"2015-01-01\", freq=\"D\", periods=365, tz=\"UTC\")\n    np.random.seed(0)\n    return pd.Series(np.random.rand(365) * 30 + 45, index=index).asfreq(\"h\").ffill()\n\n\ndef test_modeled_savings_cdd_hdd_billing(\n    baseline_model_billing, reporting_model_billing, normal_year_temperature_data\n):\n    # results, error_bands = modeled_savings(\n    #     baseline_model_billing,\n    #     reporting_model_billing,\n    #     pd.date_range(\"2015-01-01\", freq=\"D\", periods=365, tz=\"UTC\"),\n    #     normal_year_temperature_data,\n    # )\n    meter_data = meter_data = pd.DataFrame(\n        {\"observed\": np.nan}, index=normal_year_temperature_data.index\n    )\n    results = baseline_model_billing.predict(\n        BillingReportingData.from_series(\n            meter_data, normal_year_temperature_data, is_electricity_data=True\n        )\n    )\n\n    assert list(results.columns) == [\n        \"season\",\n        \"day_of_week\",\n        \"weekday_weekend\",\n        \"temperature\",\n        \"predicted\",\n        \"predicted_unc\",\n        \"heating_load\",\n        \"cooling_load\",\n        \"model_split\",\n        \"model_type\",\n    ]\n    predicted_sum = results.predicted.sum()\n    assert np.isclose(predicted_sum, 8245.37, rtol=1e-2)\n\n\n@pytest.fixture\ndef reporting_meter_data_billing_not_aligned():\n    index = pd.date_range(\"2001-01-01\", freq=\"MS\", periods=13, tz=\"UTC\")\n    return pd.DataFrame({\"value\": None}, index=index)\n\n\ndef test_metered_savings_not_aligned_reporting_data(\n    reporting_meter_data_billing_not_aligned,\n    reporting_temperature_data,\n):\n    with pytest.raises(ValueError):\n        BillingReportingData.from_series(\n            reporting_meter_data_billing_not_aligned,\n            reporting_temperature_data,\n            is_electricity_data=True,\n        )\n\n\n@pytest.fixture\ndef baseline_model_billing_single_record(il_electricity_cdd_hdd_billing_monthly):\n    # using two records until bounds failure is fixed\n    baseline_meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"][-3:]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_billing_monthly[\"blackout_start_date\"]\n    baseline_data = create_caltrack_billing_design_matrix(\n        baseline_meter_data, temperature_data\n    ).rename(columns={\"meter_value\": \"observed\", \"temperature_mean\": \"temperature\"})\n    model_results = BillingModel().fit(\n        BillingBaselineData(baseline_data, is_electricity_data=True),\n        ignore_disqualification=True,\n    )\n    return model_results\n\n\ndef test_metered_savings_model_single_record(\n    baseline_model_billing_single_record,\n    reporting_meter_data_billing,\n    reporting_temperature_data,\n):\n    # results, error_bands = metered_savings(\n    #     baseline_model_billing_single_record,\n    #     reporting_meter_data_billing,\n    #     reporting_temperature_data,\n    #     billing_data=True,\n    # )\n\n    results = baseline_model_billing_single_record.predict(\n        BillingReportingData.from_series(\n            reporting_meter_data_billing,\n            reporting_temperature_data,\n            is_electricity_data=True,\n        ),\n        ignore_disqualification=True,\n    )\n    assert list(results.columns) == [\n        \"season\",\n        \"day_of_week\",\n        \"weekday_weekend\",\n        \"temperature\",\n        \"observed\",\n        \"predicted\",\n        \"predicted_unc\",\n        \"heating_load\",\n        \"cooling_load\",\n        \"model_split\",\n        \"model_type\",\n    ]\n    metered_savings = (results.predicted - results.observed).sum()\n    assert np.isclose(metered_savings, 1436.72, rtol=1e-3)\n\n\n@pytest.fixture\ndef baseline_model_hourly_single_segment(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date\n    )\n    preliminary_hourly_design_matrix = create_caltrack_hourly_preliminary_design_matrix(\n        baseline_meter_data, temperature_data\n    )\n    segmentation = segment_time_series(\n        preliminary_hourly_design_matrix.index, \"three_month_weighted\"\n    )\n    occupancy_lookup = estimate_hour_of_week_occupancy(\n        preliminary_hourly_design_matrix, segmentation=segmentation\n    )\n    occupied_temperature_bins, unoccupied_temperature_bins = fit_temperature_bins(\n        preliminary_hourly_design_matrix,\n        segmentation=segmentation,\n        occupancy_lookup=occupancy_lookup,\n    )\n    design_matrices = create_caltrack_hourly_segmented_design_matrices(\n        preliminary_hourly_design_matrix,\n        segmentation,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n    )\n    segmented_model = fit_caltrack_hourly_model(\n        design_matrices,\n        occupancy_lookup,\n        occupied_temperature_bins,\n        unoccupied_temperature_bins,\n        segment_type=\"three_month_weighted\",\n    )\n    return segmented_model\n"
  },
  {
    "path": "tests/test_exceptions.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter.common.exceptions import (\n    EEMeterError,\n    NoBaselineDataError,\n    NoReportingDataError,\n    MissingModelParameterError,\n    UnrecognizedModelTypeError,\n)\n\nimport pytest\n\n\ndef test_eemeter_error():\n    with pytest.raises(EEMeterError):\n        raise EEMeterError\n\n\ndef test_no_baseline_data_error():\n    with pytest.raises(NoBaselineDataError):\n        raise NoBaselineDataError\n    assert isinstance(NoBaselineDataError(), EEMeterError)\n\n\ndef test_no_reporting_data_error():\n    with pytest.raises(NoReportingDataError):\n        raise NoReportingDataError\n    assert isinstance(NoReportingDataError(), EEMeterError)\n\n\ndef test_missing_model_parameter_error():\n    with pytest.raises(MissingModelParameterError):\n        raise MissingModelParameterError\n    assert isinstance(MissingModelParameterError(), EEMeterError)\n\n\ndef test_unrecognized_model_type_error():\n    with pytest.raises(UnrecognizedModelTypeError):\n        raise UnrecognizedModelTypeError\n    assert isinstance(UnrecognizedModelTypeError(), EEMeterError)\n"
  },
  {
    "path": "tests/test_features.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.eemeter.common.features import (\n    compute_occupancy_feature,\n    compute_temperature_features,\n    compute_temperature_bin_features,\n    compute_time_features,\n    compute_usage_per_day_feature,\n    estimate_hour_of_week_occupancy,\n    get_missing_hours_of_week_warning,\n    fit_temperature_bins,\n    merge_features,\n)\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import segment_time_series\n\n\ndef test_compute_temperature_features_no_freq_index(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    temperature_data.index.freq = None\n    with pytest.raises(ValueError):\n        compute_temperature_features(meter_data.index, temperature_data)\n\n\ndef test_compute_temperature_features_no_meter_data_tz(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    meter_data.index = meter_data.index.tz_localize(None)\n    with pytest.raises(ValueError):\n        compute_temperature_features(meter_data.index, temperature_data)\n\n\ndef test_compute_temperature_features_no_temp_data_tz(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    temperature_data = temperature_data.tz_localize(None)\n    with pytest.raises(ValueError):\n        compute_temperature_features(meter_data.index, temperature_data)\n\n\ndef test_compute_temperature_features_hourly_temp_mean(il_electricity_cdd_hdd_hourly):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert list(sorted(df.columns)) == [\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n        \"temperature_mean\",\n    ]\n    assert df.shape == (2952, 3)\n\n    assert round(df.temperature_mean.mean()) == 62.0\n\n\ndef test_compute_temperature_features_hourly_hourly_degree_days(\n    il_electricity_cdd_hdd_hourly, snapshot\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n    )\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    assert df.shape == (2952, 6)\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_hourly_hourly_degree_days_use_mean_false(\n    il_electricity_cdd_hdd_hourly, snapshot\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n        use_mean_daily_values=False,\n    )\n    assert df.shape == (2952, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_hourly_daily_degree_days_fail(\n    il_electricity_cdd_hdd_hourly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"daily\",\n        )\n\n\ndef test_compute_temperature_features_hourly_daily_missing_explicit_freq(\n    il_electricity_cdd_hdd_hourly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n\n    meter_data.index.freq = None\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"daily\",\n        )\n\n\ndef test_compute_temperature_features_hourly_bad_degree_days(\n    il_electricity_cdd_hdd_hourly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"UNKNOWN\",\n        )\n\n\ndef test_compute_temperature_features_hourly_data_quality(\n    il_electricity_cdd_hdd_hourly,\n):\n    # pick a slice with both hdd and cdd\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"][\"2016-03-01\":\"2016-07-01\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"][\n        \"2016-03-01\":\"2016-07-01\"\n    ]\n\n    df = compute_temperature_features(\n        meter_data.index, temperature_data, temperature_mean=False, data_quality=True\n    )\n    assert df.shape == (2952, 4)\n    assert list(sorted(df.columns)) == [\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(df.temperature_not_null.mean(), 2) == 1.0\n    assert round(df.temperature_null.mean(), 2) == 0.0\n\n\ndef test_compute_temperature_features_daily_temp_mean(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert df.shape == (810, 3)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n    ]\n\n    assert round(df.temperature_mean.mean()) == 55.0\n\n\ndef test_compute_temperature_features_daily_daily_degree_days(\n    il_electricity_cdd_hdd_daily, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"daily\",\n    )\n    assert df.shape == (810, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_days_kept.mean(), 2),\n            round(df.n_days_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_daily_daily_degree_days_use_mean_false(\n    il_electricity_cdd_hdd_daily, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"daily\",\n        use_mean_daily_values=False,\n    )\n    assert df.shape == (810, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_days_kept.mean(), 2),\n            round(df.n_days_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_daily_hourly_degree_days(\n    il_electricity_cdd_hdd_daily, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n    )\n    assert df.shape == (810, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_daily_hourly_degree_days_use_mean_false(\n    il_electricity_cdd_hdd_daily, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n        use_mean_daily_values=False,\n    )\n    assert df.shape == (810, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_daily_bad_degree_days(\n    il_electricity_cdd_hdd_daily,\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"UNKNOWN\",\n        )\n\n\ndef test_compute_temperature_features_daily_data_quality(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index, temperature_data, temperature_mean=False, data_quality=True\n    )\n    assert df.shape == (810, 4)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(df.temperature_not_null.mean(), 2) == 23.99\n    assert round(df.temperature_null.mean(), 2) == 0.00\n\n\ndef test_compute_temperature_features_billing_monthly_temp_mean(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert df.shape == (27, 3)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n    ]\n    assert round(df.temperature_mean.mean()) == 55.0\n\n\ndef test_compute_temperature_features_billing_monthly_daily_degree_days(\n    il_electricity_cdd_hdd_billing_monthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"daily\",\n    )\n    assert df.shape == (27, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_days_kept.mean(), 2),\n            round(df.n_days_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_monthly_daily_degree_days_use_mean_false(\n    il_electricity_cdd_hdd_billing_monthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"daily\",\n        use_mean_daily_values=False,\n    )\n    assert df.shape == (27, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_days_kept.mean(), 2),\n            round(df.n_days_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_monthly_hourly_degree_days(\n    il_electricity_cdd_hdd_billing_monthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n    )\n    assert df.shape == (27, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_monthly_hourly_degree_days_use_mean_false(\n    il_electricity_cdd_hdd_billing_monthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n        use_mean_daily_values=False,\n    )\n    assert df.shape == (27, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_monthly_bad_degree_day_method(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"UNKNOWN\",\n        )\n\n\ndef test_compute_temperature_features_billing_monthly_data_quality(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index, temperature_data, temperature_mean=False, data_quality=True\n    )\n    assert df.shape == (27, 4)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(df.temperature_not_null.mean(), 2) == 729.23\n    assert round(df.temperature_null.mean(), 2) == 0.0\n\n\ndef test_compute_temperature_features_billing_bimonthly_temp_mean(\n    il_electricity_cdd_hdd_billing_bimonthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_bimonthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_bimonthly[\"temperature_data\"]\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert df.shape == (14, 3)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n    ]\n    assert round(df.temperature_mean.mean()) == 55.0\n\n\ndef test_compute_temperature_features_billing_bimonthly_daily_degree_days(\n    il_electricity_cdd_hdd_billing_bimonthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_bimonthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_bimonthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"daily\",\n    )\n    assert df.shape == (14, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_days_dropped\",\n        \"n_days_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_days_kept.mean(), 2),\n            round(df.n_days_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_bimonthly_hourly_degree_days(\n    il_electricity_cdd_hdd_billing_bimonthly, snapshot\n):\n    meter_data = il_electricity_cdd_hdd_billing_bimonthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_bimonthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[60, 61],\n        cooling_balance_points=[65, 66],\n        temperature_mean=False,\n        degree_day_method=\"hourly\",\n    )\n    assert df.shape == (14, 6)\n    assert list(sorted(df.columns)) == [\n        \"cdd_65\",\n        \"cdd_66\",\n        \"hdd_60\",\n        \"hdd_61\",\n        \"n_hours_dropped\",\n        \"n_hours_kept\",\n    ]\n    snapshot.assert_match(\n        [\n            round(df.hdd_60.mean(), 2),\n            round(df.hdd_61.mean(), 2),\n            round(df.cdd_65.mean(), 2),\n            round(df.cdd_66.mean(), 2),\n            round(df.n_hours_kept.mean(), 2),\n            round(df.n_hours_dropped.mean(), 2),\n        ],\n        \"values\",\n    )\n\n\ndef test_compute_temperature_features_billing_bimonthly_bad_degree_days(\n    il_electricity_cdd_hdd_billing_bimonthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_bimonthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_bimonthly[\"temperature_data\"]\n    with pytest.raises(ValueError):\n        compute_temperature_features(\n            meter_data.index,\n            temperature_data,\n            heating_balance_points=[60, 61],\n            cooling_balance_points=[65, 66],\n            degree_day_method=\"UNKNOWN\",\n        )\n\n\ndef test_compute_temperature_features_billing_bimonthly_data_quality(\n    il_electricity_cdd_hdd_billing_bimonthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_bimonthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_bimonthly[\"temperature_data\"]\n    df = compute_temperature_features(\n        meter_data.index, temperature_data, temperature_mean=False, data_quality=True\n    )\n    assert df.shape == (14, 4)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_not_null\",\n        \"temperature_null\",\n    ]\n    assert round(df.temperature_not_null.mean(), 2) == 1478.77\n    assert round(df.temperature_null.mean(), 2) == 0.0\n\n\ndef test_compute_temperature_features_shorter_temperature_data(\n    il_electricity_cdd_hdd_daily,\n):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n\n    # drop some data\n    temperature_data = temperature_data[:-200]\n\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert df.shape == (810, 3)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n    ]\n    assert round(df.temperature_mean.sum()) == 43958.0\n\n\ndef test_compute_temperature_features_shorter_meter_data(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_daily[\"temperature_data\"]\n\n    # drop some data\n    meter_data = meter_data[:-10]\n\n    df = compute_temperature_features(meter_data.index, temperature_data)\n    assert df.shape == (800, 3)\n    assert list(sorted(df.columns)) == [\n        \"n_days_dropped\",\n        \"n_days_kept\",\n        \"temperature_mean\",\n    ]\n    assert round(df.temperature_mean.sum()) == 43904.0\n    # ensure last row is NaN'ed\n    assert pd.isnull(df.iloc[-1].n_days_kept)\n\n\ndef test_compute_temperature_features_with_duplicated_index(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n\n    # these are specifically formed to give a less readable error if\n    # duplicates are not caught\n    meter_data = pd.concat([meter_data, meter_data]).sort_index()\n    temperature_data = temperature_data.iloc[8000:]\n\n    with pytest.raises(ValueError) as excinfo:\n        compute_temperature_features(meter_data.index, temperature_data)\n    assert str(excinfo.value) == \"Duplicates found in input meter trace index.\"\n\n\ndef test_compute_temperature_features_empty_temperature_data():\n    index = pd.DatetimeIndex([], tz=\"UTC\", name=\"dt\", freq=\"h\")\n    temperature_data = pd.Series({\"value\": []}, index=index).astype(float)\n    result_index = temperature_data.resample(\"D\").sum().index\n    meter_data_hack = pd.DataFrame({\"value\": 0}, index=result_index)\n\n    with pytest.raises(ValueError):\n        df = compute_temperature_features(\n            meter_data_hack.index,\n            temperature_data,\n            heating_balance_points=[65],\n            cooling_balance_points=[65],\n            degree_day_method=\"daily\",\n            use_mean_daily_values=False,\n        )\n\n\ndef test_compute_temperature_features_empty_meter_data():\n    index = pd.DatetimeIndex([], tz=\"UTC\", name=\"dt\", freq=\"h\")\n    temperature_data = pd.Series({\"value\": 0}, index=index)\n    result_index = temperature_data.resample(\"D\").sum().index\n    meter_data_hack = pd.DataFrame({\"value\": []}, index=result_index)\n    meter_data_hack.index.freq = None\n\n    with pytest.raises(ValueError):\n        df = compute_temperature_features(\n            meter_data_hack.index,\n            temperature_data,\n            heating_balance_points=[65],\n            cooling_balance_points=[65],\n            degree_day_method=\"daily\",\n            use_mean_daily_values=False,\n        )\n\n\ndef test_merge_features():\n    index = pd.date_range(\"2017-01-01\", periods=100, freq=\"h\", tz=\"UTC\")\n    features = merge_features(\n        [\n            pd.Series(1, index=index, name=\"a\"),\n            pd.DataFrame({\"b\": 2}, index=index),\n            pd.DataFrame({\"c\": 3, \"d\": 4}, index=index),\n        ]\n    )\n    assert list(features.columns) == [\"a\", \"b\", \"c\", \"d\"]\n    assert features.shape == (100, 4)\n    assert features.sum().sum() == 1000\n    assert features.a.sum() == 100\n    assert features.b.sum() == 200\n    assert features.c.sum() == 300\n    assert features.d.sum() == 400\n    assert features.index[0] == index[0]\n    assert features.index[-1] == index[-1]\n\n\ndef test_merge_features_empty_raises():\n    with pytest.raises(ValueError):\n        features = merge_features([])\n\n\n@pytest.fixture\ndef meter_data_hourly():\n    index = pd.date_range(\"2017-01-01\", periods=100, freq=\"h\", tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\ndef test_compute_usage_per_day_feature_hourly(meter_data_hourly):\n    usage_per_day = compute_usage_per_day_feature(meter_data_hourly)\n    assert usage_per_day.name == \"usage_per_day\"\n    assert usage_per_day[\"2017-01-01T00:00:00Z\"] == 24\n    assert usage_per_day.sum() == 2376.0\n\n\ndef test_compute_usage_per_day_feature_hourly_series_name(meter_data_hourly):\n    usage_per_day = compute_usage_per_day_feature(\n        meter_data_hourly, series_name=\"meter_value\"\n    )\n    assert usage_per_day.name == \"meter_value\"\n\n\n@pytest.fixture\ndef meter_data_daily():\n    index = pd.date_range(\"2017-01-01\", periods=100, freq=\"D\", tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\ndef test_compute_usage_per_day_feature_daily(meter_data_daily):\n    usage_per_day = compute_usage_per_day_feature(meter_data_daily)\n    assert usage_per_day[\"2017-01-01T00:00:00Z\"] == 1\n    assert usage_per_day.sum() == 99.0\n\n\n@pytest.fixture\ndef meter_data_billing():\n    index = pd.date_range(\"2017-01-01\", periods=100, freq=\"MS\", tz=\"UTC\")\n    return pd.DataFrame({\"value\": 1}, index=index)\n\n\ndef test_compute_usage_per_day_feature_billing(meter_data_billing):\n    usage_per_day = compute_usage_per_day_feature(meter_data_billing)\n    assert usage_per_day[\"2017-01-01T00:00:00Z\"] == 1.0 / 31\n    assert usage_per_day.sum().round(3) == 3.257\n\n\n@pytest.fixture\ndef complete_hour_of_week_feature():\n    index = pd.date_range(\"2017-01-01\", periods=168, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index, hour_of_week=True)\n    hour_of_week_feature = time_features.hour_of_week\n    return hour_of_week_feature\n\n\ndef test_get_missing_hours_of_week_warning_ok(complete_hour_of_week_feature):\n    warning = get_missing_hours_of_week_warning(complete_hour_of_week_feature)\n    assert warning is None\n\n\n@pytest.fixture\ndef partial_hour_of_week_feature():\n    index = pd.date_range(\"2017-01-01\", periods=84, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index, hour_of_week=True)\n    hour_of_week_feature = time_features.hour_of_week\n    return hour_of_week_feature\n\n\ndef test_get_missing_hours_of_week_warning_triggered(partial_hour_of_week_feature):\n    warning = get_missing_hours_of_week_warning(partial_hour_of_week_feature)\n    assert warning.qualified_name is not None\n    assert warning.description is not None\n    assert warning.data[\"missing_hours_of_week\"] == list(range(60, 144))\n\n\ndef test_compute_time_features_bad_freq():\n    index = pd.date_range(\"2017-01-01\", periods=168, freq=\"D\", tz=\"UTC\")\n    with pytest.raises(ValueError):\n        compute_time_features(index)\n\n\ndef test_compute_time_features_all():\n    index = pd.date_range(\"2017-01-01\", periods=168, freq=\"h\", tz=\"UTC\")\n    features = compute_time_features(index)\n    assert list(features.columns) == [\"day_of_week\", \"hour_of_day\", \"hour_of_week\"]\n    assert features.shape == (168, 3)\n    assert features.astype(float).sum().sum() == 16464.0\n    with pytest.raises(TypeError):  # categoricals\n        features.day_of_week.sum()\n    with pytest.raises(TypeError):\n        features.hour_of_day.sum()\n    with pytest.raises(TypeError):\n        features.hour_of_week.sum()\n    assert features.day_of_week.astype(\"float\").sum() == sum(range(7)) * 24\n    assert features.hour_of_day.astype(\"float\").sum() == sum(range(24)) * 7\n    assert features.hour_of_week.astype(\"float\").sum() == sum(range(168))\n    assert features.index[0] == index[0]\n    assert features.index[-1] == index[-1]\n\n\ndef test_compute_time_features_none():\n    index = pd.date_range(\"2017-01-01\", periods=168, freq=\"h\", tz=\"UTC\")\n    with pytest.raises(ValueError):\n        compute_time_features(\n            index, hour_of_week=False, day_of_week=False, hour_of_day=False\n        )\n\n\n@pytest.fixture\ndef occupancy_precursor(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    time_features = compute_time_features(meter_data.index)\n    temperature_features = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[50],\n        cooling_balance_points=[65],\n        degree_day_method=\"hourly\",\n    )\n    return merge_features(\n        [meter_data.value.to_frame(\"meter_value\"), temperature_features, time_features]\n    )\n\n\ndef test_estimate_hour_of_week_occupancy_no_segmentation(occupancy_precursor):\n    occupancy = estimate_hour_of_week_occupancy(occupancy_precursor)\n    assert list(occupancy.columns) == [\"occupancy\"]\n    assert occupancy.shape == (168, 1)\n    assert occupancy.sum().sum() == 0\n\n\n@pytest.fixture\ndef one_month_segmentation(occupancy_precursor):\n    return segment_time_series(occupancy_precursor.index, segment_type=\"one_month\")\n\n\ndef test_estimate_hour_of_week_occupancy_one_month_segmentation(\n    occupancy_precursor, one_month_segmentation\n):\n    occupancy = estimate_hour_of_week_occupancy(\n        occupancy_precursor, segmentation=one_month_segmentation\n    )\n    assert list(occupancy.columns) == [\n        \"jan\",\n        \"feb\",\n        \"mar\",\n        \"apr\",\n        \"may\",\n        \"jun\",\n        \"jul\",\n        \"aug\",\n        \"sep\",\n        \"oct\",\n        \"nov\",\n        \"dec\",\n    ]\n    assert occupancy.shape == (168, 12)\n    assert occupancy.sum().sum() == 84.0\n\n\n@pytest.fixture\ndef temperature_means():\n    index = pd.date_range(\"2017-01-01\", periods=2000, freq=\"h\", tz=\"UTC\")\n    return pd.DataFrame({\"temperature_mean\": [10, 35, 55, 80, 100] * 400}, index=index)\n\n\ndef test_fit_temperature_bins_no_segmentation(temperature_means):\n    bins = fit_temperature_bins(\n        temperature_means, segmentation=None, occupancy_lookup=None\n    )\n    assert list(bins.columns) == [\"keep_bin_endpoint\"]\n    assert bins.shape == (6, 1)\n    assert bins.sum().sum() == 4\n\n\n@pytest.fixture\ndef occupancy_lookup_no_segmentation(occupancy_precursor):\n    occupancy = estimate_hour_of_week_occupancy(occupancy_precursor)\n    return occupancy\n\n\ndef test_fit_temperature_bins_no_segmentation_with_occupancy(\n    temperature_means, occupancy_lookup_no_segmentation\n):\n    occupied_bins, unoccupied_bins = fit_temperature_bins(\n        temperature_means,\n        segmentation=None,\n        occupancy_lookup=occupancy_lookup_no_segmentation,\n    )\n    assert list(occupied_bins.columns) == [\"keep_bin_endpoint\"]\n    assert occupied_bins.shape == (6, 1)\n    assert occupied_bins.sum().sum() == 0\n\n    assert list(unoccupied_bins.columns) == [\"keep_bin_endpoint\"]\n    assert unoccupied_bins.shape == (6, 1)\n    assert unoccupied_bins.sum().sum() == 4\n\n\ndef test_fit_temperature_bins_one_month_segmentation(\n    temperature_means, one_month_segmentation\n):\n    bins = fit_temperature_bins(temperature_means, segmentation=one_month_segmentation)\n    assert list(bins.columns) == [\n        \"jan\",\n        \"feb\",\n        \"mar\",\n        \"apr\",\n        \"may\",\n        \"jun\",\n        \"jul\",\n        \"aug\",\n        \"sep\",\n        \"oct\",\n        \"nov\",\n        \"dec\",\n    ]\n    assert bins.shape == (6, 12)\n    assert bins.sum().sum() == 12\n\n\n@pytest.fixture\ndef occupancy_lookup_one_month_segmentation(\n    occupancy_precursor, one_month_segmentation\n):\n    occupancy_lookup = estimate_hour_of_week_occupancy(\n        occupancy_precursor, segmentation=one_month_segmentation\n    )\n    return occupancy_lookup\n\n\ndef test_fit_temperature_bins_with_occupancy_lookup(\n    temperature_means, one_month_segmentation, occupancy_lookup_one_month_segmentation\n):\n    occupied_bins, unoccupied_bins = fit_temperature_bins(\n        temperature_means,\n        segmentation=one_month_segmentation,\n        occupancy_lookup=occupancy_lookup_one_month_segmentation,\n    )\n    assert list(occupied_bins.columns) == [\n        \"jan\",\n        \"feb\",\n        \"mar\",\n        \"apr\",\n        \"may\",\n        \"jun\",\n        \"jul\",\n        \"aug\",\n        \"sep\",\n        \"oct\",\n        \"nov\",\n        \"dec\",\n    ]\n    assert occupied_bins.shape == (6, 12)\n    assert occupied_bins.sum().sum() == 0\n\n    assert list(unoccupied_bins.columns) == [\n        \"jan\",\n        \"feb\",\n        \"mar\",\n        \"apr\",\n        \"may\",\n        \"jun\",\n        \"jul\",\n        \"aug\",\n        \"sep\",\n        \"oct\",\n        \"nov\",\n        \"dec\",\n    ]\n    assert unoccupied_bins.shape == (6, 12)\n    assert unoccupied_bins.sum().sum() == 12\n\n\ndef test_fit_temperature_bins_empty(temperature_means):\n    bins = fit_temperature_bins(temperature_means.iloc[:0])\n    assert list(bins.columns) == [\"keep_bin_endpoint\"]\n    assert bins.shape == (6, 1)\n    assert bins.sum().sum() == 0\n\n\ndef test_compute_temperature_bin_features(temperature_means):\n    temps = temperature_means.temperature_mean\n    bin_features = compute_temperature_bin_features(temps, [25, 75])\n    assert list(bin_features.columns) == [\"bin_0\", \"bin_1\", \"bin_2\"]\n    assert bin_features.shape == (2000, 3)\n    assert bin_features.sum().sum() == 112000.0\n\n\n@pytest.fixture\ndef even_occupancy():\n    return pd.Series([i % 2 == 0 for i in range(168)], index=pd.Categorical(range(168)))\n\n\ndef test_compute_occupancy_feature(even_occupancy):\n    index = pd.date_range(\"2017-01-01\", periods=1000, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index, hour_of_week=True)\n    hour_of_week = time_features.hour_of_week\n    occupancy = compute_occupancy_feature(hour_of_week, even_occupancy)\n    assert occupancy.name == \"occupancy\"\n    assert occupancy.shape == (1000,)\n    assert occupancy.sum().sum() == 500\n\n\ndef test_compute_occupancy_feature_with_nans(even_occupancy):\n    \"\"\"If there are less than 168 periods, the NaN at the end causes problems\"\"\"\n    index = pd.date_range(\"2017-01-01\", periods=100, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index, hour_of_week=True)\n    time_features.iloc[-1, time_features.columns.get_loc(\"hour_of_week\")] = np.nan\n    hour_of_week = time_features.hour_of_week\n    #  comment out line below to see the error from not dropping na when\n    # calculationg _add_weights when there are less than 168 periods.\n\n    # TODO (ssuffian): Refactor so get_missing_hours_warnings propogates.\n    # right now, it will error if the dropna below isn't used.\n    hour_of_week.dropna(inplace=True)\n    occupancy = compute_occupancy_feature(hour_of_week, even_occupancy)\n    assert occupancy.name == \"occupancy\"\n    assert occupancy.shape == (99,)\n    assert occupancy.sum().sum() == 50\n\n\n@pytest.fixture\ndef occupancy_precursor_only_nan(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    meter_data = meter_data[\"2017-01-04\":\"2017-06-01\"].copy()\n    meter_data.iloc[-1] = np.nan\n    # Simulates a segment where there is only a single nan value\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n    time_features = compute_time_features(meter_data.index)\n    temperature_features = compute_temperature_features(\n        meter_data.index,\n        temperature_data,\n        heating_balance_points=[50],\n        cooling_balance_points=[65],\n        degree_day_method=\"hourly\",\n    )\n    return merge_features(\n        [meter_data.value.to_frame(\"meter_value\"), temperature_features, time_features]\n    )\n\n\n@pytest.fixture\ndef segmentation_only_nan(occupancy_precursor_only_nan):\n    return segment_time_series(\n        occupancy_precursor_only_nan.index, segment_type=\"three_month_weighted\"\n    )\n\n\ndef test_estimate_hour_of_week_occupancy_segmentation_only_nan(\n    occupancy_precursor_only_nan, segmentation_only_nan\n):\n    occupancy = estimate_hour_of_week_occupancy(\n        occupancy_precursor_only_nan, segmentation=segmentation_only_nan\n    )\n\n\ndef test_compute_occupancy_feature_hour_of_week_has_nan(even_occupancy):\n    index = pd.date_range(\"2017-01-01\", periods=72, freq=\"h\", tz=\"UTC\")\n    time_features = compute_time_features(index, hour_of_week=True)\n    hour_of_week = time_features.hour_of_week\n    hour_of_week.iloc[-1] = np.nan\n    occupancy = compute_occupancy_feature(hour_of_week, even_occupancy)\n    assert occupancy.name == \"occupancy\"\n    assert occupancy.shape == (72,)\n    assert occupancy.sum() == 36\n"
  },
  {
    "path": "tests/test_io.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport gzip\nfrom tempfile import TemporaryFile\nimport importlib.resources\nimport platform\n\nimport pandas as pd\nimport pytest\n\nfrom opendsm.eemeter.utilities.io import (\n    meter_data_from_csv,\n    meter_data_from_json,\n    meter_data_to_csv,\n    temperature_data_from_csv,\n    temperature_data_from_json,\n    temperature_data_to_csv,\n)\n\n\ndef test_meter_data_from_csv(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    meter_data_filename = meter_item[\"meter_data_filename\"]\n\n    fname = str(\n        importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n            meter_data_filename\n        )\n    )\n    with gzip.open(fname) as f:\n        meter_data = meter_data_from_csv(f)\n    assert meter_data.shape == (810, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_csv_gzipped(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    meter_data_filename = meter_item[\"meter_data_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        meter_data_filename\n    ).open(\"rb\") as f:\n        meter_data = meter_data_from_csv(f, gzipped=True)\n    assert meter_data.shape == (810, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_csv_with_tz(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    meter_data_filename = meter_item[\"meter_data_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        meter_data_filename\n    ).open(\"rb\") as f:\n        meter_data = meter_data_from_csv(f, gzipped=True, tz=\"US/Eastern\")\n    assert meter_data.shape == (810, 1)\n    assert str(meter_data.index.tz) == \"US/Eastern\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_csv_hourly_freq(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    meter_data_filename = meter_item[\"meter_data_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        meter_data_filename\n    ).open(\"rb\") as f:\n        meter_data = meter_data_from_csv(f, gzipped=True, freq=\"hourly\")\n    assert meter_data.shape == (19417, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq == \"h\"\n\n\ndef test_meter_data_from_csv_daily_freq(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    meter_data_filename = meter_item[\"meter_data_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        meter_data_filename\n    ).open(\"rb\") as f:\n        meter_data = meter_data_from_csv(f, gzipped=True, freq=\"daily\")\n    assert meter_data.shape == (810, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq == \"D\"\n\n\ndef test_meter_data_from_csv_custom_columns(sample_metadata):\n    with TemporaryFile() as f:\n        f.write(b\"start_custom,kWh\\n\" b\"2017-01-01T00:00:00,10\\n\")\n        f.seek(0)\n        meter_data = meter_data_from_csv(f, start_col=\"start_custom\", value_col=\"kWh\")\n    assert meter_data.shape == (1, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_none(sample_metadata):\n    data = None\n    meter_data = meter_data_from_json(data)\n    assert meter_data.shape == (0, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_orient_list(sample_metadata):\n    data = [[\"2017-01-01T00:00:00Z\", 11], [\"2017-01-02T00:00:00Z\", 10]]\n    meter_data = meter_data_from_json(data, orient=\"list\")\n    assert meter_data.shape == (2, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_orient_list_empty(sample_metadata):\n    data = []\n    meter_data = meter_data_from_json(data)\n    assert meter_data.shape == (0, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_orient_records(sample_metadata):\n    data = [\n        {\"start\": \"2017-01-01T00:00:00Z\", \"value\": 11},\n        {\"start\": \"2017-01-02T00:00:00Z\", \"value\": \"\"},\n        {\"start\": \"2017-01-03T00:00:00Z\", \"value\": 10},\n    ]\n    meter_data = meter_data_from_json(data, orient=\"records\")\n    assert meter_data.shape == (3, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_orient_records_empty(sample_metadata):\n    data = []\n    meter_data = meter_data_from_json(data, orient=\"records\")\n    assert meter_data.shape == (0, 1)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n\n\ndef test_meter_data_from_json_orient_records_with_estimated_true(sample_metadata):\n    data = [\n        {\"start\": \"2017-01-01T00:00:00Z\", \"value\": 11, \"estimated\": True},\n        {\"start\": \"2017-01-02T00:00:00Z\", \"value\": 10, \"estimated\": \"true\"},\n        {\"start\": \"2017-01-03T00:00:00Z\", \"value\": 10, \"estimated\": \"True\"},\n        {\"start\": \"2017-01-04T00:00:00Z\", \"value\": 10, \"estimated\": \"1\"},\n        {\"start\": \"2017-01-05T00:00:00Z\", \"value\": 10, \"estimated\": 1},\n    ]\n    meter_data = meter_data_from_json(data, orient=\"records\")\n    assert meter_data.shape == (5, 2)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n    assert meter_data.estimated.sum() == 5\n\n\ndef test_meter_data_from_json_orient_records_with_estimated_false(sample_metadata):\n    data = [\n        {\"start\": \"2017-01-01T00:00:00Z\", \"value\": 10, \"estimated\": False},\n        {\"start\": \"2017-01-02T00:00:00Z\", \"value\": 10, \"estimated\": \"false\"},\n        {\"start\": \"2017-01-03T00:00:00Z\", \"value\": 10, \"estimated\": \"False\"},\n        {\"start\": \"2017-01-04T00:00:00Z\", \"value\": 10, \"estimated\": \"\"},\n        {\"start\": \"2017-01-05T00:00:00Z\", \"value\": 10, \"estimated\": None},\n        {\"start\": \"2017-01-05T00:00:00Z\", \"value\": 10},\n    ]\n    meter_data = meter_data_from_json(data, orient=\"records\")\n    assert meter_data.shape == (6, 2)\n    assert str(meter_data.index.tz) == \"UTC\"\n    assert meter_data.index.freq is None\n    assert meter_data.estimated.sum() == 0\n\n\ndef test_meter_data_from_json_bad_orient(sample_metadata):\n    data = [[\"2017-01-01T00:00:00Z\", 11], [\"2017-01-02T00:00:00Z\", 10]]\n    with pytest.raises(ValueError):\n        meter_data_from_json(data, orient=\"NOT_ALLOWED\")\n\n\ndef test_meter_data_to_csv(sample_metadata):\n    df = pd.DataFrame(\n        {\"value\": [5]}, index=pd.to_datetime([\"2017-01-01T00:00:00Z\"], utc=True)\n    )\n    with TemporaryFile(\"w+\") as f:\n        meter_data_to_csv(df, f)\n        f.seek(0)\n        if platform.system() == \"Windows\":\n            assert f.read() == (\"start,value\\n\\n\" \"2017-01-01 00:00:00+00:00,5\\n\\n\")\n        else:\n            assert f.read() == (\"start,value\\n\" \"2017-01-01 00:00:00+00:00,5\\n\")\n\n\ndef test_temperature_data_from_csv(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    temperature_filename = meter_item[\"temperature_filename\"]\n\n    fname = str(\n        importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n            temperature_filename\n        )\n    )\n    with gzip.open(fname) as f:\n        temperature_data = temperature_data_from_csv(f)\n    assert temperature_data.shape == (19417,)\n    assert str(temperature_data.index.tz) == \"UTC\"\n    assert temperature_data.index.freq is None\n\n\ndef test_temperature_data_from_csv_gzipped(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    temperature_filename = meter_item[\"temperature_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        temperature_filename\n    ).open(\"rb\") as f:\n        temperature_data = temperature_data_from_csv(f, gzipped=True)\n    assert temperature_data.shape == (19417,)\n    assert str(temperature_data.index.tz) == \"UTC\"\n    assert temperature_data.index.freq is None\n\n\ndef test_temperature_data_from_csv_with_tz(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    temperature_filename = meter_item[\"temperature_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        temperature_filename\n    ).open(\"rb\") as f:\n        temperature_data = temperature_data_from_csv(f, gzipped=True, tz=\"US/Eastern\")\n    assert temperature_data.shape == (19417,)\n    assert str(temperature_data.index.tz) == \"US/Eastern\"\n    assert temperature_data.index.freq is None\n\n\ndef test_temperature_data_from_csv_hourly_freq(sample_metadata):\n    meter_item = sample_metadata[\"il-electricity-cdd-hdd-daily\"]\n    temperature_filename = meter_item[\"temperature_filename\"]\n\n    with importlib.resources.files(\"opendsm.eemeter.samples\").joinpath(\n        temperature_filename\n    ).open(\"rb\") as f:\n        temperature_data = temperature_data_from_csv(f, gzipped=True, freq=\"hourly\")\n    assert temperature_data.shape == (19417,)\n    assert str(temperature_data.index.tz) == \"UTC\"\n    assert temperature_data.index.freq == \"h\"\n\n\ndef test_temperature_data_from_csv_custom_columns(sample_metadata):\n    with TemporaryFile() as f:\n        f.write(b\"dt_custom,tempC\\n\" b\"2017-01-01T00:00:00,10\\n\")\n        f.seek(0)\n        temperature_data = temperature_data_from_csv(\n            f, date_col=\"dt_custom\", temp_col=\"tempC\"\n        )\n    assert temperature_data.shape == (1,)\n    assert str(temperature_data.index.tz) == \"UTC\"\n    assert temperature_data.index.freq is None\n\n\ndef test_temperature_data_from_json_orient_list(sample_metadata):\n    data = [[\"2017-01-01T00:00:00Z\", 11], [\"2017-01-02T00:00:00Z\", 10]]\n    temperature_data = temperature_data_from_json(data, orient=\"list\")\n    assert temperature_data.shape == (2,)\n    assert str(temperature_data.index.tz) == \"UTC\"\n    assert temperature_data.index.freq is None\n\n\ndef test_temperature_data_from_json_bad_orient(sample_metadata):\n    data = [[\"2017-01-01T00:00:00Z\", 11], [\"2017-01-02T00:00:00Z\", 10]]\n    with pytest.raises(ValueError):\n        temperature_data_from_json(data, orient=\"NOT_ALLOWED\")\n\n\ndef test_temperature_data_to_csv(sample_metadata):\n    series = pd.Series(10, index=pd.to_datetime([\"2017-01-01T00:00:00Z\"], utc=True))\n    with TemporaryFile(\"w+\") as f:\n        temperature_data_to_csv(series, f)\n        f.seek(0)\n        if platform.system() == \"Windows\":\n            assert f.read() == (\"dt,temperature\\n\\n\" \"2017-01-01 00:00:00+00:00,10\\n\\n\")\n        else:\n            assert f.read() == (\"dt,temperature\\n\" \"2017-01-01 00:00:00+00:00,10\\n\")\n"
  },
  {
    "path": "tests/test_json_serialization.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\n\nfrom opendsm.eemeter.samples import load_sample\nfrom opendsm.eemeter.common.transform import get_baseline_data, get_reporting_data\nfrom opendsm.eemeter import (\n    DailyBaselineData,\n    DailyReportingData,\n    DailyModel,\n    BillingBaselineData,\n    BillingReportingData,\n    BillingModel,\n    HourlyCaltrackModel,\n    HourlyCaltrackBaselineData,\n    HourlyCaltrackReportingData,\n)\n\n\ndef test_json_daily():\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-daily\"\n    )\n\n    blackout_start_date = sample_metadata[\"blackout_start_date\"]\n    blackout_end_date = sample_metadata[\"blackout_end_date\"]\n\n    # fit baseline model\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date, max_days=365\n    )\n    baseline_data = DailyBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n    baseline_model = DailyModel().fit(baseline_data, ignore_disqualification=True)\n\n    # predict on reporting year and calculate savings\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=365\n    )\n    reporting_data = DailyReportingData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n    metered_savings_dataframe = baseline_model.predict(reporting_data)\n    total_metered_savings = (\n        metered_savings_dataframe[\"observed\"] - metered_savings_dataframe[\"predicted\"]\n    ).sum()\n\n    # serialize, deserialize model\n    json_str = baseline_model.to_json()\n    loaded_model = DailyModel.from_json(json_str)\n\n    # compute metered savings from the loaded model\n    prediction_json = loaded_model.predict(reporting_data)\n    total_metered_savings_loaded = (\n        prediction_json[\"observed\"] - prediction_json[\"predicted\"]\n    ).sum()\n\n    # compare results\n    assert total_metered_savings == total_metered_savings_loaded\n\n\ndef test_json_billing():\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-billing_monthly\"\n    )\n\n    blackout_start_date = sample_metadata[\"blackout_start_date\"]\n    blackout_end_date = sample_metadata[\"blackout_end_date\"]\n\n    # fit baseline model\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date, max_days=365\n    )\n    baseline_data = BillingBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n    baseline_model = BillingModel().fit(baseline_data, ignore_disqualification=True)\n\n    # predict on reporting year and calculate savings\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=365\n    )\n    reporting_data = BillingReportingData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n    metered_savings_dataframe = baseline_model.predict(reporting_data)\n    total_metered_savings = (\n        metered_savings_dataframe[\"observed\"] - metered_savings_dataframe[\"predicted\"]\n    ).sum()\n\n    # serialize, deserialize model\n    json_str = baseline_model.to_json()\n    loaded_model = BillingModel.from_json(json_str)\n\n    # compute metered savings from the loaded model\n    prediction_json = loaded_model.predict(reporting_data)\n    total_metered_savings_loaded = (\n        prediction_json[\"observed\"] - prediction_json[\"predicted\"]\n    ).sum()\n\n    # compare results\n    assert total_metered_savings == total_metered_savings_loaded\n\n\ndef test_json_hourly_with_zeros():\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-hourly\"\n    )\n    meter_data[\"value\"] = 0\n    baseline = HourlyCaltrackBaselineData.from_series(\n        meter_data, temperature_data, is_electricity_data=True\n    )\n    assert baseline.df[\"observed\"].isnull().all()\n    reporting = HourlyCaltrackReportingData.from_series(\n        meter_data, temperature_data, is_electricity_data=True\n    )\n    assert reporting.df[\"observed\"].isnull().all()\n\n\ndef test_json_caltrack_hourly():\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-hourly\"\n    )\n\n    blackout_start_date = sample_metadata[\"blackout_start_date\"]\n    blackout_end_date = sample_metadata[\"blackout_end_date\"]\n\n    # get meter data suitable for fitting a baseline model\n    baseline_meter_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date, max_days=365\n    )\n    baseline = HourlyCaltrackBaselineData.from_series(\n        baseline_meter_data, temperature_data, is_electricity_data=True\n    )\n\n    # build a CalTRACK hourly model\n    baseline_model = HourlyCaltrackModel().fit(baseline)\n\n    # get a year of reporting period data\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=365\n    )\n    reporting = HourlyCaltrackReportingData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n\n    result1 = baseline_model.predict(reporting)\n\n    # serialize, deserialize\n    json_str = baseline_model.to_json()\n    m = HourlyCaltrackModel.from_json(json_str)\n\n    result2 = m.predict(reporting)\n\n    assert result1[\"predicted\"].sum() == result2[\"predicted\"].sum()\n\n    # Check that model metrics are properly serialized/serialized\n    assert (\n        baseline_model.model.totals_metrics[\"dec-jan-feb-weighted\"].observed_length\n        == m.model.totals_metrics[\"dec-jan-feb-weighted\"].observed_length\n    )\n\n\ndef test_legacy_deserialization_daily():\n    legacy_model_dict = {\n        \"model_type\": \"hdd_only\",\n        \"formula\": \"meter_value ~ hdd_46\",\n        \"status\": \"QUALIFIED\",\n        \"model_params\": {\"intercept\": 12, \"beta_hdd\": 2, \"heating_balance_point\": 50},\n        \"r_squared_adj\": 0.3,\n        \"warnings\": [],\n    }\n    serialized_str = json.dumps(legacy_model_dict)\n    baseline_model = DailyModel.from_2_0_json(serialized_str)\n\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-daily\"\n    )\n    blackout_end_date = sample_metadata[\"blackout_end_date\"]\n\n    # predict on reporting year and calculate savings\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=365\n    )\n    reporting_data = DailyReportingData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n    metered_savings_dataframe = baseline_model.predict(reporting_data)\n    total_metered_savings = (\n        metered_savings_dataframe[\"observed\"] - metered_savings_dataframe[\"predicted\"]\n    ).sum()\n\n    assert round(total_metered_savings, 2) == 891.2\n\n\ndef test_legacy_deserialization_hourly(request):\n    with open(request.fspath.dirname + \"/legacy_hourly.json\", \"r\") as f:\n        legacy_str = f.read()\n    baseline_model = HourlyCaltrackModel.from_2_0_json(legacy_str)\n\n    meter_data, temperature_data, sample_metadata = load_sample(\n        \"il-electricity-cdd-hdd-hourly\"\n    )\n    blackout_end_date = sample_metadata[\"blackout_end_date\"]\n\n    reporting_meter_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=365\n    )\n    reporting = HourlyCaltrackReportingData.from_series(\n        reporting_meter_data, temperature_data, is_electricity_data=True\n    )\n\n    metered_savings_dataframe = baseline_model.predict(reporting)\n    total_metered_savings = (\n        metered_savings_dataframe[\"observed\"] - metered_savings_dataframe[\"predicted\"]\n    ).sum()\n\n    assert round(total_metered_savings, 2) == -52454.02\n"
  },
  {
    "path": "tests/test_samples.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport datetime\n\nimport pytest\nimport pytz\n\nfrom opendsm.eemeter.samples import samples, load_sample\n\n\ndef test_samples():\n    assert samples() == [\n        \"il-electricity-cdd-hdd-billing_bimonthly\",\n        \"il-electricity-cdd-hdd-billing_monthly\",\n        \"il-electricity-cdd-hdd-daily\",\n        \"il-electricity-cdd-hdd-hourly\",\n        \"il-electricity-cdd-only-billing_bimonthly\",\n        \"il-electricity-cdd-only-billing_monthly\",\n        \"il-electricity-cdd-only-daily\",\n        \"il-electricity-cdd-only-hourly\",\n        \"il-gas-hdd-only-billing_bimonthly\",\n        \"il-gas-hdd-only-billing_monthly\",\n        \"il-gas-hdd-only-daily\",\n        \"il-gas-hdd-only-hourly\",\n        \"il-gas-intercept-only-billing_bimonthly\",\n        \"il-gas-intercept-only-billing_monthly\",\n        \"il-gas-intercept-only-daily\",\n        \"il-gas-intercept-only-hourly\",\n        \"uk-electricity-hdd-only-hourly-sample-0\",\n        \"uk-electricity-hdd-only-hourly-sample-1\",\n        \"uk-electricity-hdd-only-hourly-sample-2\",\n        \"uk-gas-hdd-only-hourly-sample-0\",\n    ]\n\n\ndef test_load_sample_hourly():\n    meter_data, temperature_data, metadata = load_sample(\n        \"il-electricity-cdd-hdd-hourly\"\n    )\n\n    assert meter_data.shape == (19417, 1)\n    assert meter_data.index.freq == \"h\"\n    assert temperature_data.shape == (19417,)\n    assert temperature_data.index.freq == \"h\"\n    assert metadata == {\n        \"annual_baseline_base_load\": 2000.0,\n        \"annual_baseline_cooling_load\": 4000.0,\n        \"annual_baseline_heating_load\": 4000.0,\n        \"annual_baseline_total_load\": 10000,\n        \"annual_reporting_base_load\": 1800.0,\n        \"annual_reporting_cooling_load\": 3600.0,\n        \"annual_reporting_heating_load\": 3600.0,\n        \"annual_reporting_total_load\": 9000.0,\n        \"baseline_cooling_balance_point\": 65,\n        \"baseline_heating_balance_point\": 60,\n        \"blackout_end_date\": datetime.datetime(2017, 1, 4, 0, 0, tzinfo=pytz.UTC),\n        \"blackout_start_date\": datetime.datetime(2016, 12, 26, 0, 0, tzinfo=pytz.UTC),\n        \"freq\": \"hourly\",\n        \"id\": \"il-electricity-cdd-hdd-hourly\",\n        \"interpretation\": \"electricity\",\n        \"meter_data_filename\": \"il-electricity-cdd-hdd-hourly.csv.gz\",\n        \"reporting_cooling_balance_point\": 65,\n        \"reporting_heating_balance_point\": 60,\n        \"temperature_filename\": \"il-tempF.csv.gz\",\n        \"unit\": \"kWh\",\n        \"usaf_id\": \"724390\",\n    }\n\n\ndef test_load_sample_daily():\n    meter_data, temperature_data, metadata = load_sample(\"il-electricity-cdd-hdd-daily\")\n\n    assert meter_data.shape == (810, 1)\n    assert meter_data.index.freq == \"D\"\n    assert temperature_data.shape == (19417,)\n    assert temperature_data.index.freq == \"h\"\n    assert metadata is not None\n\n\ndef test_load_sample_billing_monthly():\n    meter_data, temperature_data, metadata = load_sample(\n        \"il-electricity-cdd-hdd-billing_monthly\"\n    )\n\n    assert meter_data.shape == (27, 1)\n    assert meter_data.index.freq is None\n    assert temperature_data.shape == (19417,)\n    assert temperature_data.index.freq == \"h\"\n    assert metadata is not None\n\n\ndef test_load_sample_unknown():\n    with pytest.raises(ValueError):\n        load_sample(\"unknown\")\n"
  },
  {
    "path": "tests/test_segmentation.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport json\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom opendsm.eemeter.models.hourly_caltrack.segmentation import (\n    CalTRACKSegmentModel,\n    SegmentedModel,\n    segment_time_series,\n    iterate_segmented_dataset,\n)\n\n\n@pytest.fixture\ndef index_8760():\n    return pd.date_range(\"2017-01-01\", periods=365 * 24, freq=\"h\", tz=\"UTC\")\n\n\ndef test_segment_time_series_invalid_type(index_8760):\n    with pytest.raises(ValueError):\n        segment_time_series(index_8760, segment_type=\"unknown\")\n\n\ndef test_segment_time_series_single(index_8760):\n    weights = segment_time_series(index_8760, segment_type=\"single\")\n    assert list(weights.columns) == [\"all\"]\n    assert weights.shape == (8760, 1)\n    assert weights.sum().sum() == 8760.0\n\n\ndef test_segment_time_series_one_month(index_8760):\n    weights = segment_time_series(index_8760, segment_type=\"one_month\")\n    assert list(weights.columns) == [\n        \"jan\",\n        \"feb\",\n        \"mar\",\n        \"apr\",\n        \"may\",\n        \"jun\",\n        \"jul\",\n        \"aug\",\n        \"sep\",\n        \"oct\",\n        \"nov\",\n        \"dec\",\n    ]\n    assert weights.shape == (8760, 12)\n    assert weights.sum().sum() == 8760.0\n\n\ndef test_segment_time_series_three_month(index_8760):\n    weights = segment_time_series(index_8760, segment_type=\"three_month\")\n    assert list(weights.columns) == [\n        \"dec-jan-feb\",\n        \"jan-feb-mar\",\n        \"feb-mar-apr\",\n        \"mar-apr-may\",\n        \"apr-may-jun\",\n        \"may-jun-jul\",\n        \"jun-jul-aug\",\n        \"jul-aug-sep\",\n        \"aug-sep-oct\",\n        \"sep-oct-nov\",\n        \"oct-nov-dec\",\n        \"nov-dec-jan\",\n    ]\n    assert weights.shape == (8760, 12)\n    assert weights.sum().sum() == 26280.0\n\n\ndef test_segment_time_series_three_month_weighted(index_8760):\n    weights = segment_time_series(index_8760, segment_type=\"three_month_weighted\")\n    assert list(weights.columns) == [\n        \"dec-jan-feb-weighted\",\n        \"jan-feb-mar-weighted\",\n        \"feb-mar-apr-weighted\",\n        \"mar-apr-may-weighted\",\n        \"apr-may-jun-weighted\",\n        \"may-jun-jul-weighted\",\n        \"jun-jul-aug-weighted\",\n        \"jul-aug-sep-weighted\",\n        \"aug-sep-oct-weighted\",\n        \"sep-oct-nov-weighted\",\n        \"oct-nov-dec-weighted\",\n        \"nov-dec-jan-weighted\",\n    ]\n    assert weights.shape == (8760, 12)\n    assert weights.sum().sum() == 17520.0\n\n\ndef test_segment_time_series_drop_zero_weight_segments(index_8760):\n    weights = segment_time_series(\n        index_8760[:100], segment_type=\"one_month\", drop_zero_weight_segments=True\n    )\n    assert list(weights.columns) == [\"jan\"]\n    assert weights.shape == (100, 1)\n    assert weights.sum().sum() == 100.0\n\n\n@pytest.fixture\ndef dataset():\n    index = pd.date_range(\"2017-01-01\", periods=1000, freq=\"h\", tz=\"UTC\")\n    return pd.DataFrame({\"a\": 1, \"b\": 2}, index=index, columns=[\"a\", \"b\"])\n\n\ndef test_iterate_segmented_dataset_no_segmentation(dataset):\n    iterator = iterate_segmented_dataset(dataset, segmentation=None)\n    segment_name, data = next(iterator)\n    assert segment_name is None\n    assert list(data.columns) == [\"a\", \"b\", \"weight\"]\n    assert data.shape == (1000, 3)\n    assert data.sum().sum() == 4000\n\n    with pytest.raises(StopIteration):\n        next(iterator)\n\n\n@pytest.fixture\ndef segmentation(dataset):\n    return segment_time_series(dataset.index, segment_type=\"one_month\")\n\n\ndef test_iterate_segmented_dataset_with_segmentation(dataset, segmentation):\n    iterator = iterate_segmented_dataset(dataset, segmentation=segmentation)\n    segment_name, data = next(iterator)\n    assert segment_name == \"jan\"\n    assert list(data.columns) == [\"a\", \"b\", \"weight\"]\n    assert data.shape == (744, 3)\n    assert data.sum().sum() == 2976.0\n\n    segment_name, data = next(iterator)\n    assert segment_name == \"feb\"\n    assert list(data.columns) == [\"a\", \"b\", \"weight\"]\n    assert data.shape == (256, 3)\n    assert data.sum().sum() == 1024.0\n\n    segment_name, data = next(iterator)\n    assert segment_name == \"mar\"\n    assert list(data.columns) == [\"a\", \"b\", \"weight\"]\n    assert data.shape == (0, 3)\n    assert data.sum().sum() == 0.0\n\n\ndef test_iterate_segmented_dataset_with_processor(dataset, segmentation):\n    feature_processor_segment_names = []\n\n    def feature_processor(\n        segment_name, dataset, column_mapping=None\n    ):  # rename some columns\n        feature_processor_segment_names.append(segment_name)\n        return dataset.rename(columns=column_mapping).assign(weight=1)\n\n    iterator = iterate_segmented_dataset(\n        dataset,\n        segmentation=segmentation,\n        feature_processor=feature_processor,\n        feature_processor_kwargs={\"column_mapping\": {\"a\": \"c\", \"b\": \"d\"}},\n        feature_processor_segment_name_mapping={\"jan\": \"jan2\", \"feb\": \"feb2\"},\n    )\n    segment_name, data = next(iterator)\n    assert feature_processor_segment_names == [\"jan2\"]\n    assert segment_name == \"jan\"\n    assert list(data.columns) == [\"c\", \"d\", \"weight\"]\n    assert data.shape == (1000, 3)\n    assert data.sum().sum() == 4000.0\n\n    segment_name, data = next(iterator)\n    assert feature_processor_segment_names == [\"jan2\", \"feb2\"]\n    assert segment_name == \"feb\"\n    assert list(data.columns) == [\"c\", \"d\", \"weight\"]\n    assert data.shape == (1000, 3)\n    assert data.sum().sum() == 4000.0\n\n\ndef test_segment_model():\n    segment_model = CalTRACKSegmentModel(\n        segment_name=\"segment\",\n        model=None,\n        formula=\"meter_value ~ C(hour_of_week) + a - 1\",\n        model_params={\"C(hour_of_week)[1]\": 1, \"a\": 1},\n        warnings=None,\n    )\n    index = pd.date_range(\"2017-01-01\", periods=2, freq=\"h\", tz=\"UTC\")\n    data = pd.DataFrame({\"a\": [1, 1], \"hour_of_week\": [1, 1]}, index=index)\n    prediction = segment_model.predict(data)\n    assert prediction.sum() == 4\n\n\ndef test_segmented_model():\n    segment_model = CalTRACKSegmentModel(\n        segment_name=\"jan\",\n        model=None,\n        formula=\"meter_value ~ C(hour_of_week) + a- 1\",\n        model_params={\"C(hour_of_week)[1]\": 1, \"a\": 1},\n        warnings=None,\n    )\n\n    def fake_feature_processor(segment_name, segment_data):\n        return pd.DataFrame(\n            {\"hour_of_week\": 1, \"a\": 1, \"weight\": segment_data.weight},\n            index=segment_data.index,\n        )\n\n    segmented_model = SegmentedModel(\n        segment_models=[segment_model],\n        prediction_segment_type=\"one_month\",\n        prediction_segment_name_mapping=None,\n        prediction_feature_processor=fake_feature_processor,\n        prediction_feature_processor_kwargs=None,\n    )\n\n    # make this cover jan and feb but only supply jan model\n    index = pd.date_range(\"2017-01-01\", periods=24 * 50, freq=\"h\", tz=\"UTC\")\n    temps = pd.Series(np.linspace(0, 100, 24 * 50), index=index)\n    prediction = segmented_model.predict(temps.index, temps).result.predicted_usage\n    assert prediction.sum() == 1488.0\n\n\ndef test_segment_model_serialized():\n    segment_model = CalTRACKSegmentModel(\n        segment_name=\"jan\",\n        model=None,\n        formula=\"meter_value ~ a + b - 1\",\n        model_params={\"a\": 1, \"b\": 1},\n        warnings=None,\n    )\n    assert segment_model.json()[\"formula\"] == \"meter_value ~ a + b - 1\"\n    assert segment_model.json()[\"model_params\"] == {\"a\": 1, \"b\": 1}\n    assert segment_model.json()[\"warnings\"] == []\n    assert json.dumps(segment_model.json())\n\n\ndef test_segmented_model_serialized():\n    segment_model = CalTRACKSegmentModel(\n        segment_name=\"jan\",\n        model=None,\n        formula=\"meter_value ~ a + b - 1\",\n        model_params={\"a\": 1, \"b\": 1},\n        warnings=None,\n    )\n\n    def fake_feature_processor(segment_name, segment_data):  # pragma: no cover\n        return pd.DataFrame(\n            {\"a\": 1, \"b\": 1, \"weight\": segment_data.weight}, index=segment_data.index\n        )\n\n    segmented_model = SegmentedModel(\n        segment_models=[segment_model],\n        prediction_segment_type=\"one_month\",\n        prediction_segment_name_mapping=None,\n        prediction_feature_processor=fake_feature_processor,\n        prediction_feature_processor_kwargs=None,\n    )\n    assert segmented_model.json()[\"prediction_segment_type\"] == \"one_month\"\n    assert (\n        segmented_model.json()[\"prediction_feature_processor\"]\n        == \"fake_feature_processor\"\n    )\n    assert json.dumps(segmented_model.json())\n"
  },
  {
    "path": "tests/test_transform.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom datetime import datetime, timedelta\n\nimport numpy as np\nimport pandas as pd\nimport pytest\nimport pytz\n\nfrom opendsm.eemeter.common.transform import (\n    as_freq,\n    clean_caltrack_billing_data,\n    downsample_and_clean_caltrack_daily_data,\n    clean_caltrack_billing_daily_data,\n    day_counts,\n    get_baseline_data,\n    get_reporting_data,\n    get_terms,\n    remove_duplicates,\n    NoBaselineDataError,\n    NoReportingDataError,\n    overwrite_partial_rows_with_nan,\n    add_freq,\n    trim,\n    format_energy_data_for_caltrack,\n    format_temperature_data_for_caltrack,\n)\n\n\ndef test_as_freq_not_series(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    assert meter_data.shape == (27, 1)\n    with pytest.raises(ValueError):\n        as_freq(meter_data, freq=\"h\")\n\n\ndef test_as_freq_hourly(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    assert meter_data.shape == (27, 1)\n    as_hourly = as_freq(meter_data.value, freq=\"h\")\n    assert as_hourly.shape == (18961,)\n    assert round(meter_data.value.sum(), 1) == round(as_hourly.sum(), 1) == 21290.2\n\n\ndef test_as_freq_daily(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    assert meter_data.shape == (27, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\")\n    assert as_daily.shape == (792,)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.sum(), 1) == 21290.2\n\n\ndef test_as_freq_daily_all_nones_instantaneous(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    meter_data[\"value\"] = np.nan\n    assert meter_data.shape == (27, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\", series_type=\"instantaneous\")\n    assert as_daily.shape == (792,)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.sum(), 1) == 0\n\n\ndef test_as_freq_daily_all_nones(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    meter_data[\"value\"] = np.nan\n    assert meter_data.shape == (27, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\")\n    assert as_daily.shape == (792,)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.sum(), 1) == 0\n\n\ndef test_as_freq_month_start(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    assert meter_data.shape == (27, 1)\n    as_month_start = as_freq(meter_data.value, freq=\"MS\")\n    assert as_month_start.shape == (28,)\n    assert round(meter_data.value.sum(), 1) == round(as_month_start.sum(), 1) == 21290.2\n\n\ndef test_as_freq_hourly_temperature(il_electricity_cdd_hdd_billing_monthly):\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    assert temperature_data.shape == (19417,)\n    as_hourly = as_freq(temperature_data, freq=\"h\", series_type=\"instantaneous\")\n    assert as_hourly.shape == (19417,)\n    assert round(temperature_data.mean(), 1) == round(as_hourly.mean(), 1) == 54.6\n\n\ndef test_as_freq_daily_temperature(il_electricity_cdd_hdd_billing_monthly):\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    assert temperature_data.shape == (19417,)\n    as_daily = as_freq(temperature_data, freq=\"D\", series_type=\"instantaneous\")\n    assert as_daily.shape == (811,)\n    assert abs(temperature_data.mean() - as_daily.mean()) <= 0.1\n\n\ndef test_as_freq_month_start_temperature(il_electricity_cdd_hdd_billing_monthly):\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    assert temperature_data.shape == (19417,)\n    as_month_start = as_freq(temperature_data, freq=\"MS\", series_type=\"instantaneous\")\n    assert as_month_start.shape == (29,)\n    assert round(as_month_start.mean(), 1) == 53.4\n\n\ndef test_as_freq_daily_temperature_monthly(il_electricity_cdd_hdd_billing_monthly):\n    temperature_data = il_electricity_cdd_hdd_billing_monthly[\"temperature_data\"]\n    temperature_data = temperature_data.groupby(pd.Grouper(freq=\"MS\")).mean()\n    assert temperature_data.shape == (28,)\n    as_daily = as_freq(temperature_data, freq=\"D\", series_type=\"instantaneous\")\n    assert as_daily.shape == (824,)\n    assert round(as_daily.mean(), 1) == 54.5\n\n\ndef test_as_freq_empty():\n    meter_data = pd.DataFrame({\"value\": []})\n    empty_meter_data = as_freq(meter_data.value, freq=\"h\")\n    assert empty_meter_data.empty\n\n\ndef test_as_freq_perserves_nulls(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    monthly_with_nulls = meter_data[meter_data.index.year != 2016].reindex(\n        meter_data.index\n    )\n    daily_with_nulls = as_freq(monthly_with_nulls.value, freq=\"D\")\n    assert (\n        round(monthly_with_nulls.value.sum(), 2)\n        == round(daily_with_nulls.sum(), 2)\n        == 11094.05\n    )\n    assert monthly_with_nulls.value.isnull().sum() == 13\n    assert daily_with_nulls.isnull().sum() == 365\n\n\ndef test_day_counts(il_electricity_cdd_hdd_billing_monthly):\n    data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"].value\n    counts = day_counts(data.index)\n    assert counts.shape == (27,)\n    assert counts.iloc[0] == 29.0\n    assert pd.isnull(counts.iloc[-1])\n    assert counts.sum() == 790.0\n\n\ndef test_day_counts_empty_series():\n    index = pd.DatetimeIndex([])\n    index.freq = None\n    data = pd.Series([], index=index)\n    counts = day_counts(data.index)\n    assert counts.shape == (0,)\n\n\ndef test_get_baseline_data(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(meter_data)\n    assert meter_data.shape == baseline_data.shape == (19417, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_with_timezones(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data.tz_convert(\"America/New_York\")\n    )\n    assert len(warnings) == 0\n    baseline_data, warnings = get_baseline_data(\n        meter_data.tz_convert(\"Australia/Sydney\")\n    )\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_with_end(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    baseline_data, warnings = get_baseline_data(meter_data, end=blackout_start_date)\n    assert meter_data.shape != baseline_data.shape == (8761, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_with_end_no_max_days(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data, end=blackout_start_date, max_days=None\n    )\n    assert meter_data.shape != baseline_data.shape == (9595, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_empty(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_start_date = il_electricity_cdd_hdd_hourly[\"blackout_start_date\"]\n    with pytest.raises(NoBaselineDataError):\n        get_baseline_data(meter_data, end=pd.Timestamp(\"2000\").tz_localize(\"UTC\"))\n\n\ndef test_get_baseline_data_start_gap(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    start = meter_data.index.min() - timedelta(days=1)\n    baseline_data, warnings = get_baseline_data(meter_data, start=start, max_days=None)\n    assert meter_data.shape == baseline_data.shape == (19417, 1)\n    assert len(warnings) == 1\n    warning = warnings[0]\n    assert warning.qualified_name == \"eemeter.get_baseline_data.gap_at_baseline_start\"\n    assert (\n        warning.description\n        == \"Data does not have coverage at requested baseline start date.\"\n    )\n    assert warning.data == {\n        \"data_start\": \"2015-11-22T06:00:00+00:00\",\n        \"requested_start\": \"2015-11-21T06:00:00+00:00\",\n    }\n\n\ndef test_get_baseline_data_end_gap(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    end = meter_data.index.max() + timedelta(days=1)\n    baseline_data, warnings = get_baseline_data(meter_data, end=end, max_days=None)\n    assert meter_data.shape == baseline_data.shape == (19417, 1)\n    assert len(warnings) == 1\n    warning = warnings[0]\n    assert warning.qualified_name == \"eemeter.get_baseline_data.gap_at_baseline_end\"\n    assert (\n        warning.description\n        == \"Data does not have coverage at requested baseline end date.\"\n    )\n    assert warning.data == {\n        \"data_end\": \"2018-02-08T06:00:00+00:00\",\n        \"requested_end\": \"2018-02-09T06:00:00+00:00\",\n    }\n\n\ndef test_get_baseline_data_with_overshoot(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=32,\n        allow_billing_period_overshoot=True,\n    )\n    assert baseline_data.shape == (2, 1)\n    assert round(baseline_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=32,\n        allow_billing_period_overshoot=False,\n    )\n    assert baseline_data.shape == (1, 1)\n    assert round(baseline_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=True,\n    )\n    assert baseline_data.shape == (1, 1)\n    assert round(baseline_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_with_ignored_gap(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=45,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (2, 1)\n    assert round(baseline_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=45,\n        ignore_billing_period_gap_for_day_count=False,\n    )\n    assert baseline_data.shape == (1, 1)\n    assert round(baseline_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (1, 1)\n    assert round(baseline_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_with_overshoot_and_ignored_gap(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=True,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (2, 1)\n    assert round(baseline_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2016, 11, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=False,\n        ignore_billing_period_gap_for_day_count=False,\n    )\n    assert baseline_data.shape == (1, 1)\n    assert round(baseline_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_n_days_billing_period_overshoot(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=datetime(2017, 11, 9, tzinfo=pytz.UTC),\n        max_days=45,\n        allow_billing_period_overshoot=True,\n        n_days_billing_period_overshoot=45,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (2, 1)\n    assert round(baseline_data.value.sum(), 2) == 526.25\n    assert len(warnings) == 0\n\n\ndef test_get_baseline_data_too_far_from_date(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    end_date = datetime(2020, 11, 9, tzinfo=pytz.UTC)\n    max_days = 45\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=end_date,\n        max_days=max_days,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (2, 1)\n    assert round(baseline_data.value.sum(), 2) == 1393.4\n    assert len(warnings) == 0\n    with pytest.raises(NoBaselineDataError):\n        get_baseline_data(\n            meter_data,\n            end=end_date,\n            max_days=max_days,\n            n_days_billing_period_overshoot=45,\n            ignore_billing_period_gap_for_day_count=True,\n        )\n    baseline_data, warnings = get_baseline_data(\n        meter_data,\n        end=end_date,\n        max_days=max_days,\n        allow_billing_period_overshoot=True,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert baseline_data.shape == (3, 1)\n    assert round(baseline_data.value.sum(), 2) == 2043.92\n    assert len(warnings) == 0\n    # Includes 3 data points because data at index -3 is closer to start target\n    # then data at index -2\n    start_target = baseline_data.index[-1] - timedelta(days=max_days)\n    assert abs((baseline_data.index[0] - start_target).days) < abs(\n        (baseline_data.index[1] - start_target).days\n    )\n    with pytest.raises(NoBaselineDataError):\n        get_baseline_data(\n            meter_data,\n            end=end_date,\n            max_days=max_days,\n            allow_billing_period_overshoot=True,\n            n_days_billing_period_overshoot=45,\n            ignore_billing_period_gap_for_day_count=True,\n        )\n\n\ndef test_get_reporting_data(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    reporting_data, warnings = get_reporting_data(meter_data)\n    assert meter_data.shape == reporting_data.shape == (19417, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_with_timezones(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    reporting_data, warnings = get_reporting_data(\n        meter_data.tz_convert(\"America/New_York\")\n    )\n    assert len(warnings) == 0\n    reporting_data, warnings = get_reporting_data(\n        meter_data.tz_convert(\"Australia/Sydney\")\n    )\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_with_start(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_end_date = il_electricity_cdd_hdd_hourly[\"blackout_end_date\"]\n    reporting_data, warnings = get_reporting_data(meter_data, start=blackout_end_date)\n    assert meter_data.shape != reporting_data.shape == (8761, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_with_start_no_max_days(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_end_date = il_electricity_cdd_hdd_hourly[\"blackout_end_date\"]\n    reporting_data, warnings = get_reporting_data(\n        meter_data, start=blackout_end_date, max_days=None\n    )\n    assert meter_data.shape != reporting_data.shape == (9607, 1)\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_empty(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    blackout_end_date = il_electricity_cdd_hdd_hourly[\"blackout_end_date\"]\n    with pytest.raises(NoReportingDataError):\n        get_reporting_data(meter_data, start=pd.Timestamp(\"2030\").tz_localize(\"UTC\"))\n\n\ndef test_get_reporting_data_start_gap(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    start = meter_data.index.min() - timedelta(days=1)\n    reporting_data, warnings = get_reporting_data(\n        meter_data, start=start, max_days=None\n    )\n    assert meter_data.shape == reporting_data.shape == (19417, 1)\n    assert len(warnings) == 1\n    warning = warnings[0]\n    assert warning.qualified_name == \"eemeter.get_reporting_data.gap_at_reporting_start\"\n    assert (\n        warning.description\n        == \"Data does not have coverage at requested reporting start date.\"\n    )\n    assert warning.data == {\n        \"data_start\": \"2015-11-22T06:00:00+00:00\",\n        \"requested_start\": \"2015-11-21T06:00:00+00:00\",\n    }\n\n\ndef test_get_reporting_data_end_gap(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    end = meter_data.index.max() + timedelta(days=1)\n    reporting_data, warnings = get_reporting_data(meter_data, end=end, max_days=None)\n    assert meter_data.shape == reporting_data.shape == (19417, 1)\n    assert len(warnings) == 1\n    warning = warnings[0]\n    assert warning.qualified_name == \"eemeter.get_reporting_data.gap_at_reporting_end\"\n    assert (\n        warning.description\n        == \"Data does not have coverage at requested reporting end date.\"\n    )\n    assert warning.data == {\n        \"data_end\": \"2018-02-08T06:00:00+00:00\",\n        \"requested_end\": \"2018-02-09T06:00:00+00:00\",\n    }\n\n\ndef test_get_reporting_data_with_overshoot(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=30,\n        allow_billing_period_overshoot=True,\n    )\n    assert reporting_data.shape == (2, 1)\n    assert round(reporting_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=30,\n        allow_billing_period_overshoot=False,\n    )\n    assert reporting_data.shape == (1, 1)\n    assert round(reporting_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=True,\n    )\n    assert reporting_data.shape == (1, 1)\n    assert round(reporting_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_with_ignored_gap(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=45,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert reporting_data.shape == (2, 1)\n    assert round(reporting_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=45,\n        ignore_billing_period_gap_for_day_count=False,\n    )\n    assert reporting_data.shape == (1, 1)\n    assert round(reporting_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert reporting_data.shape == (1, 1)\n    assert round(reporting_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_reporting_data_with_overshoot_and_ignored_gap(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=True,\n        ignore_billing_period_gap_for_day_count=True,\n    )\n    assert reporting_data.shape == (2, 1)\n    assert round(reporting_data.value.sum(), 2) == 632.31\n    assert len(warnings) == 0\n\n    reporting_data, warnings = get_reporting_data(\n        meter_data,\n        start=datetime(2016, 9, 9, tzinfo=pytz.UTC),\n        max_days=25,\n        allow_billing_period_overshoot=False,\n        ignore_billing_period_gap_for_day_count=False,\n    )\n    assert reporting_data.shape == (1, 1)\n    assert round(reporting_data.value.sum(), 2) == 0\n    assert len(warnings) == 0\n\n\ndef test_get_terms_unrecognized_method(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    with pytest.raises(ValueError):\n        get_terms(meter_data.index, term_lengths=[365], method=\"unrecognized\")\n\n\ndef test_get_terms_unsorted_index(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    with pytest.raises(ValueError):\n        get_terms(meter_data.index[::-1], term_lengths=[365])\n\n\ndef test_get_terms_bad_term_labels(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    with pytest.raises(ValueError):\n        terms = get_terms(\n            meter_data.index,\n            term_lengths=[60, 60, 60],\n            term_labels=[\"abc\", \"def\"],  # too short\n        )\n\n\ndef test_get_terms_default_term_labels(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    terms = get_terms(meter_data.index, term_lengths=[60, 60, 60])\n    assert [t.label for t in terms] == [\"term_001\", \"term_002\", \"term_003\"]\n\n\ndef test_get_terms_custom_term_labels(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    terms = get_terms(\n        meter_data.index, term_lengths=[60, 60, 60], term_labels=[\"abc\", \"def\", \"ghi\"]\n    )\n    assert [t.label for t in terms] == [\"abc\", \"def\", \"ghi\"]\n\n\ndef test_get_terms_empty_index_input(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    terms = get_terms(meter_data.index[:0], term_lengths=[60, 60, 60])\n    assert len(terms) == 0\n\n\ndef test_get_terms_strict(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    strict_terms = get_terms(\n        meter_data.index,\n        term_lengths=[365, 365],\n        term_labels=[\"year1\", \"year2\"],\n        start=datetime(2016, 1, 15, tzinfo=pytz.UTC),\n        method=\"strict\",\n    )\n\n    assert len(strict_terms) == 2\n\n    year1 = strict_terms[0]\n    assert year1.label == \"year1\"\n    assert year1.index.shape == (12,)\n    assert (\n        year1.target_start_date\n        == pd.Timestamp(\"2016-01-15 00:00:00+0000\", tz=\"UTC\").to_pydatetime()\n    )\n    assert (\n        year1.target_end_date\n        == pd.Timestamp(\"2017-01-14 00:00:00+0000\", tz=\"UTC\").to_pydatetime()\n    )\n    assert year1.target_term_length_days == 365\n    assert (\n        year1.actual_start_date\n        == year1.index[0]\n        == pd.Timestamp(\"2016-01-22 06:00:00+0000\", tz=\"UTC\")\n    )\n    assert (\n        year1.actual_end_date\n        == year1.index[-1]\n        == pd.Timestamp(\"2016-12-19 06:00:00+0000\", tz=\"UTC\")\n    )\n    assert year1.actual_term_length_days == 332\n    assert year1.complete\n\n    year2 = strict_terms[1]\n    assert year2.index.shape == (13,)\n    assert year2.label == \"year2\"\n    assert year2.target_start_date == pd.Timestamp(\"2016-12-19 06:00:00+0000\", tz=\"UTC\")\n    assert (\n        year2.target_end_date\n        == pd.Timestamp(\"2018-01-14 00:00:00+0000\", tz=\"UTC\").to_pydatetime()\n    )\n    assert year2.target_term_length_days == 365\n    assert (\n        year2.actual_start_date\n        == year2.index[0]\n        == pd.Timestamp(\"2016-12-19 06:00:00+00:00\", tz=\"UTC\")\n    )\n    assert (\n        year2.actual_end_date\n        == year2.index[-1]\n        == pd.Timestamp(\"2017-12-22 06:00:00+0000\", tz=\"UTC\")\n    )\n    assert year2.actual_term_length_days == 368\n    assert year2.complete\n\n\ndef test_get_terms_nearest(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    nearest_terms = get_terms(\n        meter_data.index,\n        term_lengths=[365, 365],\n        term_labels=[\"year1\", \"year2\"],\n        start=datetime(2016, 1, 15, tzinfo=pytz.UTC),\n        method=\"nearest\",\n    )\n\n    assert len(nearest_terms) == 2\n\n    year1 = nearest_terms[0]\n    assert year1.label == \"year1\"\n    assert year1.index.shape == (13,)\n    assert year1.index[0] == pd.Timestamp(\"2016-01-22 06:00:00+0000\", tz=\"UTC\")\n    assert year1.index[-1] == pd.Timestamp(\"2017-01-21 06:00:00+0000\", tz=\"UTC\")\n    assert (\n        year1.target_start_date\n        == pd.Timestamp(\"2016-01-15 00:00:00+0000\", tz=\"UTC\").to_pydatetime()\n    )\n    assert year1.target_term_length_days == 365\n    assert year1.actual_term_length_days == 365\n    assert year1.complete\n\n    year2 = nearest_terms[1]\n    assert year2.label == \"year2\"\n    assert year2.index.shape == (13,)\n    assert year2.index[0] == pd.Timestamp(\"2017-01-21 06:00:00+0000\", tz=\"UTC\")\n    assert year2.index[-1] == pd.Timestamp(\"2018-01-20 06:00:00+0000\", tz=\"UTC\")\n    assert year2.target_start_date == pd.Timestamp(\"2017-01-21 06:00:00+0000\", tz=\"UTC\")\n    assert year1.target_term_length_days == 365\n    assert year2.actual_term_length_days == 364\n    assert not year2.complete  # no remaining index\n\n    # check completeness case with a shorter final term\n    nearest_terms = get_terms(\n        meter_data.index,\n        term_lengths=[365, 340],\n        term_labels=[\"year1\", \"year2\"],\n        start=datetime(2016, 1, 15, tzinfo=pytz.UTC),\n        method=\"nearest\",\n    )\n    year2 = nearest_terms[1]\n    assert year2.label == \"year2\"\n    assert year2.index.shape == (12,)\n    assert year2.index[0] == pd.Timestamp(\"2017-01-21 06:00:00+0000\", tz=\"UTC\")\n    assert year2.index[-1] == pd.Timestamp(\"2017-12-22 06:00:00+00:00\", tz=\"UTC\")\n    assert year2.target_start_date == pd.Timestamp(\"2017-01-21 06:00:00+0000\", tz=\"UTC\")\n    assert year2.target_term_length_days == 340\n    assert year2.actual_term_length_days == 335\n    assert year2.complete  # has remaining index\n\n\ndef test_term_repr(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n\n    terms = get_terms(meter_data.index, term_lengths=[60, 60, 60])\n    assert repr(terms[0]) == (\n        \"Term(label=term_001, target_term_length_days=60, actual_term_length_days=29,\"\n        \" complete=True)\"\n    )\n\n\ndef test_remove_duplicates_df():\n    index = pd.DatetimeIndex([\"2017-01-01\", \"2017-01-02\", \"2017-01-02\"])\n    df = pd.DataFrame({\"value\": [1, 2, 3]}, index=index)\n    assert df.shape == (3, 1)\n    df_dedupe = remove_duplicates(df)\n    assert df_dedupe.shape == (2, 1)\n    assert list(df_dedupe.value) == [1, 2]\n\n\ndef test_remove_duplicates_series():\n    index = pd.DatetimeIndex([\"2017-01-01\", \"2017-01-02\", \"2017-01-02\"])\n    series = pd.Series([1, 2, 3], index=index)\n    assert series.shape == (3,)\n    series_dedupe = remove_duplicates(series)\n    assert series_dedupe.shape == (2,)\n    assert list(series_dedupe) == [1, 2]\n\n\ndef test_as_freq_hourly_to_daily(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n\n    meter_data.iloc[-1, meter_data.columns.get_loc(\"value\")] = np.nan\n    assert meter_data.shape == (19417, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\")\n    assert as_daily.shape == (811,)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.sum(), 1) == 21926.0\n\n\ndef test_as_freq_daily_to_daily(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    assert meter_data.shape == (810, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\")\n    assert as_daily.shape == (810,)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.sum(), 1) == 21925.8\n\n\ndef test_as_freq_hourly_to_daily_include_coverage(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    meter_data.iloc[-1, meter_data.columns.get_loc(\"value\")] = np.nan\n    assert meter_data.shape == (19417, 1)\n    as_daily = as_freq(meter_data.value, freq=\"D\", include_coverage=True)\n    assert as_daily.shape == (811, 2)\n    assert round(meter_data.value.sum(), 1) == round(as_daily.value.sum(), 1) == 21926.0\n\n\ndef test_clean_caltrack_billing_daily_data_billing(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    cleaned_data = clean_caltrack_billing_daily_data(meter_data, \"billing_monthly\")\n    assert cleaned_data.shape == (27, 1)\n    pd.testing.assert_frame_equal(meter_data, cleaned_data)\n\n\ndef test_clean_caltrack_billing_daily_data_daily(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    cleaned_data = clean_caltrack_billing_daily_data(meter_data, \"daily\")\n    assert cleaned_data.shape == (810, 1)\n    pd.testing.assert_frame_equal(meter_data, cleaned_data)\n\n\ndef test_clean_caltrack_billing_daily_data_daily_local_tz(il_electricity_cdd_hdd_daily):\n    meter_data = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    meter_data.index += timedelta(hours=6)\n    meter_data = meter_data.tz_convert(\"America/Chicago\")\n    cleaned_data = clean_caltrack_billing_daily_data(meter_data, \"daily\")\n    assert cleaned_data.shape == (810, 1)\n    pd.testing.assert_frame_equal(meter_data, cleaned_data)\n\n\ndef test_clean_caltrack_billing_daily_data_hourly(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    cleaned_data = clean_caltrack_billing_daily_data(meter_data, \"hourly\")\n    assert cleaned_data.shape == (811, 1)\n\n\ndef test_clean_caltrack_daily_data_hourly(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    cleaned_data = downsample_and_clean_caltrack_daily_data(meter_data)\n    assert cleaned_data.shape == (811, 1)\n\n\ndef test_clean_caltrack_daily_data_hourly_local_tz(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    meter_data = meter_data.tz_convert(\"America/Chicago\")\n    cleaned_data = downsample_and_clean_caltrack_daily_data(meter_data)\n    assert cleaned_data.shape == (810, 1)\n\n\ndef test_clean_caltrack_billing_data_estimated(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    meter_data[\"estimated\"] = False\n    estimated_col_index = meter_data.columns.get_loc(\"estimated\")\n    meter_data.iloc[:, estimated_col_index] = False\n    meter_data.iloc[2, estimated_col_index] = True\n    meter_data.iloc[5, estimated_col_index] = True\n    meter_data.iloc[6, estimated_col_index] = True\n    meter_data.iloc[10, estimated_col_index] = True\n\n    cleaned_data = clean_caltrack_billing_data(meter_data, \"billing_monthly\")\n    assert cleaned_data.dropna().shape[0] == cleaned_data.shape[0] - 2\n\n\ndef test_clean_caltrack_billing_data_uneven_datetimes(\n    il_electricity_cdd_hdd_billing_monthly,\n):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    too_short_meter_data = pd.concat(\n        [\n            meter_data,\n            pd.DataFrame(\n                data=[{\"value\": 100}],\n                index=[datetime(2017, 1, 1, 6).replace(tzinfo=pytz.UTC)],\n            ),\n        ]\n    ).sort_index()\n    cleaned_data = clean_caltrack_billing_data(too_short_meter_data, \"billing_monthly\")\n    assert cleaned_data.dropna().shape[0] == cleaned_data.shape[0] - 3\n\n    too_long_meter_data = meter_data.drop(\n        [datetime(2016, 12, 19, 6).replace(tzinfo=pytz.UTC)]\n    )\n    cleaned_data = clean_caltrack_billing_data(too_long_meter_data, \"billing_monthly\")\n\n    too_long_meter_data = meter_data.drop(\n        [\n            datetime(2016, 12, 19, 6).replace(tzinfo=pytz.UTC),\n            datetime(2017, 1, 21, 6).replace(tzinfo=pytz.UTC),\n        ]\n    )\n    cleaned_data = clean_caltrack_billing_data(too_long_meter_data, \"billing_bimonthly\")\n    assert cleaned_data.dropna().shape[0] == cleaned_data.shape[0] - 2\n    assert cleaned_data.dropna().shape[0] == cleaned_data.shape[0] - 2\n\n    pre_empty_meter_data = meter_data[:0]\n    cleaned_data = clean_caltrack_billing_data(pre_empty_meter_data, \"billing_monthly\")\n    assert cleaned_data.empty\n\n    post_empty_meter_data = meter_data[:4].drop(\n        [\n            datetime(2015, 12, 21, 6).replace(tzinfo=pytz.UTC),\n            datetime(2016, 1, 22, 6).replace(tzinfo=pytz.UTC),\n        ]\n    )\n    assert not post_empty_meter_data[\"value\"].dropna().empty\n    cleaned_data = clean_caltrack_billing_data(post_empty_meter_data, \"billing_monthly\")\n    assert cleaned_data.empty\n\n\ndef test_overwrite_partial_rows_with_nan(il_electricity_cdd_hdd_billing_monthly):\n    meter_data = il_electricity_cdd_hdd_billing_monthly[\"meter_data\"]\n    meter_data[\"other_column\"] = meter_data[\"value\"]\n    meter_data.iloc[:3, meter_data.columns.get_loc(\"other_column\")] = np.nan\n    meter_data_nanned = overwrite_partial_rows_with_nan(meter_data)\n    assert pd.isnull(meter_data_nanned[\"value\"][:3]).all()\n\n\nimport pandas as pd\n\n\ndef test_add_freq(il_electricity_cdd_hdd_hourly):\n    meter_data = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n\n    # make DateTimeIndex timezone-naive\n    meter_data.index = meter_data.index.tz_localize(None)\n\n    # infer frequency\n    meter_data.index = add_freq(meter_data.index)\n    assert meter_data.index.freq == \"h\"\n\n\ndef test_trim_two_dataframes(\n    uk_electricity_hdd_only_hourly_sample_1, uk_electricity_hdd_only_hourly_sample_2\n):\n    df1 = uk_electricity_hdd_only_hourly_sample_1[\"meter_data\"]\n    df2 = uk_electricity_hdd_only_hourly_sample_2[\"meter_data\"]\n\n    df1_trimmed, df2_trimmed = trim(df1, df2)\n\n    assert (\n        df1.index[0] == df1.index.min()\n        and df2.index[0] == df2.index.min()\n        and df1.index[0] != df2.index[0]\n    )\n\n    assert (\n        df1.index[-1] == df1.index.max()\n        and df2.index[-1] == df2.index.max()\n        and df1.index[-1] != df2.index[-1]\n    )\n\n    assert df1_trimmed.index[0] == df2_trimmed.index[0]\n    assert df1_trimmed.index.min() == df2_trimmed.index.min()\n    assert df1_trimmed.index[-1] == df2_trimmed.index[-1]\n    assert df1_trimmed.index.max() == df2_trimmed.index.max()\n\n\ndef test_format_temperature_data_for_caltrack(il_electricity_cdd_hdd_hourly):\n    temperature_data = il_electricity_cdd_hdd_hourly[\"temperature_data\"]\n\n    # temperature_data to pd.DateFrame\n    temperature_data = pd.DataFrame(temperature_data)\n\n    # flipping df\n    temperature_data = temperature_data.reindex(index=temperature_data.index[::-1])\n\n    # inserting new value of 0.04 at 09.34 22/11/2015\n    new_start = pd.to_datetime(\"22/11/2015 09:34\", dayfirst=True).tz_localize(\"UTC\")\n    temperature_data.loc[new_start] = [0.04]\n\n    # rename column name to 'consumption'\n    temperature_data.rename(columns={\"value\": \"consumption\"}, inplace=True)\n\n    temperature_data_reformatted = format_temperature_data_for_caltrack(\n        temperature_data\n    )\n\n    assert isinstance(temperature_data_reformatted, pd.Series)\n    assert (\n        temperature_data_reformatted.index[0] < temperature_data_reformatted.index[-1]\n    )\n    assert temperature_data_reformatted.index.freq == \"h\"\n    assert temperature_data_reformatted.index.tzinfo is not None\n\n\ndef test_format_energy_data_for_caltrack_hourly(il_electricity_cdd_hdd_hourly):\n    df = il_electricity_cdd_hdd_hourly[\"meter_data\"]\n    # flipping df\n    df = df.reindex(index=df.index[::-1])\n\n    # inserting new value of 0.04 at 09.34 22/11/2015\n    new_start = pd.to_datetime(\"22/11/2015 09:34\", dayfirst=True).tz_localize(\"UTC\")\n    df.loc[new_start] = [0.04]\n\n    # rename column name to 'consumption'\n    df.rename(columns={\"value\": \"consumption\"}, inplace=True)\n\n    # df_flipped to pd.Series\n    df = df.squeeze()\n\n    df_reformatted = format_energy_data_for_caltrack(df, method=\"hourly\")\n\n    assert isinstance(df_reformatted, pd.DataFrame)\n    assert df_reformatted.index[0] < df_reformatted.index[-1]\n    assert df_reformatted.index.freq == \"h\"\n    assert df_reformatted.columns[0] == \"value\"\n    assert df_reformatted.index.tzinfo is not None\n    assert len(df_reformatted.columns) == 1\n\n\ndef test_format_energy_data_for_caltrack_daily(il_electricity_cdd_hdd_daily):\n    df = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    # flipping df\n    df = df.reindex(index=df.index[::-1])\n\n    # inserting new value of 0.04 at 09.34 22/11/2015\n    new_start = pd.to_datetime(\"22/11/2015 09:34\", dayfirst=True).tz_localize(\"UTC\")\n    df.loc[new_start] = [0.04]\n\n    # rename column name to 'consumption'\n    df.rename(columns={\"value\": \"consumption\"}, inplace=True)\n\n    # df_flipped to pd.Series\n    df = df.squeeze()\n\n    df_reformatted = format_energy_data_for_caltrack(df, method=\"daily\")\n\n    assert isinstance(df_reformatted, pd.DataFrame)\n    assert df_reformatted.index[0] < df_reformatted.index[-1]\n    assert df_reformatted.index.freq == \"D\"\n    assert df_reformatted.columns[0] == \"value\"\n    assert df_reformatted.index.tzinfo is not None\n    assert len(df_reformatted.columns) == 1\n\n\ndef test_format_energy_data_for_caltrack_billing(il_electricity_cdd_hdd_daily):\n    df = il_electricity_cdd_hdd_daily[\"meter_data\"]\n    # flipping df\n    df = df.reindex(index=df.index[::-1])\n\n    # inserting new value of 0.04 at 09.34 22/11/2015\n    new_start = pd.to_datetime(\"22/11/2015 09:34\", dayfirst=True).tz_localize(\"UTC\")\n    df.loc[new_start] = [0.04]\n\n    # rename column name to 'consumption'\n    df.rename(columns={\"value\": \"consumption\"}, inplace=True)\n\n    # df_flipped to pd.Series\n    df = df.squeeze()\n\n    df_reformatted = format_energy_data_for_caltrack(df, method=\"billing\")\n\n    assert isinstance(df_reformatted, pd.DataFrame)\n    assert df_reformatted.index[0] < df_reformatted.index[-1]\n    assert df_reformatted.index.freq == pd.tseries.offsets.MonthEnd()\n    assert df_reformatted.columns[0] == \"value\"\n    assert df_reformatted.index.tzinfo is not None\n    assert len(df_reformatted.columns) == 1\n"
  },
  {
    "path": "tests/test_version.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nimport opendsm\n\n\ndef test_version():\n    assert opendsm.__version__.startswith(\"1\")\n"
  },
  {
    "path": "tests/test_warnings.py",
    "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n#  Copyright 2014-2025 OpenDSM contributors\n#  Licensed under the Apache License, Version 2.0 (the \"License\");\n#  you may not use this file except in compliance with the License.\n#  You may obtain a copy of the License at\n#      http://www.apache.org/licenses/LICENSE-2.0\n#  Unless required by applicable law or agreed to in writing, software\n#  distributed under the License is distributed on an \"AS IS\" BASIS,\n#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n#  See the License for the specific language governing permissions and\n#  limitations under the License.\n\nfrom opendsm.eemeter.common.warnings import EEMeterWarning\n\n\ndef test_eemeter_warning():\n    eemeter_warning = EEMeterWarning(\n        qualified_name=\"qualified_name\", description=\"description\", data={}\n    )\n    assert eemeter_warning.qualified_name == \"qualified_name\"\n    assert eemeter_warning.description == \"description\"\n    assert eemeter_warning.data == {}\n    assert str(eemeter_warning).startswith(\"EEMeterWarning\")\n    assert eemeter_warning.json() == {\n        \"data\": {},\n        \"description\": \"description\",\n        \"qualified_name\": \"qualified_name\",\n    }\n"
  },
  {
    "path": "tox.ini",
    "content": "[tox]\nenvlist = 3.{10, 11, 12}\n\n[testenv]\ndeps =\n    pytest\n    pytest-cov\n    pytest-xdist\n    !3.12: snapshottest  # breaks due to importlib changes\n    numpy<2  # nlopt2.7.1 does not have a ceiling and breaks, nlopt2.9.0 will upgrade to numpy2 as needed\ncommands =\n    pytest tests/\n\n[testenv:3.12]\ncommands =\n    # we'll need to change snapshot libraries or refactor these tests for python>=3.12\n    pytest tests/ --ignore=tests/test_features.py"
  }
]