Showing preview only (218K chars total). Download the full file or copy to clipboard to get everything.
Repository: godatadriven/evol
Branch: master
Commit: 1274b675b2ca
Files: 77
Total size: 199.8 KB
Directory structure:
gitextract_y6cd46fo/
├── .gitignore
├── .gitpod.yml
├── LICENSE
├── Makefile
├── README.md
├── azure-pipelines.yml
├── doc/
│ ├── _static/
│ │ └── css/
│ │ └── custom.css
│ ├── api/
│ │ ├── evol.helpers.combiners.rst
│ │ ├── evol.helpers.mutators.rst
│ │ ├── evol.helpers.rst
│ │ ├── evol.problems.functions.rst
│ │ ├── evol.problems.routing.rst
│ │ ├── evol.problems.rst
│ │ ├── evol.rst
│ │ └── modules.rst
│ ├── conf.py
│ ├── development.rst
│ ├── index.rst
│ ├── population.rst
│ ├── problems.rst
│ └── quickstart.rst
├── evol/
│ ├── __init__.py
│ ├── conditions.py
│ ├── evolution.py
│ ├── exceptions.py
│ ├── helpers/
│ │ ├── __init__.py
│ │ ├── combiners/
│ │ │ ├── __init__.py
│ │ │ ├── permutation.py
│ │ │ └── utils.py
│ │ ├── groups.py
│ │ ├── mutators/
│ │ │ ├── __init__.py
│ │ │ └── permutation.py
│ │ ├── pickers.py
│ │ └── utils.py
│ ├── individual.py
│ ├── logger.py
│ ├── population.py
│ ├── problems/
│ │ ├── __init__.py
│ │ ├── functions/
│ │ │ ├── __init__.py
│ │ │ └── variableinput.py
│ │ ├── problem.py
│ │ └── routing/
│ │ ├── __init__.py
│ │ ├── coordinates.py
│ │ ├── magicsanta.py
│ │ └── tsp.py
│ ├── serialization.py
│ ├── step.py
│ └── utils.py
├── examples/
│ ├── number_of_parents.py
│ ├── population_demo.py
│ ├── rock_paper_scissors.py
│ ├── simple_callback.py
│ ├── simple_logging.py
│ ├── simple_nonlinear.py
│ ├── travelling_salesman.py
│ └── very_basic_tsp.py
├── setup.cfg
├── setup.py
├── test_local.sh
└── tests/
├── conftest.py
├── helpers/
│ ├── combiners/
│ │ └── test_permutation_combiners.py
│ ├── mutators/
│ │ └── test_permutation_mutators.py
│ ├── test_groups.py
│ └── test_helpers_utils.py
├── problems/
│ ├── test_functions.py
│ ├── test_santa.py
│ └── test_tsp.py
├── test_callback.py
├── test_conditions.py
├── test_evolution.py
├── test_examples.py
├── test_individual.py
├── test_logging.py
├── test_parallel_population.py
├── test_population.py
├── test_serialization.py
└── test_utils.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Created by .ignore support plugin (hsz.mobi)
### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# dotenv
.env
# virtualenv
.venv
venv/
ENV/
evol-env/
# Spyder project settings
.spyderproject
# Rope project settings
.ropeproject
# VSCode
.vscode/
# PyCharm
.idea/
# Mac
.DS_Store
# Pytest
.pytest_cache/
# documentation build folder
docs
================================================
FILE: .gitpod.yml
================================================
tasks:
- init: pyenv local 3.7.2 && pip install -e ".[dev]"
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2017-2020 GoDataDriven
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: Makefile
================================================
.PHONY: docs
flake:
python setup.py flake8
install:
pip install -e .
develop:
pip install -e ".[dev]"
python setup.py develop
test:
python setup.py test
check: test flake
docs:
sphinx-apidoc -f -o doc/api evol
sphinx-build doc docs
clean:
rm -rf .cache
rm -rf .eggs
rm -rf .pytest_cache
rm -rf build
rm -rf dist
rm -rf evol.egg-info
rm -rf .ipynb_checkpoints
push:
rm -rf dist
python setup.py sdist
python setup.py bdist_wheel --universal
twine upload dist/*
================================================
FILE: README.md
================================================
[](https://evol.readthedocs.io/en/latest/?badge=latest)[](https://pepy.tech/project/evol)
[](https://dev.azure.com/godatadriven/evol/_build/latest?definitionId=9&branchName=master) [](https://evol.readthedocs.io/en/latest/?badge=latest)[](https://pepy.tech/project/evol)
`Evol` is clear dsl for composable evolutionary algorithms that optimised for joy.
## Installation
We currently support python3.6 and python3.7 and you can install it via pip.
```
pip install evol
```
## Documentation
For more details you can read the [docs](https://evol.readthedocs.io/en/latest/) but we advice everyone to get start by first checking out the examples in the `/examples` directory. These stand alone examples should show the spirit of usage better than the docs.
## The Gist
The main idea is that you should be able to define a complex algorithm
in a composable way. To explain what we mean by this: let's consider
two evolutionary algorithms for travelling salesman problems.
The first approach takes a collections of solutions and applies:
1. a survival where only the top 50% solutions survive
2. the population reproduces using a crossover of genes
3. certain members mutate
4. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/is9g07u.png" alt="Drawing" style="width: 100%;"/>
We can also think of another approach:
1. pick the best solution of the population
2. make random changes to this parent and generate new solutions
3. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/JRSWbTd.png" alt="Drawing" style="width: 100%;"/>
One could even combine the two algorithms into a new one:
1. run algorithm 1 50 times
2. run algorithm 2 10 times
3. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/SZTBWX2.png" alt="Drawing" style="width: 100%;"/>
You might notice that many parts of these algorithms are similar and it
is the goal of this library is to automate these parts. We hope to
provide an API that is fun to use and easy to tweak your heuristics in.
A working example of something silimar to what is depicted above is shown below. You can also find this code as an example in the `/examples/simple_nonlinear.py`.
```python
import random
from evol import Population, Evolution
random.seed(42)
def random_start():
"""
This function generates a random (x,y) coordinate
"""
return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20
def func_to_optimise(xy):
"""
This is the function we want to optimise (maximize)
"""
x, y = xy
return -(1-x)**2 - 2*(2-x**2)**2
def pick_random_parents(pop):
"""
This is how we are going to select parents from the population
"""
mom = random.choice(pop)
dad = random.choice(pop)
return mom, dad
def make_child(mom, dad):
"""
This function describes how two candidates combine into a new candidate
Note that the output is a tuple, just like the output of `random_start`
We leave it to the developer to ensure that chromosomes are of the same type
"""
child_x = (mom[0] + dad[0])/2
child_y = (mom[1] + dad[1])/2
return child_x, child_y
def add_noise(chromosome, sigma):
"""
This is a function that will add some noise to the chromosome.
"""
new_x = chromosome[0] + (random.random()-0.5) * sigma
new_y = chromosome[1] + (random.random()-0.5) * sigma
return new_x, new_y
# We start by defining a population with candidates.
pop = Population(chromosomes=[random_start() for _ in range(200)],
eval_function=func_to_optimise, maximize=True)
# We define a sequence of steps to change these candidates
evo1 = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(func=add_noise, sigma=1))
# We define another sequence of steps to change these candidates
evo2 = (Evolution()
.survive(n=1)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(func=add_noise, sigma=0.2))
# We are combining two evolutions into a third one. You don't have to
# but this approach demonstrates the flexibility of the library.
evo3 = (Evolution()
.repeat(evo1, n=50)
.repeat(evo2, n=10)
.evaluate())
# In this step we are telling evol to apply the evolutions
# to the population of candidates.
pop = pop.evolve(evo3, n=5)
print(f"the best score found: {max([i.fitness for i in pop])}")
```
Getting Started
---------------------------------------
The best place to get started is the `/examples` folder on github.
This folder contains self contained examples that work out of the
box.
## How does it compare to ...
- [... deap?](https://github.com/DEAP/deap) We think our library is more composable and pythonic while not removing any functionality. Our library may be a bit slower though.
- [... hyperopt?](http://jaberg.github.io/hyperopt/) Since we force the user to make the actual algorithm we are less black boxy. Hyperopt is meant for hyperparameter tuning for machine learning and has better support for search in scikit learn.
- [... inspyred?](https://pypi.org/project/inspyred/) The library offers a simple way to get started but it seems the project is less actively maintained than ours.
================================================
FILE: azure-pipelines.yml
================================================
trigger:
- master
pr:
- master
pool:
vmImage: 'ubuntu-latest'
stages:
- stage: Test
jobs:
- job: TestJob
strategy:
matrix:
Python36:
python.version: '3.6'
Python37:
python.version: '3.7'
Python38:
python.version: '3.8'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '$(python.version)'
- bash: |
pip install --upgrade pip
pip install -e .[dev]
displayName: 'Install'
- bash: flake8
displayName: 'Flake'
- bash: python setup.py test
displayName: 'Tests'
- bash: |
set -e
python examples/simple_nonlinear.py
python examples/number_of_parents.py --n-parents=2 --workers=1
python examples/number_of_parents.py --n-parents=3 --workers=1
python examples/number_of_parents.py --n-parents=4 --workers=1
python examples/number_of_parents.py --n-parents=2 --workers=2
python examples/number_of_parents.py --n-parents=3 --workers=2
python examples/number_of_parents.py --n-parents=4 --workers=2
python examples/very_basic_tsp.py
python examples/simple_logging.py
python examples/rock_paper_scissors.py
- stage: Docs
condition: eq(variables['build.sourceBranch'], 'refs/heads/master')
jobs:
- job: DocsJob
steps:
- bash: |
set -e
pip install --upgrade pip
pip install -e .[docs]
sphinx-apidoc -f -o doc/api evol
sphinx-build doc public
- task: PublishBuildArtifacts@1
inputs:
pathToPublish: public
artifactName: BuildOutput
================================================
FILE: doc/_static/css/custom.css
================================================
.wy-nav-side{
background-color: #f2f2f2;
color: black;
}
.wy-side-nav-search{
background-color: #404040;
}
.wy-nav-content{
background-color: #ffffff;
}
.wy-nav-content-wrap{
background-color: #ffffff;
}
a.reference.internal{
color: black;
}
pre{
background: #eeeeee29;
}
================================================
FILE: doc/api/evol.helpers.combiners.rst
================================================
evol.helpers.combiners package
==============================
Submodules
----------
evol.helpers.combiners.generic module
-------------------------------------
.. automodule:: evol.helpers.combiners.generic
:members:
:undoc-members:
:show-inheritance:
evol.helpers.combiners.permutation module
-----------------------------------------
.. automodule:: evol.helpers.combiners.permutation
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.helpers.combiners
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.helpers.mutators.rst
================================================
evol.helpers.mutators package
=============================
Submodules
----------
evol.helpers.mutators.permutation module
----------------------------------------
.. automodule:: evol.helpers.mutators.permutation
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.helpers.mutators
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.helpers.rst
================================================
evol.helpers package
====================
Subpackages
-----------
.. toctree::
evol.helpers.combiners
evol.helpers.mutators
Submodules
----------
evol.helpers.pickers module
---------------------------
.. automodule:: evol.helpers.pickers
:members:
:undoc-members:
:show-inheritance:
evol.helpers.utils module
-------------------------
.. automodule:: evol.helpers.utils
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.helpers
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.problems.functions.rst
================================================
evol.problems.functions package
===============================
Submodules
----------
evol.problems.functions.variableinput module
--------------------------------------------
.. automodule:: evol.problems.functions.variableinput
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.problems.functions
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.problems.routing.rst
================================================
evol.problems.routing package
=============================
Submodules
----------
evol.problems.routing.coordinates module
----------------------------------------
.. automodule:: evol.problems.routing.coordinates
:members:
:undoc-members:
:show-inheritance:
evol.problems.routing.magicsanta module
---------------------------------------
.. automodule:: evol.problems.routing.magicsanta
:members:
:undoc-members:
:show-inheritance:
evol.problems.routing.tsp module
--------------------------------
.. automodule:: evol.problems.routing.tsp
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.problems.routing
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.problems.rst
================================================
evol.problems package
=====================
Subpackages
-----------
.. toctree::
evol.problems.functions
evol.problems.routing
Submodules
----------
evol.problems.problem module
----------------------------
.. automodule:: evol.problems.problem
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol.problems
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/evol.rst
================================================
evol package
============
Subpackages
-----------
.. toctree::
evol.helpers
evol.problems
Submodules
----------
evol.evolution module
---------------------
.. automodule:: evol.evolution
:members:
:undoc-members:
:show-inheritance:
evol.individual module
----------------------
.. automodule:: evol.individual
:members:
:undoc-members:
:show-inheritance:
evol.logger module
------------------
.. automodule:: evol.logger
:members:
:undoc-members:
:show-inheritance:
evol.population module
----------------------
.. automodule:: evol.population
:members:
:undoc-members:
:show-inheritance:
evol.serialization module
-------------------------
.. automodule:: evol.serialization
:members:
:undoc-members:
:show-inheritance:
evol.step module
----------------
.. automodule:: evol.step
:members:
:undoc-members:
:show-inheritance:
Module contents
---------------
.. automodule:: evol
:members:
:undoc-members:
:show-inheritance:
================================================
FILE: doc/api/modules.rst
================================================
evol
====
.. toctree::
:maxdepth: 4
evol
================================================
FILE: doc/conf.py
================================================
import evol
# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
# full list see the documentation:
# http://www.sphinx-doc.org/en/master/config
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- Project information -----------------------------------------------------
project = 'evol'
copyright = '2019, Vincent D. Warmerdam & Rogier van der Geer'
author = 'Vincent D. Warmerdam & Rogier van der Geer'
# The short X.Y version
version = ''
# The full version, including alpha/beta/rc tags
release = evol.__version__
# -- General configuration ---------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.mathjax',
'sphinx.ext.viewcode',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of doc filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to doc directory, that match files and
# directories to ignore when looking for doc files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = None
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# The default sidebars (for documents that don't match any pattern) are
# defined by theme itself. Builtin themes are using these templates by
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
# 'searchbox.html']``.
#
# html_sidebars = {}
# -- Options for HTMLHelp output ---------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'evoldoc'
# -- Options for LaTeX output ------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (doc start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'evol.tex', 'evol Documentation',
'Vincent D. Warmerdam \\& Rogier van der Geer', 'manual'),
]
# -- Options for manual page output ------------------------------------------
# One entry per manual page. List of tuples
# (doc start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'evol', 'evol Documentation',
[author], 1)
]
# -- Options for Texinfo output ----------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (doc start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'evol', 'evol Documentation',
author, 'evol', 'One line description of project.',
'Miscellaneous'),
]
# -- Options for Epub output -------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = project
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#
# epub_identifier = ''
# A unique identification for the text.
#
# epub_uid = ''
# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']
# -- Extension configuration -------------------------------------------------
def setup(app):
print("Custom part of setup is now running...")
app.add_stylesheet('css/custom.css')
print("Custom part of setup is now complete.")
================================================
FILE: doc/development.rst
================================================
.. image:: https://i.imgur.com/7MHcIq1.png
:align: center
Development
===========
Installing from PyPi
^^^^^^^^^^^^^^^^^^^^
We currently support python3.6 and python3.7 and you can install it via pip.
.. code-block:: bash
pip install evol
Developing Locally with Makefile
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can also fork/clone the repository on Github_ to work on it locally. we've
added a `Makefile` to the project that makes it easy to install everything ready
for development.
.. code-block:: bash
make develop
There's some other helpful commands in there. For example, testing can be done via;
.. code-block:: bash
make test
This will pytest and possibly in the future also the docstring tests.
Generating Documentation
^^^^^^^^^^^^^^^^^^^^^^^^
The easiest way to generate documentation is by running:
.. code-block:: bash
make docs
This will populate the `/docs` folder locally. Note that we ignore the
contents of the this folder per git ignore because building the documentation
is something that we outsource to the read-the-docs service.
.. _Github: https://scikit-learn.org/stable/modules/compose.html
================================================
FILE: doc/index.rst
================================================
.. evol documentation master file, created by
sphinx-quickstart on Thu Apr 4 09:34:54 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
**Evol** is a clear dsl for composable evolutionary algorithms, optimised for joy.
.. image:: _static/evol.png
:align: center
.. code-block:: bash
pip install evol
The Gist
********
The main idea is that you should be able to define a complex algorithm
in a composable way. To explain what we mean by this: let's consider
two evolutionary algorithms for travelling salesman problems.
The first approach takes a collections of solutions and applies:
1. a survival where only the top 50% solutions survive
2. the population reproduces using a crossover of genes
3. certain members mutate
4. repeat this, maybe 1000 times or more!
.. image:: https://i.imgur.com/is9g07u.png
:align: center
We can also think of another approach:
1. pick the best solution of the population
2. make random changes to this parent and generate new solutions
3. repeat this, maybe 1000 times or more!
.. image:: https://i.imgur.com/JRSWbTd.png
:align: center
One could even combine the two algorithms into a new one:
1. run algorithm 1 50 times
2. run algorithm 2 10 times
3. repeat this, maybe 1000 times or more!
.. image:: https://i.imgur.com/SZTBWX2.png
:align: center
You might notice that many parts of these algorithms are similar and it
is the goal of this library is to automate these parts. We hope to
provide an API that is fun to use and easy to tweak your heuristics in.
.. toctree::
:maxdepth: 2
:caption: Contents:
quickstart
population
problems
development
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
================================================
FILE: doc/population.rst
================================================
Population Guide
================
The "Population" object in **evol** is the base container for all your
candidate solutions. Each candidate solution (sometimes referred to
as a chromosome in the literature) lives inside of the population as
an "Individual" and has a fitness score attached as a property.
.. image:: _static/population-1.png
:align: center
You do not typically deal with "Individual" objects directly but it
is useful to know that they are data containers that have a chromosome
property as well as a fitness property.
Creation
********
In order to create a population you need an evaluation function and either:
1. a collection of candidate solutions
2. a function that can generate candidate solutions
Both methods of initialising a population are demonstrated below.
.. literalinclude:: ../examples/population_demo.py
:lines: 1-25
Lazy Evaluation
***************
If we were to now query the contents of the population object
you can use a for loop to view some of the contents.
.. code-block:: python
> [i for i in pop1]
[<individual id:05c4f0 fitness:None>,
<individual id:cdd150 fitness:None>,
<individual id:110e12 fitness:None>,
<individual id:a77886 fitness:None>,
<individual id:8a71e9 fitness:None>]
> [i.chromosome for i in pop1]
[ 0.13942679845788375,
-0.47498924477733306,
-0.22497068163088074,
-0.27678926185117725,
0.2364712141640124]
You might be slightly suprised by the following result though.
.. code-block:: python
> [i.fitness for i in pop1]
[None, None, None, None, None]
The fitness property seems to not exist. But if we call the "evaluate"
method first then suddenly it does seem to make an appearance.
.. code-block:: python
> [i.fitness for i in pop1.evaluate()]
[ 0.2788535969157675,
-0.9499784895546661,
-0.4499413632617615,
-0.5535785237023545,
0.4729424283280248]
There is some logic behind this. Typically the evaluation function
can be very expensive to calculate so you might want to consider running
it as late as possible and as few times as possible. The only command
that needs a fitness is the "survive" method. All other methods can apply
transformations to the chromosome without needing to evaluate the fitness.
More Lazyness
*************
To demonstrate the effect of this lazyness, let's see the effect
of the fitness of the individuals.
First, note that after a survive method everything is evaluated.
.. code-block:: python
> [i.fitness for i in pop1.survive(n=3)]
[0.4729424283280248, 0.2788535969157675, -0.4499413632617615]
If we were to add a mutate step afterwards we will see that the
lazyness kicks in again. Only if we add an evaluate step will we
see fitness values again.
.. code-block:: python
def add_noise(x):
return 0.1 * (random.random() - 0.5) + x
> [i.fitness for i in pop1.survive(n=3).mutate(add_noise)]
[None, None, None]
> [i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()]
[0.3564375534260752, 0.30990154209234466, -0.5356458732043454]
If you want to work with fitness values explicitly it is good to know
about this, otherwise the library will try to be as conservative as
possible when it comes to evaluating the fitness function.
================================================
FILE: doc/problems.rst
================================================
Problem Guide
=============
Certain problems are general enough, if only for educational
purposes, to include into our API. This guide will demonstrate
some of problems that are included in evol.
General Idea
------------
In general a problem in evol is nothing more than an object
that has `.eval_function()` implemented. This object can
usually be initialised in different ways but the method
must always be implemented.
Function Problems
-----------------
There are a few hard functions out there that can be optimised
with heuristics. Our library offers a few objects with this
implementation.
The following functions are implemented.
.. code-block:: python
from evol.problems.functions import Rastrigin, Sphere, Rosenbrock
Rastrigin(size=1).eval_function([1])
Sphere(size=2).eval_function([2, 1])
Rosenbrock(size=3).eval_function([3, 2, 1])
You may notice that we pass a size parameter apon initialisation; this
is because these functions can also be defined in higher dimensions.
Feel free to check the wikipedia_ article for more explanation on these functions.
Routing Problems
----------------
Traveling Salesman Problem
**************************
It's a classic problem so we've included it here.
.. code-block:: python
import random
from evol.problems.routing import TSPProblem, coordinates
us_cities = coordinates.united_states_capitols
problem = TSPProblem.from_coordinates(coordinates=us_cities)
order = list(range(len(us_cities)))
for i in range(3):
random.shuffle(order)
print(problem.eval_function(order))
Note that you can also create an instance of a TSP problem
from a distance matrix instead. Also note that you can get
such a distance matrix from the object.
.. code:: python
same_problem = TSPProblem(problem.distance_matrix)
print(same_problem.eval_function(order))
Magic Santa
***********
This problem was inspired by a kaggle_ competition. It involves the logistics
of delivering gifts all around the world from the north pole. The costs of
delivering a gift depend on how tired santa's reindeer get while delivering
a sleigh full of gifts during a trip.
It is better explained on the website than here but the goal is to
minimize the weighed reindeer weariness defined below:
:math:`WRW = \sum\limits_{j=1}^{m} \sum\limits_{i=1}^{n} \Big[ \big( \sum\limits_{k=1}^{n} w_{kj} - \sum\limits_{k=1}^{i} w_{kj} \big) \cdot Dist(Loc_i, Loc_{i-1})`
In terms of setting up the problem it is very similar to a TSP except that
we now also need to attach the weight of a gift per location.
.. code:: python
import random
from evol.problems.routing import MagicSanta, coordinates
us_cities = coordinates.united_states_capitols
problem = TSPProblem.from_coordinates(coordinates=us_cities)
MagicSanta(city_coordinates=us_cities,
home_coordinate=(0, 0),
gift_weight=[random.random() for _ in us_cities])
.. _wikipedia: https://en.wikipedia.org/wiki/Test_functions_for_optimization
.. _kaggle: https://www.kaggle.com/c/santas-stolen-sleigh#evaluation
================================================
FILE: doc/quickstart.rst
================================================
Quick-Start Guide
=================
The goal is this document is to build the pipeline you see below.
.. image:: _static/quickstart-step-6.png
:align: center
This guide will offer a step by step guide on how to use evol
to write custom heuristic solutions to problems. As an example
we will try to optimise the following non-linear function:
:math:`f(x, y) = -(1-x)^2 - (2 - y^2)^2`
Step 1: Score
^^^^^^^^^^^^^
The first thing we need to do for evol is to describe how
"good" a solution to a problem is. To facilitate this we
can write a simple function.
.. literalinclude:: ../examples/simple_nonlinear.py
:lines: 15-20
You'll notice that this function accepts a "solution" to
the problem and it returns a value. In this case the "solution"
is a list that contains two elements. Inside the function we
unpack it but the function that we have needs to accept one
"candidate"-solution and return one score.
Step 2: Sample
^^^^^^^^^^^^^^
Another thing we need is something that can create
random candidates. We want our algorithm to start searching
somewhere and we prefer to start with different candidates
instead of a static set. The function below will generate
such candidates.
.. literalinclude:: ../examples/simple_nonlinear.py
:lines: 8-12
Note that one candidate from this function will create
a tuple; one that can be unpacked by the function we've defined
before.
Step 3: Create
^^^^^^^^^^^^^^
With these two functions we can create a population of
candidates. Below we generate a population with 200 random
candidates.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 54-55
This population object is merely a container for candidates.
The next step is to define things that we might want to do
with it.
If we were to draw where we currently are, it'd be here:
.. image:: _static/quickstart-step-1.png
:align: center
Step 4: Survive
^^^^^^^^^^^^^^^
Now that we have a population we might add a bit of code that
can remove candidates that are not performing as well. This means
that we add a step to our "pipeline".
.. image:: _static/quickstart-step-2.png
:align: center
To facilitate this we merely need to call a method on our population object.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 57-58
Because the population knows what it needs to optimise for
it is easy to halve the population size by simply calling this method.
This method call will return a new population object that has fewer
members. A next step might be to take these remaining candidates and
to use them to create new candidates that are similar.
Step 5: Breed
^^^^^^^^^^^^^
In order to evolve the candidates we need to start generating
new candites again. This adds another step to our pipeline:
.. image:: _static/quickstart-step-3.png
:align: center
Note that in this view the highlighted candidates are the new ones
that have been created. The candidates who were already performing
very well are still in the population.
To generate new candidates we need to do two things:
1. we need to determine what parents will be used to create a new individual
2. we need to determine how these parent candidates create a new one
Both steps needs to be defined in functions. First, we write
a simple function that will select two random parents from
the population.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 23-29
Next we need a function that can merge the properties of these
two parents such that we create a new candidate.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 32-41
With these two functions we can expand our initial pipeline
and expand it with a breed step.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 60-63
Step 6: Mutate
^^^^^^^^^^^^^^
Typically when searching for a good candidate we might want
to add some entropy in the system. The idea being that a bit
of random search might indeed help us explore areas that we
might not otherwise consider.
.. image:: _static/quickstart-step-4.png
:align: center
The idea is to add a bit of noise to every single datapoint.
This ensures that our population of candidates does not converge
too fast towards a single datapoint and that we are able to
explore the search space.
To faciliate this in our pipeline we first need to create
a function that can take a candidate and apply the noise.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 44-50
Next we need to add this as a step in our pipeline.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 65-69
Step 7: Repeat
^^^^^^^^^^^^^^
We're getting really close to where we want to be now but
we still need to discuss how to repeat our steps.
.. image:: _static/quickstart-step-5.png
:align: center
One way of getting there is to literally repeat the code
we saw earlier in a for loop.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 71-76
This sort of works, but there is a more elegant method.
Step 8: Evolve
^^^^^^^^^^^^^^
The problem with the previous method is that we don't
just want to repeat but we also want to supply settings
to our evolution steps that might change over time. To
facilitate this our api offers the `Evolution` object.
.. image:: _static/quickstart-step-6.png
:align: center
You can see a `Population` as a container for candidates
and can `Evolution` as a container for changes to the
population. You can use the exact same verbs in the method
chain to specify what you'd like to see happen but it allows
you much more fledixbility.
The code below demonstrates an example of evolution steps.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 78-82
The code below demonstrates a slightly different set of steps.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 84-88
Evolutions are kind of flexible, we can combine these two
evolutions into a third one.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 90-96
Now if you'd like to apply this evolution we've added a method
for that on top of our evolution object.
.. literalinclude:: ../examples/simple_nonlinear.py
:language: python
:lines: 98-101
Step 9: Dream
^^^^^^^^^^^^^^
These steps together give us an evolution program depicted below.
.. image:: _static/quickstart-step-7.png
:align: center
The goal of evol is to make it easy to write heuristic pipelines
that can help search towards a solution. Note that you don't need
to write a genetic algorithm here. You could also implement simulated
annealing in our library just as easily but we want to help you standardise
your code such that testing, monitoring, parallism and checkpoint becomes
more joyful.
Evol will help you structure your pipeline by giving a language that
tells you *what* is happening but not *how* this is being done. For this
you will need to write functions yourself because our library has no
notion of your specific problem.
We hope this makes writing heuristic software more fun.
================================================
FILE: evol/__init__.py
================================================
"""
`Evol` is clear dsl for composable evolutionary algorithms that optimised for joy.
Evol is a library that helps you make evolutionary algorithms. The
goal is to have a library that is fun and clear to use, but not fast.
If you're looking at the library for the first time we recommend
that you first take a look at the examples in the /examples folder
on github. It is usually a better starting point to get started.
Any details can be discovered on the docs. We hope that this
library makes it fun to write heuristics again.
The Gist
---------------------------------------
The main idea is that you should be able to define a complex algorithm
in a composable way. To explain what we mean by this: let's consider
two evolutionary algorithms for travelling salesman problems.
The first approach takes a collections of solutions and applies:
1. a survival where only the top 50% solutions survive
2. the population reproduces using a crossover of genes
3. certain members mutate
4. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/is9g07u.png" alt="Drawing" style="width: 100%;"/>
We can also think of another approach:
1. pick the best solution of the population
2. make random changes to this parent and generate new solutions
3. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/JRSWbTd.png" alt="Drawing" style="width: 100%;"/>
One could even combine the two algorithms into a new one:
1. run algorithm 1 50 times
2. run algorithm 2 10 times
3. repeat this, maybe 1000 times or more!
<img src="https://i.imgur.com/SZTBWX2.png" alt="Drawing" style="width: 100%;"/>
You might notice that many parts of these algorithms are similar and it is
the goal of this library is to automate these parts. In fact, you can
expect the code for these algorithms to look something like this.
A speudo-example of what is decribed about looks a bit like this:
import random
from evol import Population, Evolution
population = Population(init_func=init_func, eval_func=eval_func, size=100)
def pick_n_parents(population, num_parents):
return [random.choice(population) for i in range(num_parents)]
def crossover(*parents):
...
def random_copy(parent):
...
evo1 = (Evolution(name="first_algorithm")
.survive(fraction=0.5)
.breed(parentpicker=pick_n_parents,
combiner=combiner,
num_parents=2, n_max=100)
.mutate(lambda x: add_noise(x, 0.1)))
evo2 = (Evolution(name="second_algorithm")
.survive(n=1)
.breed(parentpicker=pick_n_parents,
combiner=random_copy,
num_parents=1, n_max=100))
for i in range(1001):
population.evolve(evo1, n=50).evolve(evo2, n=10)
Getting Started
---------------------------------------
The best place to get started is the `/examples` folder on github.
This folder contains self contained examples that work out of the
box.
Contributing Guide
---------------------------------------
### Local development
Python can help you. Don't reinstall all the time, rather use a
virtulenv that has a link to the code.
python setup.py develop
When you submit a pull request it will be tested in travis. Once
the build is green the review can start. Please try to keep your
branch up to date to ensure you're not missing any tests. Larger
commits need to be discussed in github before we can accept them.
### Generating New Documentation
Updating documentation is currently a manual step. From the `docs` folder:
pdoc --html --overwrite evol
cp -rf evol/* .
rm -rf evol
If you want to confirm that it works you can open the `index.html` file.
"""
from .individual import Individual
from .population import Population, ContestPopulation
from .evolution import Evolution
from .logger import BaseLogger
__version__ = "0.5.3"
================================================
FILE: evol/conditions.py
================================================
from time import monotonic
from typing import Callable, Optional, TYPE_CHECKING
from evol.exceptions import StopEvolution
if TYPE_CHECKING:
from evol.population import BasePopulation
class Condition:
"""Stop the evolution until a condition is no longer met.
:param condition: A function that accepts a Population and returns a boolean.
If the condition does not evaluate to True, then the evolution is stopped.
"""
conditions = set()
def __init__(self, condition: Optional[Callable[['BasePopulation'], bool]]):
self.condition = condition
def __enter__(self):
self.conditions.add(self)
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.conditions.remove(self)
def __call__(self, population: 'BasePopulation') -> None:
if self.condition and not self.condition(population):
raise StopEvolution()
@classmethod
def check(cls, population: 'BasePopulation'):
for condition in cls.conditions:
condition(population)
class MinimumProgress(Condition):
"""Stop the evolution if not enough progress is made.
This condition stops the evolution if the best documented fitness
does not improve enough within a given number of iterations.
:param window: Number of iterations in which the minimum improvement must be made.
:param change: Require more change in fitness than this value.
Defaults to 0, meaning any change is good enough.
"""
def __init__(self, window: int, change: float = 0):
super().__init__(condition=None)
self._history = []
self.change = change
self.window = window
def __call__(self, population: 'BasePopulation') -> None:
self._history = self._history[-self.window:]
self._history.append(population.evaluate(lazy=True).documented_best.fitness)
if len(self._history) > self.window and abs(self._history[0] - self._history[-1]) <= self.change:
raise StopEvolution()
class TimeLimit(Condition):
"""Stop the evolution after a given amount of time.
This condition stops the evolution after a given amount of time
has elapsed. Note that the time is only checked between iterations.
If your iterations take long, the evolution may potentially run
for much longer than anticipated.
:param seconds: The time in seconds that the evolution may run.
"""
def __init__(self, seconds: float):
super().__init__(condition=None)
self.time = None
self.seconds = seconds
def __call__(self, population: 'BasePopulation'):
if self.time is None:
self.time = monotonic()
elif monotonic() - self.time > self.seconds:
raise StopEvolution()
================================================
FILE: evol/evolution.py
================================================
"""
Evolution objects in `evol` are objects that describe how the
evolutionary algorithm will change members of a population.
Evolution objects contain the same methods as population objects
but because an evolution is separate from a population you can
play around with them more easily.
"""
from copy import copy
from typing import Any, Callable, Iterator, List, Optional, Sequence
from evol import Individual
from .step import CheckpointStep, CallbackStep, EvolutionStep
from .step import EvaluationStep, MapStep, FilterStep
from .step import SurviveStep, BreedStep, MutateStep, RepeatStep
class Evolution:
"""Describes the process a Population goes through when evolving."""
def __init__(self):
self.chain: List[EvolutionStep] = []
def __copy__(self) -> 'Evolution':
result = Evolution()
result.chain = copy(self.chain)
return result
def __iter__(self) -> Iterator[EvolutionStep]:
return self.chain.__iter__()
def __repr__(self) -> str:
result = 'Evolution('
for step in self:
result += '\n ' + repr(step).replace('\n', '\n ')
result += ')'
return result.strip('\n')
def evaluate(self, lazy: bool = False, name: Optional[str] = None) -> 'Evolution':
"""Add an evaluation step to the Evolution.
This evaluates the fitness of all individuals. If lazy is True, the
fitness is only evaluated when a fitness value is not yet known. In
most situations adding an explicit evaluation step is not needed, as
lazy evaluation is implicitly included in the steps that need it (most
notably in the survive step).
:param lazy: If True, do no re-evaluate the fitness if the fitness is known.
:param name: Name of the evaluation step.
:return: This Evolution with an additional step.
"""
return self._add_step(EvaluationStep(name=name, lazy=lazy))
def checkpoint(self,
target: Optional[str] = None,
method: str = 'pickle',
name: Optional[str] = None,
every: int = 1) -> 'Evolution':
"""Add a checkpoint step to the Evolution.
:param target: Directory to write checkpoint to. If None, the Serializer default target is taken,
which can be provided upon initialisation. Defaults to None.
:param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.
Defaults to 'pickle'.
:param name: Name of the map step.
:param every: Checkpoint once every 'every' iterations. Defaults to 1.
"""
return self._add_step(CheckpointStep(name=name, target=target, method=method, every=every))
def map(self, func: Callable[..., Individual], name: Optional[str] = None, **kwargs) -> 'Evolution':
"""Add a map step to the Evolution.
This applies the provided function to each individual in the
population, in place.
:param func: Function to apply to the individuals in the population.
:param name: Name of the map step.
:param kwargs: Arguments to pass to the function.
:return: This Evolution with an additional step.
"""
return self._add_step(MapStep(name=name, func=func, **kwargs))
def filter(self, func: Callable[..., bool], name: Optional[str] = None, **kwargs) -> 'Evolution':
"""Add a filter step to the Evolution.
This filters the individuals in the population using the provided function.
:param func: Function to filter the individuals in the population by.
:param name: Name of the filter step.
:param kwargs: Arguments to pass to the function.
:return: This Evolution with an additional step.
"""
return self._add_step(FilterStep(name=name, func=func, **kwargs))
def survive(self,
fraction: Optional[float] = None,
n: Optional[int] = None,
luck: bool = False,
name: Optional[str] = None,
evaluate: bool = True) -> 'Evolution':
"""Add a survive step to the Evolution.
This filters the individuals in the population according to fitness.
:param fraction: Fraction of the original population that survives.
Defaults to None.
:param n: Number of individuals of the population that survive.
Defaults to None.
:param luck: If True, individuals randomly survive (with replacement!)
with chances proportional to their fitness. Defaults to False.
:param name: Name of the filter step.
:param evaluate: If True, add a lazy evaluate step before the survive step.
Defaults to True.
:return: This Evolution with an additional step.
"""
if evaluate:
after_evaluate = self.evaluate(lazy=True)
else:
after_evaluate = self
return after_evaluate._add_step(SurviveStep(name=name, fraction=fraction, n=n, luck=luck))
def breed(self,
parent_picker: Callable[..., Sequence[Individual]],
combiner: Callable,
population_size: Optional[int] = None,
name: Optional[str] = None,
**kwargs) -> 'Evolution':
"""Add a breed step to the Evolution.
Create new individuals by combining existing individuals.
:param parent_picker: Function that selects parents.
:param combiner: Function that combines chromosomes into a new
chromosome. Must be able to handle the number of chromosomes
that the combiner returns.
:param population_size: Intended population size after breeding.
If None, take the previous intended population size.
Defaults to None.
:param name: Name of the breed step.
:param kwargs: Kwargs to pass to the parent_picker and combiner.
Arguments are only passed to the functions if they accept them.
:return: self
"""
return self._add_step(BreedStep(name=name, parent_picker=parent_picker, combiner=combiner,
population_size=population_size, **kwargs))
def mutate(self,
mutate_function: Callable[..., Any],
probability: float = 1.0,
elitist: bool = False,
name: Optional[str] = None,
**kwargs) -> 'Evolution':
"""Add a mutate step to the Evolution.
This mutates the chromosome of each individual.
:param mutate_function: Function that accepts a chromosome and returns
a mutated chromosome.
:param probability: Probability that the individual mutates.
The function is only applied in the given fraction of cases.
Defaults to 1.0.
:param elitist: If True, do not mutate the current best individual(s).
Note that this only applies to evaluated individuals. Any unevaluated
individual will be treated as normal.
Defaults to False.
:param name: Name of the mutate step.
:param kwargs: Kwargs to pass to the parent_picker and combiner.
Arguments are only passed to the functions if they accept them.
:return: self
"""
return self._add_step(MutateStep(name=name, probability=probability, elitist=elitist,
mutate_function=mutate_function, **kwargs))
def repeat(self, evolution: 'Evolution', n: int = 1, name: Optional[str] = None,
grouping_function: Optional[Callable] = None, **kwargs) -> 'Evolution':
"""Add an evolution as a step to this evolution.
This will add a step to the evolution that repeats another evolution
several times. Optionally this step can be performed in groups.
Note: if your population uses multiple concurrent workers and you use grouping,
any callbacks inside the evolution you apply here may not have the desired effect.
:param evolution: Evolution to apply.
:param n: Number of times to perform the evolution. Defaults to 1.
:param name: Name of the repeat step.
:param grouping_function: Optional function to use for grouping the population.
You can find built-in grouping functions in evol.helpers.groups.
:param kwargs: Kwargs to pass to the grouping function, for example n_groups.
:return: self
"""
return self._add_step(RepeatStep(name=name, evolution=evolution, n=n,
grouping_function=grouping_function, **kwargs))
def callback(self, callback_function: Callable[..., Any],
every: int = 1, name: Optional[str] = None, **kwargs) -> 'Evolution':
"""Call a function as a step in this evolution.
This will call the provided function with the population as argument.
Note that you can raise evol.exceptions.StopEvolution from within the
callback to stop further evolution.
:param callback_function: Function to call.
:param every: Only call the function once per `every` iterations.
Defaults to 1; every iteration.
:param name: Name of the callback step.
:return: self
"""
return self._add_step(CallbackStep(name=name, every=every, callback_function=callback_function, **kwargs))
def _add_step(self, step: EvolutionStep) -> 'Evolution':
result = copy(self)
result.chain.append(step)
return result
================================================
FILE: evol/exceptions.py
================================================
class PopulationIsNotEvaluatedException(RuntimeError):
pass
class StopEvolution(Exception):
pass
================================================
FILE: evol/helpers/__init__.py
================================================
"""
Helpers in `evol` are functions that help you when you are
designing algorithms. We archive these helping functions per usecase.
"""
================================================
FILE: evol/helpers/combiners/__init__.py
================================================
================================================
FILE: evol/helpers/combiners/permutation.py
================================================
from itertools import islice, tee
from random import choice
from typing import Any, Tuple
from .utils import select_node, construct_neighbors, identify_cycles, cycle_parity
from ..utils import select_partition
def order_one_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple:
"""Combine two chromosomes using order-1 crossover.
http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/Order1CrossoverOperator.aspx
:param parent_1: First parent.
:param parent_2: Second parent.
:return: Child chromosome.
"""
start, end = select_partition(len(parent_1))
selected_partition = parent_1[start:end + 1]
remaining_elements = filter(lambda element: element not in selected_partition, parent_2)
return tuple(islice(remaining_elements, 0, start)) + selected_partition + tuple(remaining_elements)
def edge_recombination(*parents: Tuple) -> Tuple:
"""Combine multiple chromosomes using edge recombination.
http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/EdgeRecombinationCrossoverOperator.aspx
:param parents: Chromosomes to combine.
:return: Child chromosome.
"""
return tuple(select_node(
start_node=choice([chromosome[0] for chromosome in parents]),
neighbors=construct_neighbors(*parents)
))
def cycle_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple[Tuple[Any, ...], ...]:
"""Combine two chromosomes using cycle crossover.
http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/CycleCrossoverOperator.aspx
:param parent_1: First parent.
:param parent_2: Second parent.
:return: Child chromosome.
"""
cycles = identify_cycles(parent_1, parent_2)
parity = cycle_parity(cycles=cycles)
it_a, it_b = tee((b, a) if parity[i] else (a, b) for i, (a, b) in enumerate(zip(parent_1, parent_2)))
yield tuple(x[0] for x in it_a)
yield tuple(y[1] for y in it_b)
================================================
FILE: evol/helpers/combiners/utils.py
================================================
from collections import defaultdict
from itertools import tee, islice, cycle
from random import choice
from typing import Iterable, Generator, Any, Set, List, Dict, Tuple
def construct_neighbors(*chromosome: Tuple[Any]) -> defaultdict:
result = defaultdict(set)
for element in chromosome:
for x, y in _neighbors_in(element):
result[x].add(y)
result[y].add(x)
return result
def _neighbors_in(x: Tuple[Any], cyclic=True) -> Iterable[Tuple[Any, Any]]:
a, b = tee(islice(cycle(x), 0, len(x) + (1 if cyclic else 0)))
next(b, None)
return zip(a, b)
def _remove_from_neighbors(neighbors, node):
del neighbors[node]
for _, element in neighbors.items():
element.difference_update({node})
def select_node(start_node: Any, neighbors: defaultdict) -> Generator[Any, None, None]:
node = start_node
yield node
while len(neighbors) > 1:
options = neighbors[node]
_remove_from_neighbors(neighbors, node)
if len(options) > 0:
min_len = min([len(neighbors[option]) for option in options])
node = choice([option for option in options if len(neighbors[option]) == min_len])
else:
node = choice(list(neighbors.keys()))
yield node
def identify_cycles(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any]) -> List[Set[int]]:
"""Identify all cycles between the chromosomes.
A cycle is found by following this procedure: given an index, look up the
value in the first chromosome. Then find the index of that value in the
second chromosome. Repeat, until one returns to the original index.
:param chromosome_1: First chromosome.
:param chromosome_2: Second chromosome.
:return: A list of cycles.
"""
indices = set(range(len(chromosome_1)))
cycles = []
while len(indices) > 0:
next_cycle = _identify_cycle(chromosome_1=chromosome_1, chromosome_2=chromosome_2, start_index=min(indices))
indices.difference_update(next_cycle)
cycles.append(next_cycle)
return cycles
def _identify_cycle(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any], start_index: int = 0) -> Set[int]:
"""Identify a cycle between the chromosomes starting at the provided index.
A cycle is found by following this procedure: given an index, look up the
value in the first chromosome. Then find the index of that value in the
second chromosome. Repeat, until one returns to the original index.
:param chromosome_1: First chromosome.
:param chromosome_2: Second chromosome.
:param start_index: Index to start. Defaults to 0.
:return: The set of indices in the identified cycle.
"""
indices = set()
index = start_index
while index not in indices:
indices.add(index)
value = chromosome_1[index]
index = chromosome_2.index(value)
return indices
def cycle_parity(cycles: List[Set[int]]) -> Dict[int, bool]:
"""Create a dictionary with the cycle parity of each index.
Indices in all odd cycles have parity False, while
indices in even cycles have parity True."""
return {index: bool(i % 2) for i, c in enumerate(cycles) for index in c}
================================================
FILE: evol/helpers/groups.py
================================================
from random import shuffle
from typing import List
from evol import Individual
from evol.exceptions import PopulationIsNotEvaluatedException
"""
Below are functions that allocate individuals to the
island populations. It will be passed a list of individuals plus
the kwargs passed to this method, and must return a list of lists
of integers, each sub-list representing an island and the integers
representing the index of an individual in the list. Each island
must contain at least one individual, and individual may be copied
to multiple islands.
"""
def group_duplicate(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:
"""
Group individuals into groups that each contain all individuals.
:param individuals: List of individuals to group.
:param n_groups: Number of groups to make.
:return: List of lists of ints
"""
return [list(range(len(individuals))) for _ in range(n_groups)]
def group_random(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:
"""
Group individuals randomly into groups of roughly equal size.
:param individuals: List of individuals to group.
:param n_groups: Number of groups to make.
:return: List of lists of ints
"""
indexes = list(range(len(individuals)))
shuffle(indexes)
return [indexes[i::n_groups] for i in range(n_groups)]
def group_stratified(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:
"""
Group individuals into groups of roughly equal size in a stratified manner.
This function groups such that each group contains individuals of
higher as well as lower fitness. This requires the individuals to have a fitness.
:param individuals: List of individuals to group.
:param n_groups: Number of groups to make.
:return: List of lists of ints
"""
_ensure_evaluated(individuals)
indexes = list(map(
lambda index_and_individual: index_and_individual[0],
sorted(enumerate(individuals), key=lambda index_and_individual: index_and_individual[1].fitness)
))
return [indexes[i::n_groups] for i in range(n_groups)]
def _ensure_evaluated(individuals: List[Individual]):
"""
Helper function to ensure individuals are evaluated.
:param individuals: List of individuals
:raises RuntimeError: When at least one of the individuals is not evaluated.
"""
for individual in individuals:
if individual.fitness is None:
raise PopulationIsNotEvaluatedException('Population must be evaluated.')
================================================
FILE: evol/helpers/mutators/__init__.py
================================================
================================================
FILE: evol/helpers/mutators/permutation.py
================================================
from random import sample
from typing import Any, Tuple
from ..utils import select_partition
def inversion(chromosome: Tuple[Any, ...], min_size: int = 2, max_size: int = None) -> Tuple[Any, ...]:
"""Mutate a chromosome using inversion.
Inverts a random partition of the chromosome.
:param chromosome: Original chromosome.
:param min_size: Minimum partition size. Defaults to 2.
:param max_size: Maximum partition size. Defaults to length - 1.
:return: Mutated chromosome.
"""
start, end = select_partition(len(chromosome), min_size, max_size)
return chromosome[:start] + tuple(reversed(chromosome[start:end])) + chromosome[end:]
def swap_elements(chromosome: Tuple[Any, ...]) -> Tuple[Any, ...]:
"""Randomly swap two elements of the chromosome.
:param chromosome: Original chromosome.
:return: Mutated chromosome.
"""
result = list(chromosome)
index_1, index_2 = sample(range(len(chromosome)), 2)
result[index_1], result[index_2] = result[index_2], result[index_1]
return tuple(result)
================================================
FILE: evol/helpers/pickers.py
================================================
from typing import Sequence, Tuple
from random import choice
from evol import Individual
def pick_random(parents: Sequence[Individual], n_parents: int = 2) -> Tuple:
"""Randomly selects parents with replacement
Accepted arguments:
n_parents: Number of parents to select. Defaults to 2.
"""
return tuple(choice(parents) for _ in range(n_parents))
================================================
FILE: evol/helpers/utils.py
================================================
from random import randint
from typing import Tuple
def select_partition(length: int, min_size: int = 1, max_size: int = None) -> Tuple[int, int]:
"""Select a partition of a chromosome.
:param length: Length of the chromosome.
:param min_size: Minimum length of the partition. Defaults to 1.
:param max_size: Maximum length of the partition. Defaults to length - 1.
:return: Start and end index of the partition.
"""
partition_size = randint(min_size, length - 1 if max_size is None else max_size)
partition_start = randint(0, length - partition_size)
return partition_start, partition_start + partition_size
def rotating_window(arr):
"""rotating_window([1,2,3,4]) -> [(4,1), (1,2), (2,3), (3,4)]"""
for i, city in enumerate(arr):
yield arr[i - 1], arr[i]
def sliding_window(arr):
"""sliding_window([1,2,3,4]) -> [(1,2), (2,3), (3,4)]"""
for i, city in enumerate(arr[:-1]):
yield arr[i], arr[i + 1]
================================================
FILE: evol/individual.py
================================================
"""
Individual objects in `evol` are a wrapper around a chromosome.
Internally we work with individuals because that allows us to
separate the fitness calculation from the data structure. This
saves a lot of CPU power.
"""
from random import random
from typing import Any, Callable, Optional
from uuid import uuid4
class Individual:
"""Represents an individual in a population. The individual has a chromosome.
:param chromosome: The chromosome of the individual.
:param fitness: The fitness of the individual, or None.
Defaults to None.
"""
def __init__(self, chromosome: Any, fitness: Optional[float] = None):
self.age = 0
self.chromosome = chromosome
self.fitness = fitness
self.id = f"{str(uuid4())[:6]}"
def __repr__(self):
return f"<individual id:{self.id} fitness:{self.fitness}>"
@classmethod
def from_dict(cls, data: dict) -> 'Individual':
"""Load an Individual from a dictionary.
:param data: Dictionary containing the keys 'age', 'chromosome', 'fitness' and 'id'.
:return: Individual
"""
result = cls(chromosome=data['chromosome'], fitness=data['fitness'])
result.age = data['age']
result.id = data['id']
return result
def __post_evaluate(self, result):
self.fitness = result
def evaluate(self, eval_function: Callable[..., float], lazy: bool = False):
"""Evaluate the fitness of the individual.
:param eval_function: Function that reduces a chromosome to a fitness.
:param lazy: If True, do no re-evaluate the fitness if the fitness is known.
"""
if self.fitness is None or not lazy:
self.fitness = eval_function(self.chromosome)
def mutate(self, mutate_function: Callable[..., Any], probability: float = 1.0, **kwargs):
"""Mutate the chromosome of the individual.
:param mutate_function: Function that accepts a chromosome and returns a mutated chromosome.
:param probability: Probability that the individual mutates.
The function is only applied in the given fraction of cases.
Defaults to 1.0.
:param kwargs: Arguments to pass to the mutation function.
"""
if probability == 1.0 or random() < probability:
self.chromosome = mutate_function(self.chromosome, **kwargs)
self.fitness = None
================================================
FILE: evol/logger.py
================================================
"""
Loggers help keep track of the workings of your evolutionary algorithm. By
default, each Population is initialized with a BaseLogger, which you can use
by using the .log() method of the population. If you want more complex
behaviour, you can supply another logger to the Population on initialisation.
"""
import datetime as dt
import os
import json
import logging
import sys
import uuid
from evol.exceptions import PopulationIsNotEvaluatedException
from evol.population import BasePopulation
class BaseLogger:
"""
The `evol.BaseLogger` is the most basic logger in evol.
You can supply it to a population so that the population
knows how to handle the `.log()` verb.
"""
def __init__(self, target=None, stdout=False, fmt='%(asctime)s,%(message)s'):
self.file = target
if target is not None:
if not os.path.exists(os.path.split(target)[0]):
raise RuntimeError(f"path to target {os.path.split(target)[0]} does not exist!")
formatter = logging.Formatter(fmt=fmt, datefmt='%Y-%m-%d %H:%M:%S')
self.logger = logging.getLogger(name=f"{uuid.uuid4()}")
if not self.logger.handlers:
# we do this extra step because loggers can behave in strange ways otherwise
# https://navaspot.wordpress.com/2015/09/22/same-log-messages-multiple-times-in-python-issue/
if target:
file_handler = logging.FileHandler(filename=target)
file_handler.setFormatter(fmt=formatter)
self.logger.addHandler(file_handler)
if stdout:
stream_handler = logging.StreamHandler(stream=sys.stdout)
stream_handler.setFormatter(fmt=formatter)
self.logger.addHandler(stream_handler)
self.logger.setLevel(level=logging.INFO)
@staticmethod
def check_population(population: BasePopulation) -> None:
if not population.is_evaluated:
raise PopulationIsNotEvaluatedException('Population must be evaluated when logging.')
def log(self, population, **kwargs):
"""
The logger method of the Logger object determines what will be logged.
:param population: `evol.Population` object
:return: nothing, it merely logs to a file and perhaps stdout
"""
self.check_population(population)
values = ','.join([str(item) for item in kwargs.values()])
if values != '':
values = f',{values}'
for i in population:
self.logger.info(f'{population.id},{i.id},{i.fitness}' + values)
class SummaryLogger(BaseLogger):
"""
The `evol.SummaryLogger` merely logs statistics per population and nothing else.
You are still able to log to stdout as well.
"""
def log(self, population, **kwargs):
self.check_population(population)
values = ','.join([str(item) for item in kwargs.values()])
if values != '':
values = f',{values}'
fitnesses = [i.fitness for i in population]
self.logger.info(f'{min(fitnesses)},{sum(fitnesses) / len(fitnesses)},{max(fitnesses)}' + values)
class MultiLogger:
"""
The `evol.Multilogger` is a logger object that can handle writing to two files.
It is here for demonstration purposes to show how you could customize the logging.
The only thing that matters is that all logging is handled by the `.log()`
call. So we are free to record to multiple files if we want as well. This is
not per se best practice but it would work.
"""
def __init__(self, file_individuals, file_population):
self.file_individuals = file_individuals
self.file_population = file_population
def log(self, population, **kwargs):
"""
The logger method of the Logger object determines what will be logged.
:param population: population to log
:return: generator of strings to be handled
"""
ind_generator = (f'{dt.datetime.now()},{population.id},{i.id},{i.fitness}' for i in population)
fitnesses = [i.fitness for i in population]
data = {
'ts': str(dt.datetime.now()),
'mean_ind': sum(fitnesses) / len(fitnesses),
'min_ind': min(fitnesses),
'max_ind': max(fitnesses)
}
dict_to_log = {**kwargs, **data}
self.handle(ind_generator, dict_to_log)
def handle(self, ind_generator, dict_to_log):
"""
The handler method of the Logger object determines how it will be logged.
In this case we print if there is no file and we append to a file otherwise.
"""
with open(self.file_population, 'a') as f:
f.write(json.dumps(dict_to_log))
with open(self.file_population, 'a') as f:
f.writelines(ind_generator)
================================================
FILE: evol/population.py
================================================
"""
Population objects in `evol` are a collection of chromosomes
at some point in an evolutionary algorithm. You can apply
evolutionary steps by directly calling methods on the population
or by applying an `evol.Evolution` object.
"""
from abc import ABCMeta, abstractmethod
from copy import copy
from itertools import cycle, islice
from math import ceil
from random import choices, randint
from typing import Any, Callable, Generator, Iterable, Iterator, List, Optional, Sequence, TYPE_CHECKING
from uuid import uuid4
from multiprocess.pool import Pool
from evol import Individual
from evol.conditions import Condition
from evol.exceptions import StopEvolution
from evol.helpers.groups import group_random
from evol.utils import offspring_generator, select_arguments
from evol.serialization import SimpleSerializer
if TYPE_CHECKING:
from .evolution import Evolution
class BasePopulation(metaclass=ABCMeta):
def __init__(self,
chromosomes: Iterable[Any],
eval_function: Callable,
checkpoint_target: Optional[str] = None,
concurrent_workers: Optional[int] = 1,
maximize: bool = True,
generation: int = 0,
intended_size: Optional[int] = None,
serializer=None):
self.concurrent_workers = concurrent_workers
self.documented_best = None
self.eval_function = eval_function
self.generation = generation
self.id = str(uuid4())[:6]
self.individuals = [Individual(chromosome=chromosome) for chromosome in chromosomes]
self.intended_size = intended_size or len(self.individuals)
self.maximize = maximize
self.serializer = serializer or SimpleSerializer(target=checkpoint_target)
self.pool = None if concurrent_workers == 1 else Pool(concurrent_workers)
def __iter__(self) -> Iterator[Individual]:
return self.individuals.__iter__()
def __getitem__(self, i) -> Individual:
return self.individuals[i]
def __len__(self):
return len(self.individuals)
def __repr__(self):
return f"<Population with size {len(self)} at {id(self)}>"
@property
def current_best(self) -> Individual:
evaluated_individuals = tuple(filter(lambda x: x.fitness is not None, self.individuals))
if len(evaluated_individuals) > 0:
return max(evaluated_individuals, key=lambda x: x.fitness if self.maximize else -x.fitness)
@property
def current_worst(self) -> Individual:
evaluated_individuals = tuple(filter(lambda x: x.fitness is not None, self.individuals))
if len(evaluated_individuals) > 0:
return min(evaluated_individuals, key=lambda x: x.fitness if self.maximize else -x.fitness)
@property
def chromosomes(self) -> Generator[Any, None, None]:
for individual in self.individuals:
yield individual.chromosome
@property
def is_evaluated(self) -> bool:
return all(individual.fitness is not None for individual in self)
@classmethod
def generate(cls,
init_function: Callable[[], Any],
eval_function: Callable[..., float],
size: int = 100,
**kwargs) -> 'BasePopulation':
"""Generate a population from an initialisation function.
:param init_function: Function that returns a chromosome.
:param eval_function: Function that reduces a chromosome to a fitness.
:param size: Number of individuals to generate. Defaults to 100.
:return: BasePopulation
"""
chromosomes = [init_function() for _ in range(size)]
return cls(chromosomes=chromosomes, eval_function=eval_function, **kwargs)
@classmethod
def load(cls,
target: str,
eval_function: Callable[..., float],
**kwargs) -> 'BasePopulation':
"""Load a population from a checkpoint.
:param target: Path to checkpoint directory or file.
:param eval_function: Function that reduces a chromosome to a fitness.
:param kwargs: Any argument the init method accepts.
:return: Population
"""
result = cls(chromosomes=[], eval_function=eval_function, **kwargs)
result.individuals = result.serializer.load(target=target)
return result
def checkpoint(self, target: Optional[str] = None, method: str = 'pickle') -> 'BasePopulation':
"""Checkpoint the population.
:param target: Directory to write checkpoint to. If None, the Serializer default target is taken,
which can be provided upon initialisation. Defaults to None.
:param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.
Defaults to 'pickle'.
:return: Population
"""
self.serializer.checkpoint(individuals=self.individuals, target=target, method=method)
return self
@property
def _individual_weights(self):
try:
min_fitness = min(individual.fitness for individual in self)
max_fitness = max(individual.fitness for individual in self)
except TypeError:
raise RuntimeError('Individual weights can not be computed if the individuals are not evaluated.')
if min_fitness == max_fitness:
return [1] * len(self)
elif self.maximize:
return [(individual.fitness - min_fitness) / (max_fitness - min_fitness) for individual in self]
else:
return [1 - (individual.fitness - min_fitness) / (max_fitness - min_fitness) for individual in self]
def evolve(self, evolution: 'Evolution', n: int = 1) -> 'BasePopulation': # noqa: F821
"""Evolve the population according to an Evolution.
:param evolution: Evolution to follow
:param n: Times to apply the evolution. Defaults to 1.
:return: Population
"""
result = copy(self)
try:
for _ in range(n):
Condition.check(result)
for step in evolution:
result = step.apply(result)
except StopEvolution:
pass
return result
@abstractmethod
def evaluate(self, lazy: bool = False) -> 'BasePopulation':
pass
def breed(self,
parent_picker: Callable[..., Sequence[Individual]],
combiner: Callable,
population_size: Optional[int] = None,
**kwargs) -> 'BasePopulation':
"""Create new individuals by combining existing individuals.
This increments the generation of the Population.
:param parent_picker: Function that selects parents from a collection of individuals.
:param combiner: Function that combines chromosomes into a new
chromosome. Must be able to handle the number of chromosomes
that the combiner returns.
:param population_size: Intended population size after breeding.
If None, take the previous intended population size.
Defaults to None.
:param kwargs: Kwargs to pass to the parent_picker and combiner.
Arguments are only passed to the functions if they accept them.
:return: self
"""
if population_size:
self.intended_size = population_size
offspring = offspring_generator(parents=self.individuals,
parent_picker=select_arguments(parent_picker),
combiner=select_arguments(combiner),
**kwargs)
self.individuals += list(islice(offspring, self.intended_size - len(self.individuals)))
self.generation += 1
return self
def mutate(self,
mutate_function: Callable[..., Any],
probability: float = 1.0,
elitist: bool = False, **kwargs) -> 'BasePopulation':
"""Mutate the chromosome of each individual.
:param mutate_function: Function that accepts a chromosome and returns
a mutated chromosome.
:param probability: Probability that the individual mutates.
The function is only applied in the given fraction of cases.
Defaults to 1.0.
:param elitist: If True, do not mutate the current best individual(s).
Note that this only applies to evaluated individuals. Any unevaluated
individual will be treated as normal.
Defaults to False.
:param kwargs: Arguments to pass to the mutation function.
:return: self
"""
elite_fitness: Optional[float] = self.current_best.fitness if elitist else None
for individual in self.individuals:
if elite_fitness is None or individual.fitness != elite_fitness:
individual.mutate(mutate_function, probability=probability, **kwargs)
return self
def map(self, func: Callable[..., Individual], **kwargs) -> 'BasePopulation':
"""Apply the provided function to each individual in the population.
:param func: A function to apply to each individual in the population,
which when called returns a modified individual.
:param kwargs: Arguments to pass to the function.
:return: self
"""
self.individuals = [func(individual, **kwargs) for individual in self.individuals]
return self
def filter(self, func: Callable[..., bool], **kwargs) -> 'BasePopulation':
"""Add a filter step to the Evolution.
Filters the individuals in the population using the provided function.
:param func: Function to filter the individuals in the population by,
which returns a boolean when called on an individual.
:param kwargs: Arguments to pass to the function.
:return: self
"""
self.individuals = [individual for individual in self.individuals if func(individual, **kwargs)]
return self
def survive(self, fraction: Optional[float] = None,
n: Optional[int] = None, luck: bool = False) -> 'BasePopulation':
"""Let part of the population survive.
Remove part of the population. If both fraction and n are specified,
the minimum resulting population size is taken.
:param fraction: Fraction of the original population that survives.
Defaults to None.
:param n: Number of individuals of the population that survive.
Defaults to None.
:param luck: If True, individuals randomly survive (with replacement!)
with chances proportional to their fitness. Defaults to False.
:return: self
"""
if fraction is None:
if n is None:
raise ValueError('everyone survives! must provide either "fraction" and/or "n".')
resulting_size = n
elif n is None:
resulting_size = round(fraction * len(self.individuals))
else:
resulting_size = min(round(fraction * len(self.individuals)), n)
self.evaluate(lazy=True)
if resulting_size == 0:
raise RuntimeError(f'No individual out of {len(self.individuals)} survived!')
if resulting_size > len(self.individuals):
raise ValueError(f'everyone survives in population {self.id}: '
f'{resulting_size} out of {len(self.individuals)} must survive.')
if luck:
self.individuals = choices(self.individuals, k=resulting_size, weights=self._individual_weights)
else:
sorted_individuals = sorted(self.individuals, key=lambda x: x.fitness, reverse=self.maximize)
self.individuals = sorted_individuals[:resulting_size]
return self
def callback(self, callback_function: Callable[..., None],
**kwargs) -> 'BasePopulation':
"""
Performs a callback function on the population. Can be used for
custom logging/checkpointing.
:param callback_function: Function that accepts the population
as a first argument.
:return:
"""
self.evaluate(lazy=True)
callback_function(self, **kwargs)
return self
def group(self, grouping_function: Callable[..., List[List[int]]] = group_random,
**kwargs) -> List['BasePopulation']:
"""
Group a population into islands.
Divides the population into multiple island populations, each of which
contains a subset of the original population. An individual from the
original population may end up in multiple (>= 0) island populations.
:param grouping_function: Function that allocates individuals to the
island populations. It will be passed a list of individuals plus
the kwargs passed to this method, and must return a list of lists
of integers, each sub-list representing an island and the integers
representing the index of an individual in the list. Each island
must contain at least one individual, and individual may be copied
to multiple islands.
:param kwargs: Additional keyworded arguments are passed to the
grouping function.
:return: List[Population]
"""
group_indexes = grouping_function(self.individuals, **kwargs)
if len(group_indexes) == 0:
raise ValueError('Group yielded zero islands.')
result = [self._subset(index=index, subset_id=str(i)) for i, index in enumerate(group_indexes)]
return result
@classmethod
def combine(cls, *populations: 'BasePopulation',
intended_size: Optional[int] = None,
pool: Optional[Pool] = None) -> 'BasePopulation':
"""
Combine multiple island populations into a single population.
The resulting population is reduced to its intended size.
:param populations: Populations to combine.
:param intended_size: Intended size of the resulting population.
Defaults to the sum of the intended sizes of the islands.
:param pool: Optionally provide a multiprocessing pool to be
used by the population.
:return: Population
"""
if len(populations) == 0:
raise ValueError('Cannot combine zero islands into one.')
result = copy(populations[0])
for pop in populations[1:]:
result.individuals += pop.individuals
result.intended_size = intended_size or sum([pop.intended_size for pop in populations])
result.pool = pool
result.id = result.id.split('-')[0]
return result.survive(n=result.intended_size)
def _subset(self, index: List[int], subset_id: str) -> 'BasePopulation':
"""Create a new population that is a subset of this population."""
if len(index) == 0:
raise ValueError('Grouping yielded an empty island.')
result = copy(self)
result.individuals = [result.individuals[i] for i in index]
result.intended_size = len(result.individuals)
result.pool = None # Subsets shouldn't parallelize anything
result.id += '-' + subset_id
return result
def _update_documented_best(self):
"""Update the documented best"""
current_best = self.current_best
if (self.documented_best is None or
(self.maximize and current_best.fitness > self.documented_best.fitness) or
(not self.maximize and current_best.fitness < self.documented_best.fitness)):
self.documented_best = copy(current_best)
class Population(BasePopulation):
"""Population of Individuals
:param chromosomes: Iterable of initial chromosomes of the Population.
:param eval_function: Function that reduces a chromosome to a fitness.
:param maximize: If True, fitness will be maximized, otherwise minimized.
Defaults to True.
:param generation: Generation of the Population. This is incremented after
each breed call. Defaults to 0.
:param intended_size: Intended size of the Population. The population will
be replenished to this size by .breed(). Defaults to the number of
chromosomes provided.
:param checkpoint_target: Target for the serializer of the Population. If
a serializer is provided, this target is ignored. Defaults to None.
:param serializer: Serializer for the Population. If None, a new
SimpleSerializer is created. Defaults to None.
:param concurrent_workers: If > 1, evaluate individuals in {concurrent_workers}
separate processes. If None, concurrent_workers is set to n_cpus. Defaults to 1.
"""
def __init__(self,
chromosomes: Iterable,
eval_function: Callable[..., float],
maximize: bool = True,
generation: int = 0,
intended_size: Optional[int] = None,
checkpoint_target: Optional[str] = None,
serializer=None,
concurrent_workers: Optional[int] = 1):
super().__init__(chromosomes=chromosomes,
eval_function=eval_function,
checkpoint_target=checkpoint_target,
concurrent_workers=concurrent_workers,
maximize=maximize,
generation=generation,
intended_size=intended_size,
serializer=serializer)
def __copy__(self):
result = self.__class__(chromosomes=[],
eval_function=self.eval_function,
maximize=self.maximize,
serializer=self.serializer,
intended_size=self.intended_size,
generation=self.generation,
concurrent_workers=1) # Prevent new pool from being made
result.individuals = [copy(individual) for individual in self.individuals]
result.concurrent_workers = self.concurrent_workers
result.pool = self.pool
result.documented_best = self.documented_best
result.id = self.id
return result
def evaluate(self, lazy: bool = False) -> 'Population':
"""Evaluate the individuals in the population.
This evaluates the fitness of all individuals. If lazy is True, the
fitness is only evaluated when a fitness value is not yet known. In
most situations adding an explicit evaluation step is not needed, as
lazy evaluation is implicitly included in the operations that need it
(most notably in the survive operation).
:param lazy: If True, do no re-evaluate the fitness if the fitness is known.
:return: self
"""
if self.pool:
f = self.eval_function # We cannot refer to self in the map
scores = self.pool.map(lambda i: i.fitness if (i.fitness and lazy) else f(i.chromosome), self.individuals)
for individual, fitness in zip(self.individuals, scores):
individual.fitness = fitness
else:
for individual in self.individuals:
individual.evaluate(eval_function=self.eval_function, lazy=lazy)
self._update_documented_best()
return self
class Contest:
"""A single contest among a group of competitors.
This is encapsulated in an object so that scores for many sets of
competitors can be evaluated concurrently without resorting to a
dict or some similar madness to correlate score vectors with an
ever-widening matrix of contests and competitors.
:param competitors: Iterable of Individuals in this Contest.
"""
def __init__(self, competitors: Iterable[Individual]):
self.competitors = list(competitors)
def assign_scores(self, scores: Sequence[float]) -> None:
for competitor, score in zip(self.competitors, scores):
competitor.fitness += score
@property
def competitor_chromosomes(self):
return [competitor.chromosome for competitor in self.competitors]
@classmethod
def generate(cls, individuals: Sequence[Individual],
individuals_per_contest: int, contests_per_round: int) -> List['Contest']:
"""Generate contests for a round of evaluations.
:param individuals: A sequence of competing Individuals.
:param individuals_per_contest: Number of Individuals participating in each Contest.
:param contests_per_round: Minimum number of contests each individual
takes part in for each evaluation round. The actual number of contests
per round is a multiple of individuals_per_contest.
:return: List of Contests
"""
contests = []
n_rounds = ceil(contests_per_round / individuals_per_contest)
for _ in range(n_rounds):
offsets = [0] + [randint(0, len(individuals) - 1) for _ in range(individuals_per_contest - 1)]
generators = [islice(cycle(individuals), offset, None) for offset in offsets]
for competitors in islice(zip(*generators), len(individuals)):
contests.append(Contest(competitors))
return contests
class ContestPopulation(BasePopulation):
"""Population which is evaluated through contests.
This variant of the Population is used when individuals cannot be
evaluated on a one-by-one basis, but instead can only be compared to
each other. This is typically the case for AI that performs some task
(i.e. plays a game), but can be useful in many other cases.
For each round of evaluation, each individual participates in a given
number of contests, in which a given number of individuals take part.
The resulting scores of these contests are summed to form the fitness.
Since the fitness of an individual is dependent on the other individuals
in the population, the fitness of all individuals is recalculated when
new individuals are present, and the fitness of all individuals is reset
when the population is modified (e.g. by calling survive, mutate etc).
:param chromosomes: Iterable of initial chromosomes of the Population.
:param eval_function: Function that reduces a chromosome to a fitness.
:param maximize: If True, fitness will be maximized, otherwise minimized.
Defaults to True.
:param individuals_per_contest: Number of individuals that take part in
each contest. The size of the population must be divisible by this
number. Defaults to 2.
:param contests_per_round: Minimum number of contests each individual
takes part in for each evaluation round. The actual number of contests
per round is a multiple of individuals_per_contest. Defaults to 10.
:param generation: Generation of the Population. This is incremented after
echo survive call. Defaults to 0.
:param intended_size: Intended size of the Population. The population will
be replenished to this size by .breed(). Defaults to the number of
chromosomes provided.
:param checkpoint_target: Target for the serializer of the Population. If
a serializer is provided, this target is ignored. Defaults to None.
:param serializer: Serializer for the Population. If None, a new
SimpleSerializer is created. Defaults to None.
:param concurrent_workers: If > 1, evaluate individuals in {concurrent_workers}
separate processes. If None, concurrent_workers is set to n_cpus. Defaults to 1.
"""
eval_function: Callable[..., Sequence[float]] # This population expects a different eval signature
def __init__(self,
chromosomes: Iterable,
eval_function: Callable[..., Sequence[float]],
maximize: bool = True,
individuals_per_contest=2,
contests_per_round=10,
generation: int = 0,
intended_size: Optional[int] = None,
checkpoint_target: Optional[int] = None,
serializer=None,
concurrent_workers: Optional[int] = 1):
super().__init__(chromosomes=chromosomes,
eval_function=eval_function,
maximize=maximize,
generation=generation,
intended_size=intended_size,
checkpoint_target=checkpoint_target,
serializer=serializer,
concurrent_workers=concurrent_workers)
self.contests_per_round = contests_per_round
self.individuals_per_contest = individuals_per_contest
def __copy__(self):
result = self.__class__(chromosomes=[],
eval_function=self.eval_function,
maximize=self.maximize,
contests_per_round=self.contests_per_round,
individuals_per_contest=self.individuals_per_contest,
serializer=self.serializer,
intended_size=self.intended_size,
generation=self.generation,
concurrent_workers=1)
result.individuals = [copy(individual) for individual in self.individuals]
result.pool = self.pool
result.concurrent_workers = self.concurrent_workers
result.documented_best = None
result.id = self.id
return result
def evaluate(self,
lazy: bool = False,
contests_per_round: Optional[int] = None,
individuals_per_contest: Optional[int] = None) -> 'ContestPopulation':
"""Evaluate the individuals in the population.
This evaluates the fitness of all individuals. For each round of
evaluation, each individual participates in a given number of
contests, in which a given number of individuals take part.
The resulting scores of these contests are summed to form the fitness.
This means that the score of the individual is influenced by other
chromosomes in the population.
Note that in the `ContestPopulation` two settings are passed at
initialisation which affect how we are evaluating individuals:
contests_per_round and individuals_per_contest. You may overwrite them
here if you wish.
If lazy is True, the fitness is only evaluated when a fitness value
is not yet known for all individuals.
In most situations adding an explicit evaluation step is not needed, as
lazy evaluation is implicitly included in the operations that need it
(most notably in the survive operation).
:param lazy: If True, do no re-evaluate the fitness if the fitness is known.
:param contests_per_round: If set, overwrites the population setting for the
number of contests there will be every round.
:param individuals_per_contest: If set, overwrites the population setting for
number of individuals to have in a contest during the evaluation.
:return: self
"""
if contests_per_round is None:
contests_per_round = self.contests_per_round
if individuals_per_contest is None:
individuals_per_contest = self.individuals_per_contest
if lazy and all(individual.fitness is not None for individual in self):
return self
for individual in self.individuals:
individual.fitness = 0
contests = Contest.generate(individuals=self.individuals, individuals_per_contest=individuals_per_contest,
contests_per_round=contests_per_round)
if self.pool is None:
for contest in contests:
contest.assign_scores(self.eval_function(*contest.competitor_chromosomes))
else:
f = self.eval_function # We cannot refer to self in the map
results = self.pool.map(lambda c: f(*c.competitor_chromosomes), contests)
for result, contest in zip(results, contests):
contest.assign_scores(result)
return self
def map(self, func: Callable[..., Individual], **kwargs) -> 'ContestPopulation':
"""Apply the provided function to each individual in the population.
Resets the fitness of all individuals.
:param func: A function to apply to each individual in the population,
which when called returns a modified individual.
:param kwargs: Arguments to pass to the function.
:return: self
"""
BasePopulation.map(self, func=func, **kwargs)
self.reset_fitness()
return self
def filter(self, func: Callable[..., bool], **kwargs) -> 'ContestPopulation':
"""Add a filter step to the Evolution.
Filters the individuals in the population using the provided function.
Resets the fitness of all individuals.
:param func: Function to filter the individuals in the population by,
which returns a boolean when called on an individual.
:param kwargs: Arguments to pass to the function.
:return: self
"""
BasePopulation.filter(self, func=func, **kwargs)
self.reset_fitness()
return self
def survive(self,
fraction: Optional[float] = None,
n: Optional[int] = None,
luck: bool = False) -> 'ContestPopulation':
"""Let part of the population survive.
Remove part of the population. If both fraction and n are specified,
the minimum resulting population size is taken. Resets the fitness
of all individuals.
:param fraction: Fraction of the original population that survives.
Defaults to None.
:param n: Number of individuals of the population that survive.
Defaults to None.
:param luck: If True, individuals randomly survive (with replacement!)
with chances proportional to their fitness. Defaults to False.
:return: self
"""
BasePopulation.survive(self, fraction=fraction, n=n, luck=luck)
self.reset_fitness()
return self
def reset_fitness(self):
"""Reset the fitness of all individuals."""
for individual in self:
individual.fitness = None
================================================
FILE: evol/problems/__init__.py
================================================
================================================
FILE: evol/problems/functions/__init__.py
================================================
"""
The `evol.problems.functions` part of the library contains
simple problem instances that do with known math functions.
The functions in here are typically inspired from wikipedia:
https://en.wikipedia.org/wiki/Test_functions_for_optimization
"""
from .variableinput import Rosenbrock, Sphere, Rastrigin
================================================
FILE: evol/problems/functions/variableinput.py
================================================
import math
from typing import Sequence
from evol.helpers.utils import sliding_window
from evol.problems.problem import Problem
class FunctionProblem(Problem):
def __init__(self, size=2):
self.size = size
def check_solution(self, solution: Sequence[float]) -> Sequence[float]:
if len(solution) > self.size:
raise ValueError(f"{self.__class__.__name__} has size {self.size}, \
got solution of size: {len(solution)}")
return solution
def value(self, solution):
return sum(solution)
def eval_function(self, solution: Sequence[float]) -> float:
self.check_solution(solution)
return self.value(solution)
class Sphere(FunctionProblem):
def value(self, solution: Sequence[float]) -> float:
"""
The optimal value can be found when a sequence of zeros is given.
:param solution: a sequence of x_i values
:return: the value of the Sphere function
"""
return sum([_**2 for _ in solution])
class Rosenbrock(FunctionProblem):
def value(self, solution: Sequence[float]) -> float:
"""
The optimal value can be found when a sequence of ones is given.
:param solution: a sequence of x_i values
:return: the value of the Rosenbrock function
"""
result = 0
for x_i, x_j in sliding_window(solution):
result += 100*(x_j - x_i**2)**2 + (1 - x_i)**2
return result
class Rastrigin(FunctionProblem):
def value(self, solution: Sequence[float]) -> float:
"""
The optimal value can be found when a sequence of zeros is given.
:param solution: a sequence of x_i values
:return: the value of the Rosenbrock function
"""
return (10 * self.size) + sum([_**2 - 10 * math.cos(2*math.pi*_) for _ in solution])
================================================
FILE: evol/problems/problem.py
================================================
from abc import ABCMeta, abstractmethod
class Problem(metaclass=ABCMeta):
@abstractmethod
def eval_function(self, solution):
raise NotImplementedError
================================================
FILE: evol/problems/routing/__init__.py
================================================
"""
The `evol.problems.routing` part of the library contains
simple problem instances that do with routing problems. These
are meant to be used for education and training purposes and
these problems are typically good starting points if you want
to play with the library.
"""
from .tsp import TSPProblem
from .magicsanta import MagicSanta
================================================
FILE: evol/problems/routing/coordinates.py
================================================
united_states_capitols = [
(32.361538, -86.279118, "Montgomery", "Alabama"),
(58.301935, -134.419740, "Juneau", "Alaska"),
(33.448457, -112.073844, "Phoenix", "Arizona"),
(34.736009, -92.331122, "Little Rock", "Arkansas"),
(38.555605, -121.468926, "Sacramento", "California"),
(39.7391667, -104.984167, "Denver", "Colorado"),
(41.767, -72.677, "Hartford", "Connectic"),
(39.161921, -75.526755, "Dover", "Delaware"),
(30.4518, -84.27277, "Tallahassee", "Florida"),
(33.76, -84.39, "Atlanta", "Georgia"),
(21.30895, -157.826182, "Honolulu", "Hawaii"),
(43.613739, -116.237651, "Boise", "Idaho"),
(39.783250, -89.650373, "Springfield", "Illinois"),
(39.790942, -86.147685, "Indianapolis", "Indiana"),
(41.590939, -93.620866, "Des Moines", "Iowa"),
(39.04, -95.69, "Topeka", "Kansas"),
(38.197274, -84.86311, "Frankfort", "Kentucky"),
(30.45809, -91.140229, "Baton Rouge", "Louisiana"),
(44.323535, -69.765261, "Augusta", "Maine"),
(38.972945, -76.501157, "Annapolis", "Maryland"),
(42.2352, -71.0275, "Boston", "Massachuset"),
(42.7335, -84.5467, "Lansing", "Michigan"),
(44.95, -93.094, "Saint Paul", "Minnesot"),
(32.320, -90.207, "Jackson", "Mississip"),
(38.572954, -92.189283, "Jefferson City", "Missouri"),
(46.595805, -112.027031, "Helana", "Montana"),
(40.809868, -96.675345, "Lincoln", "Nebraska"),
(39.160949, -119.753877, "Carson City", "Nevada"),
(43.220093, -71.549127, "Concord", "Hampshire"),
(40.221741, -74.756138, "Trenton", "Jersey"),
(35.667231, -105.964575, "Santa Fe", "Mexico"),
(42.659829, -73.781339, "Albany", "York"),
(35.771, -78.638, "Raleigh", "Car"),
(48.813343, -100.779004, "Bismarck", "Dakota"),
(39.962245, -83.000647, "Columbus", "Ohio"),
(35.482309, -97.534994, "Oklahoma City", "Oklahoma"),
(44.931109, -123.029159, "Salem", "Oregon"),
(40.269789, -76.875613, "Harrisburg", "Pennsylvania"),
(41.82355, -71.422132, "Providence", "Island"),
(34.000, -81.035, "Columbia", "Car"),
(44.367966, -100.336378, "Pierre", "Dakota"),
(36.165, -86.784, "Nashville", "Tennessee"),
(30.266667, -97.75, "Austin", "Texas"),
(40.7547, -111.892622, "Salt Lake City", "Utah"),
(44.26639, -72.57194, "Montpelier", "Vermont"),
(37.54, -77.46, "Richmond", "Virgini"),
(47.042418, -122.893077, "Olympia", "Washington"),
(38.349497, -81.633294, "Charleston", "Virginia"),
(43.074722, -89.384444, "Madison", "Wisconsin"),
(41.145548, -104.802042, "Cheyenne", "Wyoming")]
================================================
FILE: evol/problems/routing/magicsanta.py
================================================
import math
from collections import Counter
from itertools import chain
from typing import List, Union
from evol.helpers.utils import sliding_window
from evol.problems.problem import Problem
class MagicSanta(Problem):
def __init__(self, city_coordinates, home_coordinate, gift_weight=None, sleigh_weight=1):
"""
This problem is based on this kaggle competition:
https://www.kaggle.com/c/santas-stolen-sleigh#evaluation.
:param city_coordinates: List of tuples containing city coordinates.
:param home_coordinate: Tuple containing coordinate of home base.
:param gift_weight: Vector of weights per gift associated with cities.
:param sleigh_weight: Weight of the sleight.
"""
self.coordinates = city_coordinates
self.home_coordinate = home_coordinate
self.gift_weight = gift_weight
if gift_weight is None:
self.gift_weight = [1 for _ in city_coordinates]
self.sleigh_weight = sleigh_weight
@staticmethod
def distance(coord_a, coord_b):
return math.sqrt(sum([(z[0] - z[1]) ** 2 for z in zip(coord_a, coord_b)]))
def check_solution(self, solution: List[List[int]]):
"""
Check if the solution for the problem is valid.
:param solution: List of lists containing integers representing visited cities.
:return: None, unless errors are raised.
"""
set_visited = set(chain.from_iterable(solution))
set_problem = set(range(len(self.coordinates)))
if set_visited != set_problem:
missing = set_problem.difference(set_visited)
extra = set_visited.difference(set_problem)
raise ValueError(f"Not all cities are visited! Missing: {missing} Extra: {extra}")
city_counter = Counter(chain.from_iterable(solution))
if max(city_counter.values()) > 1:
double_cities = {key for key, value in city_counter.items() if value > 1}
raise ValueError(f"Multiple occurrences found for cities: {double_cities}")
def eval_function(self, solution: List[List[int]]) -> Union[float, int]:
"""
Calculates the cost of the current solution for the TSP problem.
:param solution: List of integers which refer to cities.
:return:
"""
self.check_solution(solution=solution)
cost = 0
for route in solution:
total_route_weight = sum([self.gift_weight[t] for t in route]) + self.sleigh_weight
distance = self.distance(self.home_coordinate, self.coordinates[route[0]])
cost += distance * total_route_weight
for t1, t2 in sliding_window(route):
total_route_weight -= self.gift_weight[t1]
city1 = self.coordinates[t1]
city2 = self.coordinates[t2]
cost += self.distance(city1, city2) * total_route_weight
last_leg_distance = self.distance(self.coordinates[route[-1]], self.home_coordinate)
cost += self.sleigh_weight * last_leg_distance
return cost
================================================
FILE: evol/problems/routing/tsp.py
================================================
import math
from typing import List, Union
from evol.problems.problem import Problem
from evol.helpers.utils import rotating_window
class TSPProblem(Problem):
def __init__(self, distance_matrix):
self.distance_matrix = distance_matrix
@classmethod
def from_coordinates(cls, coordinates: List[Union[tuple, list]]) -> 'TSPProblem':
"""
Creates a distance matrix from a list of city coordinates.
:param coordinates: An iterable that contains tuples or lists representing a x,y coordinate.
:return: A list of lists containing the distances between cities.
"""
res = [[0 for i in coordinates] for j in coordinates]
for i, coord_i in enumerate(coordinates):
for j, coord_j in enumerate(coordinates):
dist = math.sqrt(sum([(z[0] - z[1])**2 for z in zip(coord_i[:2], coord_j[:2])]))
res[i][j] = dist
res[j][i] = dist
return TSPProblem(distance_matrix=res)
def check_solution(self, solution: List[int]):
"""
Check if the solution for the TSP problem is valid.
:param solution: List of integers which refer to cities.
:return: None, unless errors are raised.
"""
set_solution = set(solution)
set_problem = set(range(len(self.distance_matrix)))
if len(solution) > len(self.distance_matrix):
raise ValueError("Solution is longer than number of towns!")
if set_solution != set_problem:
raise ValueError(f"Not all towns are visited! Am missing {set_problem.difference(set_solution)}")
def eval_function(self, solution: List[int]) -> Union[float, int]:
"""
Calculates the cost of the current solution for the TSP problem.
:param solution: List of integers which refer to cities.
:return:
"""
self.check_solution(solution=solution)
cost = 0
for t1, t2 in rotating_window(solution):
cost += self.distance_matrix[t1][t2]
return cost
================================================
FILE: evol/serialization.py
================================================
"""
Serializers help store (checkpoint) the state of your population during or
after running your evolutionary algorithm. By default, each Population is
initialized with a SimpleSerializer, which you can use to store the individuals
in your population in pickle or json format using the .checkpoint() method of
the population. Currently no other serializers are available.
"""
import json
import pickle
from datetime import datetime
from typing import List, Optional
from os import listdir
from os.path import isdir, exists, join
from evol import Individual
class SimpleSerializer:
"""The SimpleSerializer handles serialization to and from pickle and json.
:param target: Default location (directory) to store checkpoint.
This may be overridden in the `checkpoint` method. Defaults to None.
"""
def __init__(self, target: Optional[str] = None):
self.target = target
def checkpoint(self, individuals: List[Individual], target: Optional[str] = None, method: str = 'pickle') -> None:
"""Checkpoint a list of individuals.
:param individuals: List of individuals to checkpoint.
:param target: Directory to write checkpoint to. If None, the Serializer default target is taken,
which can be provided upon initialisation. Defaults to None.
:param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.
Defaults to 'pickle'.
"""
filename = self._new_checkpoint_file(target=self.target if target is None else target, method=method)
if method == 'pickle':
with open(filename, 'wb') as pickle_file:
pickle.dump(individuals, pickle_file)
elif method == 'json':
with open(filename, 'w') as json_file:
json.dump([individual.__dict__ for individual in individuals], json_file)
else:
raise ValueError('Invalid checkpointing method "{}". Choose "pickle" or "json".'.format(method))
def load(self, target: Optional[str] = None) -> List[Individual]:
"""Load a checkpoint.
If path is a file, load that file. If it is a directory, load the most recent checkpoint.
The checkpoint file must end with a '.json' or '.pkl' extension.
:param target: Path to checkpoint directory or file.
:return: List of individuals from checkpoint.
"""
filename = self._find_checkpoint(self.target if target is None else target)
if filename.endswith('.json'):
with open(filename, 'r') as json_file:
return [Individual.from_dict(d) for d in json.load(json_file)]
elif filename.endswith('.pkl'):
with open(filename, 'rb') as pickle_file:
return pickle.load(pickle_file)
@staticmethod
def _new_checkpoint_file(target: str, method: str):
"""Generate a filename for a new checkpoint."""
if target is None:
raise ValueError('Serializer requires a target to checkpoint to.')
if not isdir(target):
raise FileNotFoundError('Cannot checkpoint to "{}": is not a directory.'.format(target))
result = join(target, datetime.now().strftime("%Y%m%d-%H%M%S.%f") + ('.pkl' if method == 'pickle' else '.json'))
if exists(result):
raise FileExistsError('Cannot checkpoint to "{}": file exists.'.format(result))
return result
@classmethod
def _find_checkpoint(cls, target: str):
"""Find the most recent checkpoint file."""
if not exists(target):
raise FileNotFoundError('Cannot load from "{}": file or directory does not exists.'.format(target))
elif isdir(target):
try:
return join(target, max(filter(cls._has_valid_extension, listdir(path=target))))
except ValueError:
raise FileNotFoundError('Cannot load from "{}": directory contains no checkpoints.'.format(target))
else:
if not cls._has_valid_extension(target):
raise ValueError('Invalid extension "{}": Was expecting ".pkl" or ".json".'.format(target))
return target
@staticmethod
def _has_valid_extension(filename: str):
"""Check if a filename has a valid extension."""
return filename.endswith('.pkl') or filename.endswith('.json')
================================================
FILE: evol/step.py
================================================
from abc import ABCMeta, abstractmethod
from typing import Callable, Optional, TYPE_CHECKING
from evol.population import BasePopulation
if TYPE_CHECKING:
from evol.evolution import Evolution
class EvolutionStep(metaclass=ABCMeta):
def __init__(self, name: Optional[str], **kwargs):
self.name = name
self.kwargs = kwargs
def __repr__(self):
return f"{self.__class__.__name__}({self.name or ''})"
@abstractmethod
def apply(self, population: BasePopulation) -> BasePopulation:
pass
class EvaluationStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.evaluate(**self.kwargs)
class CheckpointStep(EvolutionStep):
def __init__(self, name, every=1, **kwargs):
EvolutionStep.__init__(self, name, **kwargs)
self.count = 0
self.every = every
def apply(self, population: BasePopulation) -> BasePopulation:
self.count += 1
if self.count >= self.every:
self.count = 0
return population.checkpoint(**self.kwargs)
return population
class MapStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.map(**self.kwargs)
class FilterStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.filter(**self.kwargs)
class SurviveStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.survive(**self.kwargs)
class BreedStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.breed(**self.kwargs)
class MutateStep(EvolutionStep):
def apply(self, population: BasePopulation) -> BasePopulation:
return population.mutate(**self.kwargs)
class RepeatStep(EvolutionStep):
def __init__(self, name: str, evolution: 'Evolution', n: int,
grouping_function: Optional[Callable] = None, **kwargs):
super().__init__(name=name, **kwargs)
self.evolution = evolution
self.n = n
self.grouping_function = grouping_function
def apply(self, population: BasePopulation) -> BasePopulation:
if self.grouping_function is None:
if len(self.kwargs) > 0:
raise ValueError(f'Unexpected argument(s) for non-grouped repeat step: {self.kwargs}')
return population.evolve(evolution=self.evolution, n=self.n)
else:
return self._apply_grouped(population=population)
def _apply_grouped(self, population: BasePopulation) -> BasePopulation:
groups = population.group(grouping_function=self.grouping_function, **self.kwargs)
if population.pool:
results = population.pool.map(lambda group: group.evolve(evolution=self.evolution, n=self.n), groups)
else:
results = [group.evolve(evolution=self.evolution, n=self.n) for group in groups]
return population.combine(*results, intended_size=population.intended_size, pool=population.pool)
def __repr__(self):
result = f"{self.__class__.__name__}({self.name or ''}) with evolution ({self.n}x):\n "
result += repr(self.evolution).replace('\n', '\n ')
return result
class CallbackStep(EvolutionStep):
def __init__(self, name, every: int = 1, **kwargs):
EvolutionStep.__init__(self, name, **kwargs)
self.count = 0
self.every = every
def apply(self, population: BasePopulation) -> BasePopulation:
self.count += 1
if self.count >= self.every:
self.count = 0
return population.callback(**self.kwargs)
return population
================================================
FILE: evol/utils.py
================================================
from inspect import signature
from typing import List, Callable, Union, Sequence, Any, Generator
from evol import Individual
def offspring_generator(parents: List[Individual],
parent_picker: Callable[..., Union[Individual, Sequence]],
combiner: Callable[..., Any],
**kwargs) -> Generator[Individual, None, None]:
"""Generator for offspring.
This helps create the right number of offspring,
especially in the case of of multiple offspring.
:param parents: List of parents.
:param parent_picker: Function that selects parents. Must accept a sequence of
individuals and must return a single individual or a sequence of individuals.
Must accept all kwargs passed (i.e. must be decorated by select_arguments).
:param combiner: Function that combines chromosomes. Must accept a tuple of
chromosomes and either return a single chromosome or yield multiple chromosomes.
Must accept all kwargs passed (i.e. must be decorated by select_arguments).
:param kwargs: Arguments
:returns: Children
"""
while True:
# Obtain parent chromosomes
selected_parents = parent_picker(parents, **kwargs)
if isinstance(selected_parents, Individual):
chromosomes = (selected_parents.chromosome,)
else:
chromosomes = tuple(individual.chromosome for individual in selected_parents)
# Create children
combined = combiner(*chromosomes, **kwargs)
if isinstance(combined, Generator):
for child in combined:
yield Individual(chromosome=child)
else:
yield Individual(chromosome=combined)
def select_arguments(func: Callable) -> Callable:
"""Decorate a function such that it accepts any keyworded arguments.
The resulting function accepts any arguments, but only arguments that
the original function accepts are passed. This allows keyworded
arguments to be passed to multiple (decorated) functions, even if they
do not (all) accept these arguments.
:param func: Function to decorate.
:return: Callable
"""
def result(*args, **kwargs):
try:
return func(*args, **kwargs)
except TypeError:
return func(*args, **{k: v for k, v in kwargs.items() if k in signature(func).parameters})
return result
================================================
FILE: examples/number_of_parents.py
================================================
"""
There are a few worthwhile things to notice in this example:
1. you can pass hyperparams into functions from the `.breed` and `.mutate` step
2. the algorithm does not care how many parents it will use in the `breed` step
"""
import random
import math
import argparse
from evol import Population, Evolution
def run_evolutionary(opt_value=1, population_size=100, n_parents=2, workers=1,
num_iter=200, survival=0.5, noise=0.1, seed=42, verbose=True):
random.seed(seed)
def init_func():
return (random.random() - 0.5) * 20 + 10
def eval_func(x, opt_value=opt_value):
return -((x - opt_value) ** 2) + math.cos(x - opt_value)
def random_parent_picker(pop, n_parents):
return [random.choice(pop) for i in range(n_parents)]
def mean_parents(*parents):
return sum(parents) / len(parents)
def add_noise(chromosome, sigma):
return chromosome + (random.random() - 0.5) * sigma
pop = Population(chromosomes=[init_func() for _ in range(population_size)],
eval_function=eval_func, maximize=True, concurrent_workers=workers).evaluate()
evo = (Evolution()
.survive(fraction=survival)
.breed(parent_picker=random_parent_picker, combiner=mean_parents, n_parents=n_parents)
.mutate(mutate_function=add_noise, sigma=noise)
.evaluate())
for i in range(num_iter):
pop = pop.evolve(evo)
if verbose:
print(f"iteration:{i} best: {pop.current_best.fitness} worst: {pop.current_worst.fitness}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Run an example evol algorithm against a simple continuous function.')
parser.add_argument('--opt-value', type=int, default=0,
help='the true optimal value of the problem')
parser.add_argument('--population-size', type=int, default=20,
help='the number of candidates to start the algorithm with')
parser.add_argument('--n-parents', type=int, default=2,
help='the number of parents the algorithm with use to generate new indivuals')
parser.add_argument('--num-iter', type=int, default=20,
help='the number of evolutionary cycles to run')
parser.add_argument('--survival', type=float, default=0.7,
help='the fraction of individuals who will survive a generation')
parser.add_argument('--noise', type=float, default=0.5,
help='the amount of noise the mutate step will add to each individual')
parser.add_argument('--seed', type=int, default=42,
help='the random seed for all this')
parser.add_argument('--workers', type=int, default=1,
help='the number of workers to run the command in')
args = parser.parse_args()
run_evolutionary(opt_value=args.opt_value, population_size=args.population_size,
n_parents=args.n_parents, num_iter=args.num_iter,
noise=args.noise, seed=args.seed, workers=args.workers)
================================================
FILE: examples/population_demo.py
================================================
import random
from evol import Population
def create_candidate():
return random.random() - 0.5
def func_to_optimise(x):
return x*2
def pick_random_parents(pop):
return random.choice(pop)
random.seed(42)
pop1 = Population(chromosomes=[create_candidate() for _ in range(5)],
eval_function=func_to_optimise, maximize=True)
pop2 = Population.generate(init_function=create_candidate,
eval_function=func_to_optimise,
size=5,
maximize=False)
print("[i for i in pop1]:")
print([i for i in pop1])
print("[i.chromosome for i in pop1]:")
print([i.chromosome for i in pop1])
print("[i.fitness for i in pop1]:")
print([i.fitness for i in pop1])
print("[i.fitness for i in pop1.evaluate()]:")
def produce_clone(parent):
return parent
def add_noise(x):
return 0.1 * (random.random() - 0.5) + x
print("[i.fitness for i in pop1.survive(n=3)]:")
print([i.fitness for i in pop1.survive(n=3)])
print("[i.fitness for i in pop1.survive(n=3).mutate(add_noise)]:")
print([i.fitness for i in pop1.survive(n=3).mutate(add_noise)])
print("[i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()]:")
print([i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()])
================================================
FILE: examples/rock_paper_scissors.py
================================================
#!/usr/bin/env python
from argparse import ArgumentParser
from collections import Counter
from random import choice, random, seed
from typing import List
from evol import Evolution, ContestPopulation
from evol.helpers.pickers import pick_random
from evol.helpers.groups import group_duplicate
class RockPaperScissorsPlayer:
arbitrariness = 0.0
elements = ('rock', 'paper', 'scissors')
def __init__(self, preference=None):
self.preference = preference if preference else choice(self.elements)
def __repr__(self):
return '{}({})'.format(self.__class__.__name__, self.preference)
def play(self):
if random() >= self.arbitrariness:
return self.preference
else:
return choice(self.elements)
def mutate(self, volatility=0.1):
if random() < volatility:
return self.__class__(choice(self.elements))
else:
return self.__class__(self.preference)
def combine(self, other):
return self.__class__(choice([self.preference, other.preference]))
class RockPaperScissorsLizardSpockPlayer(RockPaperScissorsPlayer):
elements = ('rock', 'paper', 'scissors', 'lizard', 'spock')
COMBINATIONS = [
('scissors', 'paper'),
('paper', 'rock'),
('rock', 'scissors'),
('rock', 'lizard'),
('lizard', 'spock'),
('spock', 'scissors'),
('scissors', 'lizard'),
('lizard', 'paper'),
('paper', 'spock'),
('spock', 'rock'),
]
def evaluate(player_1: RockPaperScissorsPlayer, player_2: RockPaperScissorsPlayer) -> List[float]:
choice_1, choice_2 = player_1.play(), player_2.play()
player_choices = {choice_1, choice_2}
if len(player_choices) == 1:
return [0, 0]
for combination in COMBINATIONS:
if set(combination) == player_choices:
return [1, -1] if choice_1 == combination[0] else [-1, 1]
class History:
def __init__(self):
self.history = []
def log(self, population: ContestPopulation):
preferences = Counter()
for individual in population:
preferences.update([individual.chromosome.preference])
self.history.append(dict(**preferences, id=population.id, generation=population.generation))
def plot(self):
try:
import pandas as pd
import matplotlib.pylab as plt
df = pd.DataFrame(self.history).set_index(['id', 'generation']).fillna(0)
population_size = sum(df.iloc[0].values)
n_populations = df.reset_index()['id'].nunique()
fig, axes = plt.subplots(nrows=n_populations, figsize=(12, 2*n_populations),
sharex='all', sharey='all', squeeze=False)
for row, (_, pop) in zip(axes, df.groupby('id')):
ax = row[0]
pop.reset_index(level='id', drop=True).plot(ax=ax)
ax.set_ylim([0, population_size])
ax.set_xlabel('iteration')
ax.set_ylabel('# w/ preference')
if n_populations > 1:
for i in range(0, df.reset_index().generation.max(), 50):
ax.axvline(i)
plt.show()
except ImportError:
print("If you install matplotlib and pandas you will get a pretty plot.")
def run_rock_paper_scissors(population_size: int = 100,
n_iterations: int = 200,
random_seed: int = 42,
survive_fraction: float = 0.8,
arbitrariness: float = 0.2,
concurrent_workers: int = 1,
lizard_spock: bool = False,
grouped: bool = False,
silent: bool = False):
seed(random_seed)
RockPaperScissorsPlayer.arbitrariness = arbitrariness
player_class = RockPaperScissorsLizardSpockPlayer if lizard_spock else RockPaperScissorsPlayer
pop = ContestPopulation(chromosomes=[player_class() for _ in range(population_size)],
eval_function=evaluate, maximize=True,
concurrent_workers=concurrent_workers).evaluate()
history = History()
evo = Evolution().repeat(
evolution=(Evolution()
.survive(fraction=survive_fraction)
.breed(parent_picker=pick_random, combiner=lambda x, y: x.combine(y), n_parents=2)
.mutate(lambda x: x.mutate())
.evaluate()
.callback(history.log)),
n=n_iterations // 4,
grouping_function=group_duplicate if grouped else None
)
pop.evolve(evo, n=4)
if not silent:
history.plot()
return history
def parse_arguments():
parser = ArgumentParser(description='Run the rock-paper-scissors example.')
parser.add_argument('--population-size', dest='population_size', type=int, default=100,
help='the number of candidates to start the algorithm with')
parser.add_argument('--n-iterations', dest='n_iterations', type=int, default=200,
help='the number of iterations to run')
parser.add_argument('--random-seed', dest='random_seed', type=int, default=42,
help='the random seed to set')
parser.add_argument('--survive-fraction', dest='survive_fraction', type=float, default=0.8,
help='the fraction of the population to survive each iteration')
parser.add_argument('--arbitrariness', type=float, default=0.2,
help='arbitrariness of the players. if zero, player will always choose its preference')
parser.add_argument('--concurrent_workers', type=int, default=1,
help='Concurrent workers to use to evaluate the population.')
parser.add_argument('--lizard-spock', action='store_true', default=False,
help='Play rock-paper-scissors-lizard-spock.')
parser.add_argument('--grouped', action='store_true', default=False,
help='Run the evolution in four groups.')
return parser.parse_args()
if __name__ == "__main__":
args = parse_arguments()
run_rock_paper_scissors(**args.__dict__)
================================================
FILE: examples/simple_callback.py
================================================
"""
This example demonstrates how logging works in evolutions.
"""
import random
from evol import Population, Evolution
def random_start():
"""
This function generates a random (x,y) coordinate in the searchspace
"""
return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20
def func_to_optimise(xy):
"""
This is the function we want to optimise (maximize)
"""
x, y = xy
return -(1-x)**2 - 2*(2-x**2)**2
def pick_random_parents(pop):
"""
This is how we are going to select parents from the population
"""
mom = random.choice(pop)
dad = random.choice(pop)
return mom, dad
def make_child(mom, dad):
"""
This is how two parents are going to make a child.
Note that the output of a tuple, just like the output of `random_start`
"""
child_x = (mom[0] + dad[0])/2
child_y = (mom[1] + dad[1])/2
return child_x, child_y
def add_noise(chromosome, sigma):
"""
This is a function that will add some noise to the chromosome.
"""
new_x = chromosome[0] + (random.random()-0.5) * sigma
new_y = chromosome[1] + (random.random()-0.5) * sigma
return new_x, new_y
class MyLogger():
def __init__(self):
self.i = 0
def log(self, pop):
self.i += 1
best = max([i.fitness for i in pop.evaluate()])
print(f"the best score i={self.i} => {best}")
if __name__ == "__main__":
logger = MyLogger()
random.seed(42)
pop = Population(chromosomes=[random_start() for _ in range(200)],
eval_function=func_to_optimise,
maximize=True, concurrent_workers=2)
evo1 = (Evolution()
.survive(fraction=0.1)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=0.2)
.evaluate()
.callback(logger.log))
evo2 = (Evolution()
.survive(n=10)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=0.1)
.evaluate()
.callback(logger.log))
evo3 = (Evolution()
.repeat(evo1, n=20)
.repeat(evo2, n=20))
pop = pop.evolve(evo3, n=3)
================================================
FILE: examples/simple_logging.py
================================================
"""
This example demonstrates how logging works in evolutions.
"""
import random
from tempfile import NamedTemporaryFile
from evol import Population, Evolution
from evol.logger import BaseLogger
random.seed(42)
def random_start():
"""
This function generates a random (x,y) coordinate in the searchspace
"""
return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20
def func_to_optimise(xy):
"""
This is the function we want to optimise (maximize)
"""
x, y = xy
return -(1 - x) ** 2 - 2 * (2 - x ** 2) ** 2
def pick_random_parents(pop):
"""
This is how we are going to select parents from the population
"""
mom = random.choice(pop)
dad = random.choice(pop)
return mom, dad
def make_child(mom, dad):
"""
This is how two parents are going to make a child.
Note that the output of a tuple, just like the output of `random_start`
"""
child_x = (mom[0] + dad[0]) / 2
child_y = (mom[1] + dad[1]) / 2
return child_x, child_y
def add_noise(chromosome, sigma):
"""
This is a function that will add some noise to the chromosome.
"""
new_x = chromosome[0] + (random.random() - 0.5) * sigma
new_y = chromosome[1] + (random.random() - 0.5) * sigma
return new_x, new_y
with NamedTemporaryFile() as tmpfile:
logger = BaseLogger(target=tmpfile.name)
pop = Population(chromosomes=[random_start() for _ in range(200)],
eval_function=func_to_optimise,
maximize=True, concurrent_workers=2)
evo1 = (Evolution()
.survive(fraction=0.1)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=0.2)
.callback(logger.log))
evo2 = (Evolution()
.survive(n=10)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=0.1)
.callback(logger.log))
evo3 = (Evolution()
.repeat(evo1, n=20)
.repeat(evo2, n=20))
pop = pop.evolve(evo3, n=3)
================================================
FILE: examples/simple_nonlinear.py
================================================
import random
from random import random as r
from evol import Population, Evolution
random.seed(42)
def random_start():
"""
This function generates a random (x,y) coordinate
"""
return (r() - 0.5) * 20, (r() - 0.5) * 20
def func_to_optimise(xy):
"""
This is the function we want to optimise (maximize)
"""
x, y = xy
return -(1 - x) ** 2 - (2 - y ** 2) ** 2
def pick_random_parents(pop):
"""
This is how we are going to select parents from the population
"""
mom = random.choice(pop)
dad = random.choice(pop)
return mom, dad
def make_child(mom, dad):
"""
This function describes how two candidates combine into a
new candidate. Note that the output is a tuple, just like
the output of `random_start`. We leave it to the developer
to ensure that chromosomes are of the same type.
"""
child_x = (mom[0] + dad[0]) / 2
child_y = (mom[1] + dad[1]) / 2
return child_x, child_y
def add_noise(chromosome, sigma):
"""
This is a function that will add some noise to the chromosome.
"""
new_x = chromosome[0] + (r() - 0.5) * sigma
new_y = chromosome[1] + (r() - 0.5) * sigma
return new_x, new_y
# We start by defining a population with candidates.
pop = Population(chromosomes=[random_start() for _ in range(200)],
eval_function=func_to_optimise, maximize=True)
# We do a single step here and out comes a new population
pop.survive(fraction=0.5)
# We do two steps here and out comes a new population
(pop
.survive(fraction=0.5)
.breed(parent_picker=pick_random_parents, combiner=make_child))
# We do a three steps here and out comes a new population
(pop
.survive(fraction=0.5)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=1))
# This is inelegant but it works.
for i in range(5):
pop = (pop
.survive(fraction=0.5)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=1))
# We define a sequence of steps to change these candidates
evo1 = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=1))
# We define another sequence of steps to change these candidates
evo2 = (Evolution()
.survive(n=1)
.breed(parent_picker=pick_random_parents, combiner=make_child)
.mutate(mutate_function=add_noise, sigma=0.2))
# We are combining two evolutions into a third one.
# You don't have to but this approach demonstrates
# the flexibility of the library.
evo3 = (Evolution()
.repeat(evo1, n=50)
.repeat(evo2, n=10)
.evaluate())
# In this step we are telling evol to apply the evolutions
# to the population of candidates.
pop = pop.evolve(evo3, n=5)
print(f"the best score found: {max([i.fitness for i in pop])}")
================================================
FILE: examples/travelling_salesman.py
================================================
#!/usr/bin/env python
from argparse import ArgumentParser
from math import sqrt
from random import random, seed, shuffle
from typing import List, Optional
from evol import Evolution, Population
from evol.helpers.combiners.permutation import cycle_crossover
from evol.helpers.groups import group_stratified
from evol.helpers.mutators.permutation import swap_elements
from evol.helpers.pickers import pick_random
def run_travelling_salesman(population_size: int = 100,
n_iterations: int = 10,
random_seed: int = 0,
n_destinations: int = 50,
concurrent_workers: Optional[int] = None,
n_groups: int = 4,
silent: bool = False):
seed(random_seed)
# Generate some destinations
destinations = [(random(), random()) for _ in range(n_destinations)]
# Given a list of destination indexes, this is our cost function
def evaluate(ordered_destinations: List[int]) -> float:
total = 0
for x, y in zip(ordered_destinations, ordered_destinations[1:]):
coordinates_x = destinations[x]
coordinates_y = destinations[y]
total += sqrt((coordinates_x[0] - coordinates_y[1])**2 + (coordinates_x[1] - coordinates_y[1])**2)
return total
# This generates a random solution
def generate_solution() -> List[int]:
indexes = list(range(n_destinations))
shuffle(indexes)
return indexes
def print_function(population: Population):
if population.generation % 5000 == 0 and not silent:
print(f'{population.generation}: {population.documented_best.fitness:1.2f} / '
f'{population.current_best.fitness:1.2f}')
pop = Population.generate(generate_solution, eval_function=evaluate, maximize=False,
size=population_size * n_groups, concurrent_workers=concurrent_workers)
island_evo = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random, combiner=cycle_crossover)
.mutate(swap_elements, elitist=True))
evo = (Evolution()
.evaluate(lazy=True)
.callback(print_function)
.repeat(evolution=island_evo, n=100, grouping_function=group_stratified, n_groups=n_groups))
result = pop.evolve(evolution=evo, n=n_iterations)
if not silent:
print(f'Shortest route: {result.documented_best.chromosome}')
print(f'Route length: {result.documented_best.fitness}')
def parse_arguments():
parser = ArgumentParser(description='Run the travelling salesman example.')
parser.add_argument('--population-size', type=int, default=100,
help='the number of candidates to start the algorithm with')
parser.add_argument('--n-iterations', type=int, default=10,
help='the number of iterations to run')
parser.add_argument('--random-seed', type=int, default=42,
help='the random seed to set')
parser.add_argument('--n-destinations', type=int, default=50,
help='Number of destinations in the route.')
parser.add_argument('--n-groups', type=int, default=4,
help='Number of groups to group by.')
parser.add_argument('--concurrent-workers', type=int, default=None,
help='Concurrent workers to use to evaluate the population.')
return parser.parse_args()
if __name__ == "__main__":
args = parse_arguments()
run_travelling_salesman(**args.__dict__)
================================================
FILE: examples/very_basic_tsp.py
================================================
"""
There are a few things to notice with this example.
1. from the command line you can re-run and see a different matplotlib plot
2. `n_crossover` is set via the `.breed()` method
3. the functions you need for this application can be tested with unittests
"""
import random
import math
import argparse
from evol import Population, Evolution
def run_evolutionary(num_towns=42, population_size=100, num_iter=200, seed=42): # noqa: C901
"""
Runs a simple evolutionary algorithm against a simple TSP problem.
The goal is to explain the `evol` library, this is not an algorithm
that should perform well.
"""
# First we generate random towns as candidates with a seed
random.seed(seed)
coordinates = [(random.random(), random.random()) for i in range(num_towns)]
# Next we define a few functions that we will need in order to create an algorithm.
# Think of these functions as if they are lego blocks.
def init_func(num_towns):
"""
This function generates an individual
"""
order = list(range(num_towns))
random.shuffle(order)
return order
def dist(t1, t2):
"""
Calculates the distance between two towns.
"""
return math.sqrt((t1[0] - t2[0]) ** 2 + (t1[1] - t2[1]) ** 2)
def eval_func(order):
"""
Evaluates a candidate chromosome, which is a list that represents town orders.
"""
return sum([dist(coordinates[order[i]], coordinates[order[i - 1]]) for i, t in enumerate(order)])
def pick_random(parents):
"""
This function selects two parents
"""
return random.choice(parents), random.choice(parents)
def partition(lst, n_crossover):
division = len(lst) / n_crossover
return [lst[round(division * i):round(division * (i + 1))] for i in range(n_crossover)]
def crossover_ox(mom_order, dad_order, n_crossover):
idx_split = partition(range(len(mom_order)), n_crossover=n_crossover)
dad_idx = sum([list(d) for i, d in enumerate(idx_split) if i % 2 == 0], [])
path = [-1 for _ in range(len(mom_order))]
for idx in dad_idx:
path[idx] = dad_order[idx]
cities_visited = {p for p in path if p != -1}
for i, d in enumerate(path):
if d == -1:
city = [p for p in mom_order if p not in cities_visited][0]
path[i] = city
cities_visited.add(city)
return path
def random_flip(chromosome):
result = chromosome[:]
idx1, idx2 = random.choices(list(range(len(chromosome))), k=2)
result[idx1], result[idx2] = result[idx2], result[idx1]
return result
pop = Population(chromosomes=[init_func(num_towns) for _ in range(population_size)],
eval_function=eval_func, maximize=False, concurrent_workers=2).evaluate()
evo = (Evolution()
.survive(fraction=0.1)
.breed(parent_picker=pick_random, combiner=crossover_ox, n_crossover=2)
.mutate(random_flip)
.evaluate())
print("will start the evolutionary program")
scores = []
iterations = []
for i in range(num_iter):
print(f"iteration: {i} best score: {min([individual.fitness for individual in pop])}")
for indiviual in pop:
scores.append(indiviual.fitness)
iterations.append(i)
pop = pop.evolve(evo)
try:
import matplotlib.pylab as plt
plt.scatter(iterations, scores, s=1, alpha=0.3)
plt.title("population fitness vs. iteration")
plt.show()
except ImportError:
print("If you install matplotlib you will get a pretty plot.")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='Run an example evol algorithm against a simple TSP problem.')
parser.add_argument('--num_towns', type=int, default=42,
help='the number of towns to generate for the TSP problem')
parser.add_argument('--population_size', type=int, default=100,
help='the number of candidates to start the algorithm with')
parser.add_argument('--num_iter', type=int, default=100,
help='the number of evolutionary cycles to run')
parser.add_argument('--seed', type=int, default=42,
help='the random seed for all this')
args = parser.parse_args()
print(f"i am aware of these arguments: {args}")
run_evolutionary(num_towns=args.num_towns, population_size=args.population_size,
num_iter=args.num_iter, seed=args.seed)
================================================
FILE: setup.cfg
================================================
[metadata]
description-file=README.md
[aliases]
test=pytest
[flake8]
max-complexity=10
max-line-length=120
exclude = */__init__.py, */*/__init__.py
================================================
FILE: setup.py
================================================
import codecs
from os import path
from re import search, M
from setuptools import setup, find_packages
def load_readme():
this_directory = path.abspath(path.dirname(__file__))
with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f:
return f.read()
here = path.abspath(path.dirname(__file__))
def read(*parts):
with codecs.open(path.join(here, *parts), 'r') as fp:
return fp.read()
def find_version(*file_paths):
version_file = read(*file_paths)
version_match = search(r"^__version__ = ['\"]([^'\"]*)['\"]", version_file, M)
if version_match:
return version_match.group(1)
raise RuntimeError("Unable to find version string.")
setup(
name='evol',
version=find_version('evol', '__init__.py'),
description='A Grammar for Evolutionary Algorithms and Heuristics',
long_description=load_readme(),
long_description_content_type='text/markdown',
license='MIT License',
author=['Vincent D. Warmerdam', 'Rogier van der Geer'],
author_email='vincentwarmerdam@gmail.com',
url='https://github.com/godatadriven/evol',
packages=find_packages(),
keywords=['genetic', 'algorithms', 'heuristics'],
python_requires='>=3.6',
tests_require=[
"pytest>=3.3.1", "attrs==19.1.0", "flake8>=3.7.9"
],
extras_require={
"dev": ["pytest>=3.3.1", "attrs==19.1.0", "flake8>=3.7.9"],
"docs": ["sphinx_rtd_theme", "Sphinx>=2.0.0"],
},
setup_requires=[
"pytest-runner"
],
install_requires=[
"multiprocess>=0.70.6.1"
],
classifiers=['Intended Audience :: Developers',
'Intended Audience :: Science/Research',
'Programming Language :: Python :: 3.6',
'Development Status :: 3 - Alpha',
'License :: OSI Approved :: MIT License',
'Topic :: Scientific/Engineering',
'Topic :: Scientific/Engineering :: Artificial Intelligence']
)
================================================
FILE: test_local.sh
================================================
flake8 evol
flake8 tests
if [ -x "$(command -v py.test-3)" ]; then
py.test-3
else
python -m pytest
fi
================================================
FILE: tests/conftest.py
================================================
from random import seed, shuffle
from pytest import fixture
from evol import Individual, Population, ContestPopulation, Evolution
from evol.helpers.pickers import pick_random
@fixture(scope='module')
def simple_chromosomes():
return list(range(-50, 50))
@fixture(scope='module')
def shuffled_chromosomes():
chromosomes = list(range(0, 100)) + list(range(0, 100)) + list(range(0, 100)) + list(range(0, 100))
seed(0)
shuffle(chromosomes)
return chromosomes
@fixture(scope='function')
def simple_individuals(simple_chromosomes):
result = [Individual(chromosome=chromosome) for chromosome in simple_chromosomes]
for individual in result:
individual.fitness = 0
return result
@fixture(scope='module')
def simple_evaluation_function():
def eval_func(x):
return -x ** 2
return eval_func
@fixture(scope='function')
def evaluated_individuals(simple_chromosomes, simple_evaluation_function):
result = [Individual(chromosome=chromosome) for chromosome in simple_chromosomes]
for individual in result:
individual.fitness = individual.chromosome
return result
@fixture(scope='module')
def simple_contest_evaluation_function():
def eval_func(x, y, z):
return [1, -1, 0] if x > y else [-1, 1, 0]
return eval_func
@fixture(scope='module')
def simple_evolution():
return (
Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random, n_parents=2, combiner=lambda x, y: x + y)
.mutate(lambda x: x + 1, probability=0.1)
)
@fixture(scope='function')
def simple_population(simple_chromosomes, simple_evaluation_function):
return Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
@fixture(scope='function')
def simple_contestpopulation(simple_chromosomes, simple_contest_evaluation_function):
return ContestPopulation(chromosomes=simple_chromosomes, eval_function=simple_contest_evaluation_function,
contests_per_round=35, individuals_per_contest=3)
@fixture(scope='function', params=range(2))
def any_population(request, simple_population, simple_contestpopulation):
if request.param == 0:
return simple_population
elif request.param == 1:
return simple_contestpopulation
else:
raise ValueError("invalid internal test config")
================================================
FILE: tests/helpers/combiners/test_permutation_combiners.py
================================================
from random import seed
from evol.helpers.combiners.permutation import order_one_crossover, cycle_crossover
def test_order_one_crossover_int():
seed(53) # Fix result of select_partition
x, y = (8, 4, 7, 3, 6, 2, 5, 1, 9, 0), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
a = order_one_crossover(x, y)
assert a == (0, 4, 7, 3, 6, 2, 5, 1, 8, 9)
def test_order_one_crossover_str():
seed(53) # Fix result of select_partition
x, y = ('I', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'J', 'A'), ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')
a = order_one_crossover(x, y)
assert a == ('A', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'I', 'J')
def test_cycle_crossover_int():
x, y = (8, 4, 7, 3, 6, 2, 5, 1, 9, 0), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
a, b = cycle_crossover(x, y)
assert a == (8, 1, 2, 3, 4, 5, 6, 7, 9, 0) and b == (0, 4, 7, 3, 6, 2, 5, 1, 8, 9)
def test_cycle_crossover_str():
x, y = ('I', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'J', 'A'), ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')
a, b = cycle_crossover(x, y)
assert a == ('I', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'A')
assert b == ('A', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'I', 'J')
================================================
FILE: tests/helpers/mutators/test_permutation_mutators.py
================================================
from random import seed
from evol.helpers.mutators.permutation import inversion, swap_elements
def test_inversion_int():
seed(53) # Fix result of select_partition
x = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
a = inversion(x)
assert a == (0, 1, 2, 7, 6, 5, 4, 3, 8, 9)
def test_inversion_str():
seed(53) # Fix result of select_partition
x = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')
a = inversion(x)
assert a == ('A', 'B', 'C', 'H', 'G', 'F', 'E', 'D', 'I', 'J')
def test_swap_elements_int():
seed(0)
x = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
a = swap_elements(x)
assert a == (0, 1, 2, 3, 4, 5, 9, 7, 8, 6)
def test_swap_elements_str():
seed(0)
x = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')
a = swap_elements(x)
assert a == ('A', 'B', 'C', 'D', 'E', 'F', 'J', 'H', 'I', 'G')
================================================
FILE: tests/helpers/test_groups.py
================================================
from random import seed
from pytest import mark, raises
from evol import Population
from evol.helpers.groups import group_duplicate, group_stratified, group_random
@mark.parametrize('n_groups', [1, 2, 3, 4])
def test_group_duplicate(simple_individuals, n_groups):
indexes = group_duplicate(simple_individuals, n_groups=n_groups)
assert all(len(index) == len(set(index)) for index in indexes)
assert all(len(index) == len(simple_individuals) for index in indexes)
assert len(indexes) == n_groups
def test_group_random(simple_individuals):
seed(42)
indexes = group_random(simple_individuals, n_groups=4)
assert sum(len(index) for index in indexes) == len(simple_individuals)
seed(41)
alt_indexes = group_random(simple_individuals, n_groups=4)
assert indexes != alt_indexes
class TestGroupStratified:
def test_unique(self, evaluated_individuals):
indexes = group_stratified(evaluated_individuals, n_groups=2)
assert set(index for island in indexes for index in island) == set(range(len(evaluated_individuals)))
@mark.parametrize('n_groups', (2, 4))
def test_is_stratified(self, shuffled_chromosomes, n_groups):
population = Population(shuffled_chromosomes, eval_function=lambda x: x).evaluate()
islands = population.group(group_stratified, n_groups=n_groups)
# All islands should have the same total fitness
assert len(set(sum(map(lambda i: i.fitness, island.individuals)) for island in islands)) == 1
@mark.parametrize('n_groups', (3, 5))
def test_is_nearly_stratified(self, shuffled_chromosomes, n_groups):
population = Population(shuffled_chromosomes, eval_function=lambda x: x).evaluate()
islands = population.group(group_stratified, n_groups=n_groups)
# All islands should have roughly the same total fitness
sum_fitnesses = [sum(map(lambda i: i.fitness, island.individuals)) for island in islands]
assert max(sum_fitnesses) - min(sum_fitnesses) < n_groups * len(islands[0])
def test_must_be_evaluated(self, simple_population):
with raises(RuntimeError):
simple_population.group(group_stratified)
================================================
FILE: tests/helpers/test_helpers_utils.py
================================================
from evol.helpers.utils import rotating_window, sliding_window
def test_sliding_window():
assert list(sliding_window([1, 2, 3, 4])) == [(1, 2), (2, 3), (3, 4)]
def test_rotating_window():
assert list(rotating_window([1, 2, 3, 4])) == [(4, 1), (1, 2), (2, 3), (3, 4)]
================================================
FILE: tests/problems/test_functions.py
================================================
from evol.problems.functions import Rosenbrock, Sphere, Rastrigin
def test_rosenbrock_optimality():
problem = Rosenbrock(size=2)
assert problem.eval_function((1, 1)) == 0.0
problem = Rosenbrock(size=5)
assert problem.eval_function((1, 1, 1, 1, 1)) == 0.0
def test_sphere_optimality():
problem = Sphere(size=2)
assert problem.eval_function((0, 0)) == 0.0
problem = Sphere(size=5)
assert problem.eval_function((0, 0, 0, 0, 0)) == 0.0
def test_rastrigin_optimality():
problem = Rastrigin(size=2)
assert problem.eval_function((0, 0)) == 0.0
problem = Rastrigin(size=5)
assert problem.eval_function((0, 0, 0, 0, 0)) == 0.0
================================================
FILE: tests/problems/test_santa.py
================================================
import math
import pytest
from evol.problems.routing import MagicSanta
@pytest.fixture
def base_problem():
return MagicSanta(city_coordinates=[(0, 1), (1, 0), (1, 1)],
home_coordinate=(0, 0),
gift_weight=[0, 0, 0])
@pytest.fixture
def adv_problem():
return MagicSanta(city_coordinates=[(0, 1), (1, 1), (0, 1)],
home_coordinate=(0, 0),
gift_weight=[5, 1, 1],
sleigh_weight=2)
def test_error_raised_wrong_cities(base_problem):
# we want an error if we see too many cities
with pytest.raises(ValueError) as execinfo1:
base_problem.eval_function([[0, 1, 2, 3]])
assert "Extra: {3}" in str(execinfo1.value)
# we want an error if we see too few cities
with pytest.raises(ValueError) as execinfo2:
base_problem.eval_function([[0, 2]])
assert "Missing: {1}" in str(execinfo2.value)
# we want an error if we see multiple occurences of cities
with pytest.raises(ValueError) as execinfo3:
base_problem.eval_function([[0, 2], [0, 1]])
assert "Multiple occurrences found for cities: {0}" in str(execinfo3.value)
def test_base_score_method(base_problem):
assert base_problem.distance((0, 0), (0, 2)) == 2
expected = 1 + math.sqrt(2) + 1 + math.sqrt(2)
assert base_problem.eval_function([[0, 1, 2]]) == pytest.approx(expected)
assert base_problem.eval_function([[2, 1, 0]]) == pytest.approx(expected)
base_problem.sleigh_weight = 2
assert base_problem.eval_function([[2, 1, 0]]) == pytest.approx(2*expected)
def test_sleight_gift_weights(adv_problem):
expected = (2+7) + (2+2) + (2+1) + (2+0)
assert adv_problem.eval_function([[0, 1, 2]]) == pytest.approx(expected)
def test_multiple_routes(adv_problem):
expected = (2 + 6) + (2 + 1) + math.sqrt(2)*(2 + 0) + (2 + 1) + (2 + 0)
assert adv_problem.eval_function([[0, 1], [2]]) == pytest.approx(expected)
================================================
FILE: tests/problems/test_tsp.py
================================================
import math
import pytest
from evol.problems.routing import TSPProblem
from evol.problems.routing.coordinates import united_states_capitols
def test_distance_func():
problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])
assert problem.distance_matrix[0][0] == 0
assert problem.distance_matrix[1][0] == 1
assert problem.distance_matrix[0][1] == 1
assert problem.distance_matrix[1][1] == 0
assert problem.distance_matrix[0][2] == 1
assert problem.distance_matrix[0][3] == pytest.approx(math.sqrt(2))
def test_score_method():
problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])
expected = 1 + math.sqrt(2) + 1 + math.sqrt(2)
assert problem.eval_function([0, 1, 2, 3]) == pytest.approx(expected)
def test_score_method_can_error():
problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])
with pytest.raises(ValueError) as execinfo1:
problem.eval_function([0, 1, 2, 3, 4])
assert "Solution is longer than number of towns" in str(execinfo1.value)
with pytest.raises(ValueError) as execinfo2:
problem.eval_function([0, 1, 2])
assert "3" in str(execinfo2.value)
assert "missing" in str(execinfo2.value)
with pytest.raises(ValueError) as execinfo3:
problem.eval_function([0, 1, 2, 2])
assert "3" in str(execinfo3.value)
assert "missing" in str(execinfo3.value)
def test_can_initialize_our_datasets():
problem = TSPProblem.from_coordinates(united_states_capitols)
for i in range(len(united_states_capitols)):
assert problem.distance_matrix[i][i] == 0
================================================
FILE: tests/test_callback.py
================================================
from evol import Population, Evolution
from evol.exceptions import StopEvolution
class PopCounter:
def __init__(self):
self.count = 0
self.sum = 0
def add(self, pop):
for i in pop:
self.count += 1
self.sum += i.chromosome
class TestPopulationSimple:
def test_simple_counter_works(self, simple_chromosomes, simple_evaluation_function):
counter = PopCounter()
pop = Population(chromosomes=simple_chromosomes,
eval_function=simple_evaluation_function)
evo = (Evolution()
.mutate(lambda x: x)
.callback(lambda p: counter.add(p)))
pop.evolve(evolution=evo, n=1)
assert counter.count == len(simple_chromosomes)
assert counter.sum == sum(simple_chromosomes)
pop.evolve(evolution=evo, n=10)
assert counter.count == len(simple_chromosomes) * 11
assert counter.sum == sum(simple_chromosomes) * 11
def test_simple_counter_works_every(self, simple_chromosomes, simple_evaluation_function):
counter = PopCounter()
pop = Population(chromosomes=simple_chromosomes,
eval_function=simple_evaluation_function)
evo = (Evolution()
.mutate(lambda x: x)
.callback(lambda p: counter.add(p), every=2))
pop.evolve(evolution=evo, n=10)
assert counter.count == len(simple_chromosomes) * 5
assert counter.sum == sum(simple_chromosomes) * 5
def test_is_evaluated(self, simple_population):
def assert_is_evaluated(pop: Population):
assert pop.current_best is not None
simple_population.evaluate(lazy=True)
simple_population.callback(assert_is_evaluated)
def test_stop(self, simple_population, simple_evolution):
def func(pop):
if pop.generation == 5:
raise StopEvolution
evo = simple_evolution.callback(func)
assert simple_population.evolve(evo, n=10).generation == 5
================================================
FILE: tests/test_conditions.py
================================================
from time import monotonic, sleep
from pytest import raises
from evol import Population
from evol.conditions import Condition, MinimumProgress, TimeLimit
from evol.exceptions import StopEvolution
class TestCondition:
def test_context(self):
with Condition(lambda pop: False):
assert len(Condition.conditions) == 1
with TimeLimit(60):
assert len(Condition.conditions) == 2
assert len(Condition.conditions) == 0
def test_check(self, simple_population):
with Condition(lambda pop: False):
with raises(StopEvolution):
Condition.check(simple_population)
Condition.check(simple_population)
def test_evolve(self, simple_population, simple_evolution):
with Condition(lambda pop: pop.generation < 3):
result = simple_population.evolve(simple_evolution, n=5)
assert result.generation == 3
def test_sequential(self, simple_population, simple_evolution):
with Condition(lambda pop: pop.generation < 3):
result = simple_population.evolve(simple_evolution, n=10)
assert result.generation == 3
with Condition(lambda pop: pop.generation < 6):
result = result.evolve(simple_evolution, n=10)
assert result.generation == 6
class TestMinimumProgress:
def test_evolve(self, simple_evolution):
# The initial population contains the optimal solution.
pop = Population(list(range(100)), eval_function=lambda x: x**2, maximize=False)
with MinimumProgress(window=10):
pop = pop.evolve(simple_evolution, n=100)
assert pop.generation == 10
class TestTimeLimit:
def test_evolve(self, simple_population, simple_evolution):
evo = simple_evolution.callback(lambda p: sleep(1))
start_time = monotonic()
with TimeLimit(seconds=2):
pop = simple_population.evolve(evo, n=10)
assert monotonic() - start_time > 1
assert pop.generation == 2
================================================
FILE: tests/test_evolution.py
================================================
from pytest import mark
from evol import Evolution, Population
from evol.helpers.groups import group_random, group_duplicate, group_stratified
from evol.helpers.pickers import pick_random
class TestEvolution:
def test_add_step(self):
evo = Evolution()
assert len(evo.chain) == 0
evo_step = evo._add_step('step')
assert len(evo.chain) == 0 # original unchanged
assert evo_step.chain == ['step'] # copy with extra step
def test_repr(self):
assert repr(Evolution()) == 'Evolution()'
assert repr(Evolution().evaluate()) == 'Evolution(\n EvaluationStep())'
r = 'Evolution(\n RepeatStep() with evolution (10x):\n Evolution(\n' \
' EvaluationStep()\n SurviveStep()))'
assert repr(Evolution().repeat(Evolution().survive(fraction=0.9), n=10)) == r
class TestPopulationEvolve:
def test_repeat_step(self):
pop = Population([0 for i in range(100)], lambda x: x)
evo = Evolution().repeat(Evolution().survive(fraction=0.9), n=10)
# Check whether an Evolution inside another Evolution is actually applied
assert len(pop.evolve(evo, n=2)) < 50
@mark.parametrize('n_groups', [2, 4, 5])
@mark.parametrize('grouping_function', [group_stratified, group_duplicate, group_random])
def test_repeat_step_grouped(self, n_groups, grouping_function):
calls = []
def callback(pop):
calls.append(len(pop))
sub_evo = (
Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda x, y: x + y)
.callback(callback_function=callback)
)
pop = Population([0 for _ in range(100)], lambda x: x)
evo = (
Evolution()
.evaluate(lazy=True)
.repeat(sub_evo, grouping_function=grouping_function, n_groups=n_groups)
)
assert len(pop.evolve(evo, n=2)) == 100
assert len(calls) == 2 * n_groups
================================================
FILE: tests/test_examples.py
================================================
from pytest import mark
import sys
sys.path.append('.')
from examples.number_of_parents import run_evolutionary # noqa: E402
from examples.rock_paper_scissors import run_rock_paper_scissors # noqa: E402
from examples.travelling_salesman import run_travelling_salesman # noqa: E402
N_WORKERS = [1, 2, None]
@mark.parametrize('concurrent_workers', N_WORKERS)
def test_number_of_parents(concurrent_workers):
run_evolutionary(verbose=False, workers=concurrent_workers)
@mark.parametrize('grouped', (False, True))
def test_rock_paper_scissors(grouped):
history = run_rock_paper_scissors(silent=True, n_iterations=16, grouped=grouped)
assert len(set(h['generation'] for h in history.history)) == 16
@mark.skipif(sys.version_info < (3, 7), reason='PyTest cannot deal with the multiprocessing in 3.6.')
@mark.parametrize('n_groups', (1, 4))
def test_travelling_salesman(n_groups):
run_travelling_salesman(concurrent_workers=2, n_groups=n_groups, n_iterations=4, silent=True)
================================================
FILE: tests/test_individual.py
================================================
from copy import copy
from evol import Individual
class TestIndividual:
def test_init(self):
chromosome = (3, 4)
ind = Individual(chromosome=chromosome)
assert chromosome == ind.chromosome
assert ind.fitness is None
def test_copy(self):
individual = Individual(chromosome=(1, 2))
individual.evaluate(sum)
copied_individual = copy(individual)
assert individual.chromosome == copied_individual.chromosome
assert individual.fitness == copied_individual.fitness
assert individual.id == copied_individual.id
copied_individual.mutate(lambda x: (2, 3))
assert individual.chromosome == (1, 2)
def test_evaluate(self):
ind = Individual(chromosome=(1, 2))
ind.evaluate(sum)
def eval_func1(chromosome):
raise RuntimeError
ind.evaluate(eval_function=eval_func1, lazy=True)
assert ind.fitness == 3
def eval_func2(chromosome):
return 5
ind.evaluate(eval_function=eval_func2, lazy=False)
assert ind.fitness == 5
def test_mutate(self):
ind = Individual(chromosome=(1, 2, 3))
ind.evaluate(sum)
def mutate_func(chromosome, value):
return chromosome[0], value, chromosome[2]
ind.mutate(mutate_func, value=5)
assert (1, 5, 3) == ind.chromosome
assert ind.fitness is None
================================================
FILE: tests/test_logging.py
================================================
import random
from evol import Population, Evolution
from evol.helpers.pickers import pick_random
from evol.logger import BaseLogger, SummaryLogger
class TestLoggerSimple:
def test_baselogger_write_file_no_stdout(self, tmpdir, capsys, simple_chromosomes, simple_evaluation_function):
log_file = tmpdir.join('log.txt')
logger = BaseLogger(target=log_file, stdout=False)
pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
# we should see that a file was created with an appropriate number of rows
pop.evaluate()
logger.log(pop)
with open(log_file, "r") as f:
assert len(f.readlines()) == len(simple_chromosomes)
# we should see that a file was created with an appropriate number of rows
logger.log(pop)
with open(log_file, "r") as f:
assert len(f.readlines()) == (2 * len(simple_chromosomes))
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
# there should be nothing printed
assert len(read_stdout) == 0
def test_baselogger_can_write_to_stdout(self, capsys, simple_chromosomes):
logger = BaseLogger(target=None, stdout=True)
pop = Population(chromosomes=simple_chromosomes,
eval_function=lambda x: x)
pop.evaluate()
logger.log(pop)
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == len(pop)
def test_baselogger_can_accept_kwargs(self, tmpdir, simple_chromosomes, simple_evaluation_function):
log_file = tmpdir.join('log.txt')
logger = BaseLogger(target=log_file, stdout=False)
pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop.evaluate()
# we should see that a file was created with an appropriate number of rows
logger.log(pop, foo="bar")
with open(log_file, "r") as f:
assert len(f.readlines()) == len(simple_chromosomes)
assert all(["bar" in line for line in f.readlines()])
# we should see that a file was created with an appropriate number of rows
logger.log(pop, foo="meh")
with open(log_file, "r") as f:
assert len(f.readlines()) == (2 * len(simple_chromosomes))
assert all(['meh' in line for line in f.readlines()[-10:]])
def test_baselogger_works_via_evolution_callback(self, tmpdir, capsys):
log_file = tmpdir.join('log.txt')
logger = BaseLogger(target=log_file, stdout=True)
pop = Population(chromosomes=range(10), eval_function=lambda x: x)
evo = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda mom, dad: (mom + dad) / 2 + (random.random() - 0.5),
n_parents=2)
.callback(logger.log, foo='bar'))
pop.evolve(evolution=evo, n=2)
# check characteristics of the file
with open(log_file, "r") as f:
read_file = [item.replace("\n", "") for item in f.readlines()]
# size of the log should be appropriate
assert len(read_file) == 2 * len(pop)
# bar needs to be in every single line
assert all(['bar' in row for row in read_file])
# check characteristics of stoud
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == 2 * len(pop)
assert all(['bar' in row for row in read_stdout])
def test_summarylogger_write_file_mo_stdout(self, tmpdir, capsys):
log_file = tmpdir.join('log.txt')
logger = SummaryLogger(target=log_file, stdout=False)
pop = Population(chromosomes=range(10), eval_function=lambda x: x)
# we should see that a file was created with an appropriate number of rows
pop.evaluate()
logger.log(pop)
with open(log_file, "r") as f:
assert len(f.readlines()) == 1
# we should see that a file was created with an appropriate number of rows
logger.log(pop)
with open(log_file, "r") as f:
assert len(f.readlines()) == 2
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
# there should be nothing printed
assert len(read_stdout) == 0
def test_summarylogger_can_write_to_stdout(self, capsys, simple_chromosomes, simple_evaluation_function):
logger = SummaryLogger(target=None, stdout=True)
pop = Population(chromosomes=range(10),
eval_function=lambda x: x)
pop.evaluate()
logger.log(pop)
logger.log(pop)
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == 2
def test_summary_logger_can_accept_kwargs(self, tmpdir, simple_chromosomes, simple_evaluation_function):
log_file = tmpdir.join('log.txt')
logger = SummaryLogger(target=log_file, stdout=False)
pop = Population(chromosomes=simple_chromosomes,
eval_function=simple_evaluation_function)
pop.evaluate()
# lets make a first simple log
logger.log(pop, foo="bar", buzz="meh")
with open(log_file, "r") as f:
read_lines = f.readlines()
assert len(read_lines) == 1
first_line = read_lines[0]
assert "bar" in first_line
assert "meh" in first_line
last_entry = first_line.split(",")
assert len(last_entry) == 6
# lets log another one
logger.log(pop, buzz="moh")
with open(log_file, "r") as f:
read_lines = f.readlines()
assert len(read_lines) == 2
first_line = read_lines[-1]
assert "moh" in first_line
last_entry = first_line.split(",")
assert len(last_entry) == 5
def test_summarylogger_works_via_evolution(self, tmpdir, capsys):
log_file = tmpdir.join('log.txt')
logger = SummaryLogger(target=log_file, stdout=True)
pop = Population(chromosomes=list(range(10)), eval_function=lambda x: x)
evo = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda mom, dad: (mom + dad) / 2 + (random.random() - 0.5),
n_parents=2)
.evaluate()
.callback(logger.log, foo='bar'))
pop.evolve(evolution=evo, n=5)
# check characteristics of the file
with open(log_file, "r") as f:
read_file = [item.replace("\n", "") for item in f.readlines()]
# size of the log should be appropriate
assert len(read_file) == 5
# bar needs to be in every single line
assert all(['bar' in row for row in read_file])
# check characteristics of stoud
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == 5
assert all(['bar' in row for row in read_stdout])
def test_two_populations_can_use_same_logger(self, tmpdir, capsys):
log_file = tmpdir.join('log.txt')
logger = SummaryLogger(target=log_file, stdout=True)
pop1 = Population(chromosomes=list(range(10)), eval_function=lambda x: x)
pop2 = Population(chromosomes=list(range(10)), eval_function=lambda x: x)
evo = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda mom, dad: (mom + dad) + 1,
n_parents=2)
.evaluate()
.callback(logger.log, foo="dino"))
pop1.evolve(evolution=evo, n=5)
pop2.evolve(evolution=evo, n=5)
# two evolutions have now been applied, lets check the output!
with open(log_file, "r") as f:
read_file = [item.replace("\n", "") for item in f.readlines()]
# print(read_file)
# size of the log should be appropriate
assert len(read_file) == 10
# dino needs to be in every single line
assert all(['dino' in row for row in read_file])
# check characteristics of stoud
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == 10
assert all(['dino' in row for row in read_stdout])
def test_every_mechanic_in_evolution_log(self, tmpdir, capsys):
log_file = tmpdir.join('log.txt')
logger = SummaryLogger(target=log_file, stdout=True)
pop = Population(chromosomes=list(range(10)), eval_function=lambda x: x)
evo = (Evolution()
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda mom, dad: (mom + dad) + 1,
n_parents=2)
.evaluate()
.callback(logger.log, every=2))
pop.evolve(evolution=evo, n=100)
with open(log_file, "r") as f:
read_file = [item.replace("\n", "") for item in f.readlines()]
assert len(read_file) == 50
# check characteristics of stoud
read_stdout = [line for line in capsys.readouterr().out.split('\n') if line != '']
assert len(read_stdout) == 50
================================================
FILE: tests/test_parallel_population.py
================================================
================================================
FILE: tests/test_population.py
================================================
from time import sleep, time
import os
from copy import copy
from pytest import raises, mark
from random import random, choices, seed
from evol import Population, ContestPopulation
from evol.helpers.groups import group_duplicate, group_stratified
from evol.helpers.pickers import pick_random
from evol.population import Contest
class TestPopulationSimple:
def test_filter_works(self, simple_chromosomes, simple_evaluation_function):
pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
assert len(pop.filter(func=lambda i: random() > 0.5)) < 200
def test_population_init(self, simple_chromosomes):
pop = Population(simple_chromosomes, eval_function=lambda x: x)
assert len(pop) == len(simple_chromosomes)
assert pop.intended_size == len(pop)
def test_population_generate(self, simple_evaluation_function):
def init_func():
return 1
pop = Population.generate(init_function=init_func, eval_function=simple_evaluation_function, size=200)
assert len(pop) == 200
assert pop.intended_size == 200
assert pop.individuals[0].chromosome == 1
def test_is_evaluated(self, any_population):
assert not any_population.is_evaluated
assert any_population.evaluate().is_evaluated
class TestPopulationCopy:
def test_population_copy(self, any_population):
copied_population = copy(any_population)
for key in any_population.__dict__.keys():
if key not in ('id', 'individuals'):
assert copied_population.__dict__[key] == any_population.__dict__[key]
def test_population_is_evaluated(self, any_population):
evaluated_population = any_population.evaluate()
copied_population = copy(evaluated_population)
assert evaluated_population.is_evaluated
assert copied_population.is_evaluated
class TestPopulationEvaluate:
cpus = os.cpu_count()
latency = 0.005
def test_individuals_are_not_initially_evaluated(self, any_population):
assert all([i.fitness is None for i in any_population])
def test_evaluate_lambda(self, simple_chromosomes):
# without concurrency (note that I'm abusing a boolean operator to introduce some latency)
pop = Population(simple_chromosomes, eval_function=lambda x: (sleep(self.latency) or x))
t0 = time()
pop.evaluate()
t1 = time()
single_proc_time = t1 - t0
for individual in pop:
assert individual.chromosome == individual.fitness
# with concurrency
pop = Population(simple_chromosomes, eval_function=lambda x: (sleep(self.latency) or x),
concurrent_workers=self.cpus)
t0 = time()
pop.evaluate()
t1 = time()
multi_proc_time = t1 - t0
for individual in pop:
assert individual.chromosome == individual.fitness
if self.cpus > 1:
assert multi_proc_time < single_proc_time
def test_evaluate_func(self, simple_chromosomes):
def evaluation_function(x):
sleep(self.latency)
return x * x
pop = Population(simple_chromosomes, eval_function=evaluation_function)
t0 = time()
pop.evaluate()
t1 = time()
single_proc_time = t1 - t0
for individual in pop:
assert evaluation_function(individual.chromosome) == individual.fitness
# with concurrency
pop = Population(simple_chromosomes, eval_function=evaluation_function, concurrent_workers=self.cpus)
t0 = time()
pop.evaluate()
t1 = time()
multi_proc_time = t1 - t0
for individual in pop:
assert evaluation_function(individual.chromosome) == individual.fitness
if self.cpus > 1:
assert multi_proc_time < single_proc_time
def test_evaluate_lazy(self, any_population):
pop = any_population
pop.evaluate(lazy=True) # should evaluate
def raise_function(_):
raise Exception
pop.eval_function = raise_function
pop.evaluate(lazy=True) # should not evaluate
with raises(Exception):
pop.evaluate(lazy=False)
class TestPopulationSurvive:
def test_survive_n_works(self, simple_chromosomes, simple_evaluation_function):
pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop3 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
assert len(pop1) == len(simple_chromosomes)
assert len(pop2.survive(n=50)) == 50
assert len(pop3.survive(n=75, luck=True)) == 75
def test_survive_p_works(self, simple_chromosomes, simple_evaluation_function):
pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop3 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
assert len(pop1) == len(simple_chromosomes)
assert len(pop2.survive(fraction=0.5)) == len(simple_chromosomes) * 0.5
assert len(pop3.survive(fraction=0.1, luck=True)) == len(simple_chromosomes) * 0.1
def test_survive_n_and_p_works(self, simple_evaluation_function):
chromosomes = list(range(200))
pop1 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)
pop2 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)
pop3 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)
assert len(pop1.survive(fraction=0.5, n=200)) == 100
assert len(pop2.survive(fraction=0.9, n=10)) == 10
assert len(pop3.survive(fraction=0.5, n=190, luck=True)) == 100
def test_breed_increases_generation(self, any_population):
assert any_population.breed(parent_picker=pick_random, combiner=lambda mom, dad: mom).generation == 1
def test_survive_throws_correct_errors(self, any_population):
"""If the resulting population is zero or larger than initial we need to see errors."""
with raises(RuntimeError):
any_population.survive(n=0)
with raises(ValueError):
any_population.survive(n=250)
with raises(ValueError):
any_population.survive()
class TestPopulationBreed:
def test_breed_amount_works(self, simple_chromosomes, simple_evaluation_function):
pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop1.survive(n=50).breed(parent_picker=lambda population: choices(population, k=2),
combiner=lambda mom, dad: (mom + dad) / 2)
assert len(pop1) == len(simple_chromosomes)
pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop2.survive(n=50).breed(parent_picker=lambda population: choices(population, k=2),
combiner=lambda mom, dad: (mom + dad) / 2, population_size=400)
assert len(pop2) == 400
assert pop2.intended_size == 400
assert pop1.generation == 1
assert pop2.generation == 1
def test_breed_works_with_kwargs(self, simple_chromosomes, simple_evaluation_function):
pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop1.survive(n=50).breed(parent_picker=pick_random,
combiner=lambda mom, dad: (mom + dad) / 2,
n_parents=2)
assert len(pop1) == len(simple_chromosomes)
pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)
pop2.survive(n=50).breed(parent_picker=pick_random,
combiner=lambda *parents: sum(parents)/len(parents),
population_size=400, n_parents=3)
assert len(pop2) == 400
assert pop2.intended_size == 400
def test_breed_raises_with_multiple_values_for_kwarg(self, simple_population):
(simple_population
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda x, y: x + y))
with raises(TypeError):
(simple_population
.survive(fraction=0.5)
.breed(parent_picker=pick_random,
combiner=lambda x, y: x + y, y=2))
class TestPopulationMutate:
def test_mutate_lambda(self):
pop = Population([1]*100, eval_function=lambda x: x).mutate(lambda x: x+1)
for chromosome in pop.chromosomes:
assert chromosome == 2
assert len(pop) == 100
def test_mutate_inplace(self):
pop = Population([1]*100, eval_function=lambda x: x)
pop.mutate(lambda x: x+1)
for chromosome in pop.chromosomes:
assert chromosome == 2
def test_mutate_func(self):
def mutate_func(x):
return -x
population = Population([1]*100, eval_function=lambda x: x)
population.mutate(mutate_func)
for chromosome in population.chromosomes:
assert chromosome == -1
assert len(population) == 100
def test_mutate_probability(self):
seed(0)
pop = Population([1]*100, eval_function=lambda x: x).mutate(lambda x: x+1, probability=0.5).evaluate()
assert min(individual.chromosome for individual in pop
gitextract_y6cd46fo/
├── .gitignore
├── .gitpod.yml
├── LICENSE
├── Makefile
├── README.md
├── azure-pipelines.yml
├── doc/
│ ├── _static/
│ │ └── css/
│ │ └── custom.css
│ ├── api/
│ │ ├── evol.helpers.combiners.rst
│ │ ├── evol.helpers.mutators.rst
│ │ ├── evol.helpers.rst
│ │ ├── evol.problems.functions.rst
│ │ ├── evol.problems.routing.rst
│ │ ├── evol.problems.rst
│ │ ├── evol.rst
│ │ └── modules.rst
│ ├── conf.py
│ ├── development.rst
│ ├── index.rst
│ ├── population.rst
│ ├── problems.rst
│ └── quickstart.rst
├── evol/
│ ├── __init__.py
│ ├── conditions.py
│ ├── evolution.py
│ ├── exceptions.py
│ ├── helpers/
│ │ ├── __init__.py
│ │ ├── combiners/
│ │ │ ├── __init__.py
│ │ │ ├── permutation.py
│ │ │ └── utils.py
│ │ ├── groups.py
│ │ ├── mutators/
│ │ │ ├── __init__.py
│ │ │ └── permutation.py
│ │ ├── pickers.py
│ │ └── utils.py
│ ├── individual.py
│ ├── logger.py
│ ├── population.py
│ ├── problems/
│ │ ├── __init__.py
│ │ ├── functions/
│ │ │ ├── __init__.py
│ │ │ └── variableinput.py
│ │ ├── problem.py
│ │ └── routing/
│ │ ├── __init__.py
│ │ ├── coordinates.py
│ │ ├── magicsanta.py
│ │ └── tsp.py
│ ├── serialization.py
│ ├── step.py
│ └── utils.py
├── examples/
│ ├── number_of_parents.py
│ ├── population_demo.py
│ ├── rock_paper_scissors.py
│ ├── simple_callback.py
│ ├── simple_logging.py
│ ├── simple_nonlinear.py
│ ├── travelling_salesman.py
│ └── very_basic_tsp.py
├── setup.cfg
├── setup.py
├── test_local.sh
└── tests/
├── conftest.py
├── helpers/
│ ├── combiners/
│ │ └── test_permutation_combiners.py
│ ├── mutators/
│ │ └── test_permutation_mutators.py
│ ├── test_groups.py
│ └── test_helpers_utils.py
├── problems/
│ ├── test_functions.py
│ ├── test_santa.py
│ └── test_tsp.py
├── test_callback.py
├── test_conditions.py
├── test_evolution.py
├── test_examples.py
├── test_individual.py
├── test_logging.py
├── test_parallel_population.py
├── test_population.py
├── test_serialization.py
└── test_utils.py
SYMBOL INDEX (367 symbols across 46 files)
FILE: doc/conf.py
function setup (line 183) | def setup(app):
FILE: evol/conditions.py
class Condition (line 10) | class Condition:
method __init__ (line 18) | def __init__(self, condition: Optional[Callable[['BasePopulation'], bo...
method __enter__ (line 21) | def __enter__(self):
method __exit__ (line 25) | def __exit__(self, exc_type, exc_val, exc_tb):
method __call__ (line 28) | def __call__(self, population: 'BasePopulation') -> None:
method check (line 33) | def check(cls, population: 'BasePopulation'):
class MinimumProgress (line 38) | class MinimumProgress(Condition):
method __init__ (line 49) | def __init__(self, window: int, change: float = 0):
method __call__ (line 55) | def __call__(self, population: 'BasePopulation') -> None:
class TimeLimit (line 62) | class TimeLimit(Condition):
method __init__ (line 73) | def __init__(self, seconds: float):
method __call__ (line 78) | def __call__(self, population: 'BasePopulation'):
FILE: evol/evolution.py
class Evolution (line 18) | class Evolution:
method __init__ (line 21) | def __init__(self):
method __copy__ (line 24) | def __copy__(self) -> 'Evolution':
method __iter__ (line 29) | def __iter__(self) -> Iterator[EvolutionStep]:
method __repr__ (line 32) | def __repr__(self) -> str:
method evaluate (line 39) | def evaluate(self, lazy: bool = False, name: Optional[str] = None) -> ...
method checkpoint (line 54) | def checkpoint(self,
method map (line 70) | def map(self, func: Callable[..., Individual], name: Optional[str] = N...
method filter (line 83) | def filter(self, func: Callable[..., bool], name: Optional[str] = None...
method survive (line 95) | def survive(self,
method breed (line 122) | def breed(self,
method mutate (line 147) | def mutate(self,
method repeat (line 174) | def repeat(self, evolution: 'Evolution', n: int = 1, name: Optional[st...
method callback (line 195) | def callback(self, callback_function: Callable[..., Any],
method _add_step (line 212) | def _add_step(self, step: EvolutionStep) -> 'Evolution':
FILE: evol/exceptions.py
class PopulationIsNotEvaluatedException (line 1) | class PopulationIsNotEvaluatedException(RuntimeError):
class StopEvolution (line 5) | class StopEvolution(Exception):
FILE: evol/helpers/combiners/permutation.py
function order_one_crossover (line 9) | def order_one_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple:
function edge_recombination (line 24) | def edge_recombination(*parents: Tuple) -> Tuple:
function cycle_crossover (line 38) | def cycle_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple[Tuple[Any...
FILE: evol/helpers/combiners/utils.py
function construct_neighbors (line 8) | def construct_neighbors(*chromosome: Tuple[Any]) -> defaultdict:
function _neighbors_in (line 17) | def _neighbors_in(x: Tuple[Any], cyclic=True) -> Iterable[Tuple[Any, Any]]:
function _remove_from_neighbors (line 23) | def _remove_from_neighbors(neighbors, node):
function select_node (line 29) | def select_node(start_node: Any, neighbors: defaultdict) -> Generator[An...
function identify_cycles (line 43) | def identify_cycles(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any]) ...
function _identify_cycle (line 63) | def _identify_cycle(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any], ...
function cycle_parity (line 84) | def cycle_parity(cycles: List[Set[int]]) -> Dict[int, bool]:
FILE: evol/helpers/groups.py
function group_duplicate (line 18) | def group_duplicate(individuals: List[Individual], n_groups: int = 4) ->...
function group_random (line 29) | def group_random(individuals: List[Individual], n_groups: int = 4) -> Li...
function group_stratified (line 42) | def group_stratified(individuals: List[Individual], n_groups: int = 4) -...
function _ensure_evaluated (line 61) | def _ensure_evaluated(individuals: List[Individual]):
FILE: evol/helpers/mutators/permutation.py
function inversion (line 7) | def inversion(chromosome: Tuple[Any, ...], min_size: int = 2, max_size: ...
function swap_elements (line 21) | def swap_elements(chromosome: Tuple[Any, ...]) -> Tuple[Any, ...]:
FILE: evol/helpers/pickers.py
function pick_random (line 8) | def pick_random(parents: Sequence[Individual], n_parents: int = 2) -> Tu...
FILE: evol/helpers/utils.py
function select_partition (line 5) | def select_partition(length: int, min_size: int = 1, max_size: int = Non...
function rotating_window (line 18) | def rotating_window(arr):
function sliding_window (line 24) | def sliding_window(arr):
FILE: evol/individual.py
class Individual (line 13) | class Individual:
method __init__ (line 21) | def __init__(self, chromosome: Any, fitness: Optional[float] = None):
method __repr__ (line 27) | def __repr__(self):
method from_dict (line 31) | def from_dict(cls, data: dict) -> 'Individual':
method __post_evaluate (line 42) | def __post_evaluate(self, result):
method evaluate (line 45) | def evaluate(self, eval_function: Callable[..., float], lazy: bool = F...
method mutate (line 54) | def mutate(self, mutate_function: Callable[..., Any], probability: flo...
FILE: evol/logger.py
class BaseLogger (line 18) | class BaseLogger:
method __init__ (line 25) | def __init__(self, target=None, stdout=False, fmt='%(asctime)s,%(messa...
method check_population (line 46) | def check_population(population: BasePopulation) -> None:
method log (line 50) | def log(self, population, **kwargs):
class SummaryLogger (line 64) | class SummaryLogger(BaseLogger):
method log (line 70) | def log(self, population, **kwargs):
class MultiLogger (line 79) | class MultiLogger:
method __init__ (line 88) | def __init__(self, file_individuals, file_population):
method log (line 92) | def log(self, population, **kwargs):
method handle (line 109) | def handle(self, ind_generator, dict_to_log):
FILE: evol/population.py
class BasePopulation (line 28) | class BasePopulation(metaclass=ABCMeta):
method __init__ (line 30) | def __init__(self,
method __iter__ (line 50) | def __iter__(self) -> Iterator[Individual]:
method __getitem__ (line 53) | def __getitem__(self, i) -> Individual:
method __len__ (line 56) | def __len__(self):
method __repr__ (line 59) | def __repr__(self):
method current_best (line 63) | def current_best(self) -> Individual:
method current_worst (line 69) | def current_worst(self) -> Individual:
method chromosomes (line 75) | def chromosomes(self) -> Generator[Any, None, None]:
method is_evaluated (line 80) | def is_evaluated(self) -> bool:
method generate (line 84) | def generate(cls,
method load (line 100) | def load(cls,
method checkpoint (line 115) | def checkpoint(self, target: Optional[str] = None, method: str = 'pick...
method _individual_weights (line 128) | def _individual_weights(self):
method evolve (line 141) | def evolve(self, evolution: 'Evolution', n: int = 1) -> 'BasePopulatio...
method evaluate (line 159) | def evaluate(self, lazy: bool = False) -> 'BasePopulation':
method breed (line 162) | def breed(self,
method mutate (line 192) | def mutate(self,
method map (line 216) | def map(self, func: Callable[..., Individual], **kwargs) -> 'BasePopul...
method filter (line 227) | def filter(self, func: Callable[..., bool], **kwargs) -> 'BasePopulati...
method survive (line 240) | def survive(self, fraction: Optional[float] = None,
method callback (line 276) | def callback(self, callback_function: Callable[..., None],
method group (line 289) | def group(self, grouping_function: Callable[..., List[List[int]]] = gr...
method combine (line 316) | def combine(cls, *populations: 'BasePopulation',
method _subset (line 341) | def _subset(self, index: List[int], subset_id: str) -> 'BasePopulation':
method _update_documented_best (line 352) | def _update_documented_best(self):
class Population (line 361) | class Population(BasePopulation):
method __init__ (line 381) | def __init__(self,
method __copy__ (line 399) | def __copy__(self):
method evaluate (line 414) | def evaluate(self, lazy: bool = False) -> 'Population':
class Contest (line 438) | class Contest:
method __init__ (line 449) | def __init__(self, competitors: Iterable[Individual]):
method assign_scores (line 452) | def assign_scores(self, scores: Sequence[float]) -> None:
method competitor_chromosomes (line 457) | def competitor_chromosomes(self):
method generate (line 461) | def generate(cls, individuals: Sequence[Individual],
class ContestPopulation (line 482) | class ContestPopulation(BasePopulation):
method __init__ (line 523) | def __init__(self,
method __copy__ (line 545) | def __copy__(self):
method evaluate (line 562) | def evaluate(self,
method map (line 613) | def map(self, func: Callable[..., Individual], **kwargs) -> 'ContestPo...
method filter (line 627) | def filter(self, func: Callable[..., bool], **kwargs) -> 'ContestPopul...
method survive (line 642) | def survive(self,
method reset_fitness (line 664) | def reset_fitness(self):
FILE: evol/problems/functions/variableinput.py
class FunctionProblem (line 8) | class FunctionProblem(Problem):
method __init__ (line 9) | def __init__(self, size=2):
method check_solution (line 12) | def check_solution(self, solution: Sequence[float]) -> Sequence[float]:
method value (line 18) | def value(self, solution):
method eval_function (line 21) | def eval_function(self, solution: Sequence[float]) -> float:
class Sphere (line 26) | class Sphere(FunctionProblem):
method value (line 27) | def value(self, solution: Sequence[float]) -> float:
class Rosenbrock (line 36) | class Rosenbrock(FunctionProblem):
method value (line 37) | def value(self, solution: Sequence[float]) -> float:
class Rastrigin (line 49) | class Rastrigin(FunctionProblem):
method value (line 50) | def value(self, solution: Sequence[float]) -> float:
FILE: evol/problems/problem.py
class Problem (line 4) | class Problem(metaclass=ABCMeta):
method eval_function (line 7) | def eval_function(self, solution):
FILE: evol/problems/routing/magicsanta.py
class MagicSanta (line 10) | class MagicSanta(Problem):
method __init__ (line 11) | def __init__(self, city_coordinates, home_coordinate, gift_weight=None...
method distance (line 28) | def distance(coord_a, coord_b):
method check_solution (line 31) | def check_solution(self, solution: List[List[int]]):
method eval_function (line 48) | def eval_function(self, solution: List[List[int]]) -> Union[float, int]:
FILE: evol/problems/routing/tsp.py
class TSPProblem (line 8) | class TSPProblem(Problem):
method __init__ (line 9) | def __init__(self, distance_matrix):
method from_coordinates (line 13) | def from_coordinates(cls, coordinates: List[Union[tuple, list]]) -> 'T...
method check_solution (line 27) | def check_solution(self, solution: List[int]):
method eval_function (line 40) | def eval_function(self, solution: List[int]) -> Union[float, int]:
FILE: evol/serialization.py
class SimpleSerializer (line 19) | class SimpleSerializer:
method __init__ (line 26) | def __init__(self, target: Optional[str] = None):
method checkpoint (line 29) | def checkpoint(self, individuals: List[Individual], target: Optional[s...
method load (line 48) | def load(self, target: Optional[str] = None) -> List[Individual]:
method _new_checkpoint_file (line 66) | def _new_checkpoint_file(target: str, method: str):
method _find_checkpoint (line 78) | def _find_checkpoint(cls, target: str):
method _has_valid_extension (line 93) | def _has_valid_extension(filename: str):
FILE: evol/step.py
class EvolutionStep (line 10) | class EvolutionStep(metaclass=ABCMeta):
method __init__ (line 12) | def __init__(self, name: Optional[str], **kwargs):
method __repr__ (line 16) | def __repr__(self):
method apply (line 20) | def apply(self, population: BasePopulation) -> BasePopulation:
class EvaluationStep (line 24) | class EvaluationStep(EvolutionStep):
method apply (line 26) | def apply(self, population: BasePopulation) -> BasePopulation:
class CheckpointStep (line 30) | class CheckpointStep(EvolutionStep):
method __init__ (line 32) | def __init__(self, name, every=1, **kwargs):
method apply (line 37) | def apply(self, population: BasePopulation) -> BasePopulation:
class MapStep (line 45) | class MapStep(EvolutionStep):
method apply (line 47) | def apply(self, population: BasePopulation) -> BasePopulation:
class FilterStep (line 51) | class FilterStep(EvolutionStep):
method apply (line 53) | def apply(self, population: BasePopulation) -> BasePopulation:
class SurviveStep (line 57) | class SurviveStep(EvolutionStep):
method apply (line 59) | def apply(self, population: BasePopulation) -> BasePopulation:
class BreedStep (line 63) | class BreedStep(EvolutionStep):
method apply (line 65) | def apply(self, population: BasePopulation) -> BasePopulation:
class MutateStep (line 69) | class MutateStep(EvolutionStep):
method apply (line 71) | def apply(self, population: BasePopulation) -> BasePopulation:
class RepeatStep (line 75) | class RepeatStep(EvolutionStep):
method __init__ (line 77) | def __init__(self, name: str, evolution: 'Evolution', n: int,
method apply (line 84) | def apply(self, population: BasePopulation) -> BasePopulation:
method _apply_grouped (line 92) | def _apply_grouped(self, population: BasePopulation) -> BasePopulation:
method __repr__ (line 100) | def __repr__(self):
class CallbackStep (line 106) | class CallbackStep(EvolutionStep):
method __init__ (line 107) | def __init__(self, name, every: int = 1, **kwargs):
method apply (line 112) | def apply(self, population: BasePopulation) -> BasePopulation:
FILE: evol/utils.py
function offspring_generator (line 7) | def offspring_generator(parents: List[Individual],
function select_arguments (line 42) | def select_arguments(func: Callable) -> Callable:
FILE: examples/number_of_parents.py
function run_evolutionary (line 15) | def run_evolutionary(opt_value=1, population_size=100, n_parents=2, work...
FILE: examples/population_demo.py
function create_candidate (line 5) | def create_candidate():
function func_to_optimise (line 9) | def func_to_optimise(x):
function pick_random_parents (line 13) | def pick_random_parents(pop):
function produce_clone (line 37) | def produce_clone(parent):
function add_noise (line 41) | def add_noise(x):
FILE: examples/rock_paper_scissors.py
class RockPaperScissorsPlayer (line 13) | class RockPaperScissorsPlayer:
method __init__ (line 17) | def __init__(self, preference=None):
method __repr__ (line 20) | def __repr__(self):
method play (line 23) | def play(self):
method mutate (line 29) | def mutate(self, volatility=0.1):
method combine (line 35) | def combine(self, other):
class RockPaperScissorsLizardSpockPlayer (line 39) | class RockPaperScissorsLizardSpockPlayer(RockPaperScissorsPlayer):
function evaluate (line 57) | def evaluate(player_1: RockPaperScissorsPlayer, player_2: RockPaperSciss...
class History (line 67) | class History:
method __init__ (line 69) | def __init__(self):
method log (line 72) | def log(self, population: ContestPopulation):
method plot (line 78) | def plot(self):
function run_rock_paper_scissors (line 101) | def run_rock_paper_scissors(population_size: int = 100,
function parse_arguments (line 138) | def parse_arguments():
FILE: examples/simple_callback.py
function random_start (line 9) | def random_start():
function func_to_optimise (line 16) | def func_to_optimise(xy):
function pick_random_parents (line 24) | def pick_random_parents(pop):
function make_child (line 33) | def make_child(mom, dad):
function add_noise (line 43) | def add_noise(chromosome, sigma):
class MyLogger (line 52) | class MyLogger():
method __init__ (line 53) | def __init__(self):
method log (line 56) | def log(self, pop):
FILE: examples/simple_logging.py
function random_start (line 13) | def random_start():
function func_to_optimise (line 20) | def func_to_optimise(xy):
function pick_random_parents (line 28) | def pick_random_parents(pop):
function make_child (line 37) | def make_child(mom, dad):
function add_noise (line 47) | def add_noise(chromosome, sigma):
FILE: examples/simple_nonlinear.py
function random_start (line 8) | def random_start():
function func_to_optimise (line 15) | def func_to_optimise(xy):
function pick_random_parents (line 23) | def pick_random_parents(pop):
function make_child (line 32) | def make_child(mom, dad):
function add_noise (line 44) | def add_noise(chromosome, sigma):
FILE: examples/travelling_salesman.py
function run_travelling_salesman (line 14) | def run_travelling_salesman(population_size: int = 100,
function parse_arguments (line 65) | def parse_arguments():
FILE: examples/very_basic_tsp.py
function run_evolutionary (line 17) | def run_evolutionary(num_towns=42, population_size=100, num_iter=200, se...
FILE: setup.py
function load_readme (line 7) | def load_readme():
function read (line 16) | def read(*parts):
function find_version (line 21) | def find_version(*file_paths):
FILE: tests/conftest.py
function simple_chromosomes (line 10) | def simple_chromosomes():
function shuffled_chromosomes (line 15) | def shuffled_chromosomes():
function simple_individuals (line 23) | def simple_individuals(simple_chromosomes):
function simple_evaluation_function (line 31) | def simple_evaluation_function():
function evaluated_individuals (line 38) | def evaluated_individuals(simple_chromosomes, simple_evaluation_function):
function simple_contest_evaluation_function (line 46) | def simple_contest_evaluation_function():
function simple_evolution (line 53) | def simple_evolution():
function simple_population (line 63) | def simple_population(simple_chromosomes, simple_evaluation_function):
function simple_contestpopulation (line 68) | def simple_contestpopulation(simple_chromosomes, simple_contest_evaluati...
function any_population (line 74) | def any_population(request, simple_population, simple_contestpopulation):
FILE: tests/helpers/combiners/test_permutation_combiners.py
function test_order_one_crossover_int (line 6) | def test_order_one_crossover_int():
function test_order_one_crossover_str (line 13) | def test_order_one_crossover_str():
function test_cycle_crossover_int (line 20) | def test_cycle_crossover_int():
function test_cycle_crossover_str (line 26) | def test_cycle_crossover_str():
FILE: tests/helpers/mutators/test_permutation_mutators.py
function test_inversion_int (line 6) | def test_inversion_int():
function test_inversion_str (line 13) | def test_inversion_str():
function test_swap_elements_int (line 20) | def test_swap_elements_int():
function test_swap_elements_str (line 27) | def test_swap_elements_str():
FILE: tests/helpers/test_groups.py
function test_group_duplicate (line 10) | def test_group_duplicate(simple_individuals, n_groups):
function test_group_random (line 17) | def test_group_random(simple_individuals):
class TestGroupStratified (line 26) | class TestGroupStratified:
method test_unique (line 28) | def test_unique(self, evaluated_individuals):
method test_is_stratified (line 33) | def test_is_stratified(self, shuffled_chromosomes, n_groups):
method test_is_nearly_stratified (line 40) | def test_is_nearly_stratified(self, shuffled_chromosomes, n_groups):
method test_must_be_evaluated (line 47) | def test_must_be_evaluated(self, simple_population):
FILE: tests/helpers/test_helpers_utils.py
function test_sliding_window (line 4) | def test_sliding_window():
function test_rotating_window (line 8) | def test_rotating_window():
FILE: tests/problems/test_functions.py
function test_rosenbrock_optimality (line 4) | def test_rosenbrock_optimality():
function test_sphere_optimality (line 11) | def test_sphere_optimality():
function test_rastrigin_optimality (line 18) | def test_rastrigin_optimality():
FILE: tests/problems/test_santa.py
function base_problem (line 9) | def base_problem():
function adv_problem (line 16) | def adv_problem():
function test_error_raised_wrong_cities (line 23) | def test_error_raised_wrong_cities(base_problem):
function test_base_score_method (line 38) | def test_base_score_method(base_problem):
function test_sleight_gift_weights (line 47) | def test_sleight_gift_weights(adv_problem):
function test_multiple_routes (line 52) | def test_multiple_routes(adv_problem):
FILE: tests/problems/test_tsp.py
function test_distance_func (line 7) | def test_distance_func():
function test_score_method (line 17) | def test_score_method():
function test_score_method_can_error (line 23) | def test_score_method_can_error():
function test_can_initialize_our_datasets (line 41) | def test_can_initialize_our_datasets():
FILE: tests/test_callback.py
class PopCounter (line 5) | class PopCounter:
method __init__ (line 6) | def __init__(self):
method add (line 10) | def add(self, pop):
class TestPopulationSimple (line 16) | class TestPopulationSimple:
method test_simple_counter_works (line 17) | def test_simple_counter_works(self, simple_chromosomes, simple_evaluat...
method test_simple_counter_works_every (line 33) | def test_simple_counter_works_every(self, simple_chromosomes, simple_e...
method test_is_evaluated (line 46) | def test_is_evaluated(self, simple_population):
method test_stop (line 53) | def test_stop(self, simple_population, simple_evolution):
FILE: tests/test_conditions.py
class TestCondition (line 10) | class TestCondition:
method test_context (line 12) | def test_context(self):
method test_check (line 19) | def test_check(self, simple_population):
method test_evolve (line 25) | def test_evolve(self, simple_population, simple_evolution):
method test_sequential (line 30) | def test_sequential(self, simple_population, simple_evolution):
class TestMinimumProgress (line 39) | class TestMinimumProgress:
method test_evolve (line 41) | def test_evolve(self, simple_evolution):
class TestTimeLimit (line 49) | class TestTimeLimit:
method test_evolve (line 51) | def test_evolve(self, simple_population, simple_evolution):
FILE: tests/test_evolution.py
class TestEvolution (line 8) | class TestEvolution:
method test_add_step (line 10) | def test_add_step(self):
method test_repr (line 17) | def test_repr(self):
class TestPopulationEvolve (line 25) | class TestPopulationEvolve:
method test_repeat_step (line 27) | def test_repeat_step(self):
method test_repeat_step_grouped (line 35) | def test_repeat_step_grouped(self, n_groups, grouping_function):
FILE: tests/test_examples.py
function test_number_of_parents (line 15) | def test_number_of_parents(concurrent_workers):
function test_rock_paper_scissors (line 20) | def test_rock_paper_scissors(grouped):
function test_travelling_salesman (line 27) | def test_travelling_salesman(n_groups):
FILE: tests/test_individual.py
class TestIndividual (line 6) | class TestIndividual:
method test_init (line 8) | def test_init(self):
method test_copy (line 14) | def test_copy(self):
method test_evaluate (line 24) | def test_evaluate(self):
method test_mutate (line 40) | def test_mutate(self):
FILE: tests/test_logging.py
class TestLoggerSimple (line 8) | class TestLoggerSimple:
method test_baselogger_write_file_no_stdout (line 10) | def test_baselogger_write_file_no_stdout(self, tmpdir, capsys, simple_...
method test_baselogger_can_write_to_stdout (line 27) | def test_baselogger_can_write_to_stdout(self, capsys, simple_chromosom...
method test_baselogger_can_accept_kwargs (line 37) | def test_baselogger_can_accept_kwargs(self, tmpdir, simple_chromosomes...
method test_baselogger_works_via_evolution_callback (line 53) | def test_baselogger_works_via_evolution_callback(self, tmpdir, capsys):
method test_summarylogger_write_file_mo_stdout (line 76) | def test_summarylogger_write_file_mo_stdout(self, tmpdir, capsys):
method test_summarylogger_can_write_to_stdout (line 93) | def test_summarylogger_can_write_to_stdout(self, capsys, simple_chromo...
method test_summary_logger_can_accept_kwargs (line 103) | def test_summary_logger_can_accept_kwargs(self, tmpdir, simple_chromos...
method test_summarylogger_works_via_evolution (line 129) | def test_summarylogger_works_via_evolution(self, tmpdir, capsys):
method test_two_populations_can_use_same_logger (line 153) | def test_two_populations_can_use_same_logger(self, tmpdir, capsys):
method test_every_mechanic_in_evolution_log (line 180) | def test_every_mechanic_in_evolution_log(self, tmpdir, capsys):
FILE: tests/test_population.py
class TestPopulationSimple (line 14) | class TestPopulationSimple:
method test_filter_works (line 16) | def test_filter_works(self, simple_chromosomes, simple_evaluation_func...
method test_population_init (line 20) | def test_population_init(self, simple_chromosomes):
method test_population_generate (line 25) | def test_population_generate(self, simple_evaluation_function):
method test_is_evaluated (line 34) | def test_is_evaluated(self, any_population):
class TestPopulationCopy (line 39) | class TestPopulationCopy:
method test_population_copy (line 41) | def test_population_copy(self, any_population):
method test_population_is_evaluated (line 47) | def test_population_is_evaluated(self, any_population):
class TestPopulationEvaluate (line 54) | class TestPopulationEvaluate:
method test_individuals_are_not_initially_evaluated (line 59) | def test_individuals_are_not_initially_evaluated(self, any_population):
method test_evaluate_lambda (line 62) | def test_evaluate_lambda(self, simple_chromosomes):
method test_evaluate_func (line 83) | def test_evaluate_func(self, simple_chromosomes):
method test_evaluate_lazy (line 105) | def test_evaluate_lazy(self, any_population):
class TestPopulationSurvive (line 118) | class TestPopulationSurvive:
method test_survive_n_works (line 120) | def test_survive_n_works(self, simple_chromosomes, simple_evaluation_f...
method test_survive_p_works (line 128) | def test_survive_p_works(self, simple_chromosomes, simple_evaluation_f...
method test_survive_n_and_p_works (line 136) | def test_survive_n_and_p_works(self, simple_evaluation_function):
method test_breed_increases_generation (line 145) | def test_breed_increases_generation(self, any_population):
method test_survive_throws_correct_errors (line 148) | def test_survive_throws_correct_errors(self, any_population):
class TestPopulationBreed (line 158) | class TestPopulationBreed:
method test_breed_amount_works (line 160) | def test_breed_amount_works(self, simple_chromosomes, simple_evaluatio...
method test_breed_works_with_kwargs (line 173) | def test_breed_works_with_kwargs(self, simple_chromosomes, simple_eval...
method test_breed_raises_with_multiple_values_for_kwarg (line 186) | def test_breed_raises_with_multiple_values_for_kwarg(self, simple_popu...
class TestPopulationMutate (line 200) | class TestPopulationMutate:
method test_mutate_lambda (line 202) | def test_mutate_lambda(self):
method test_mutate_inplace (line 208) | def test_mutate_inplace(self):
method test_mutate_func (line 214) | def test_mutate_func(self):
method test_mutate_probability (line 223) | def test_mutate_probability(self):
method test_mutate_zero_probability (line 232) | def test_mutate_zero_probability(self):
method test_mutate_func_kwargs (line 237) | def test_mutate_func_kwargs(self):
method test_mutate_elitist (line 244) | def test_mutate_elitist(self):
class TestPopulationWeights (line 251) | class TestPopulationWeights:
method test_weights (line 253) | def test_weights(self, simple_chromosomes, simple_evaluation_function):
class TestPopulationBest (line 268) | class TestPopulationBest:
method test_current_best (line 270) | def test_current_best(self, simple_chromosomes):
method test_current_worst (line 277) | def test_current_worst(self, simple_chromosomes):
method test_mutate_resets (line 284) | def test_mutate_resets(self):
method test_documented_best (line 292) | def test_documented_best(self):
class TestPopulationIslands (line 301) | class TestPopulationIslands:
method test_groups (line 304) | def test_groups(self, simple_population, n_groups):
method test_no_groups (line 310) | def test_no_groups(self, simple_population):
method test_empty_group (line 314) | def test_empty_group(self, simple_population):
method test_invalid_group (line 326) | def test_invalid_group(self, simple_population, result, error):
method test_not_evaluated (line 333) | def test_not_evaluated(self, simple_population):
method test_combine (line 337) | def test_combine(self, simple_population):
method test_combine_nothing (line 342) | def test_combine_nothing(self):
class TestContest (line 347) | class TestContest:
method test_assign_score (line 349) | def test_assign_score(self, simple_individuals):
method test_generate_n_contests (line 356) | def test_generate_n_contests(self, simple_individuals, individuals_per...
class TestContestPopulation (line 370) | class TestContestPopulation:
method test_init (line 372) | def test_init(self):
class TestContestPopulationBest (line 378) | class TestContestPopulationBest:
method test_no_documented (line 380) | def test_no_documented(self):
FILE: tests/test_serialization.py
class TestPickleCheckpoint (line 8) | class TestPickleCheckpoint:
method test_checkpoint (line 13) | def test_checkpoint(self, tmpdir, simple_population):
method test_unserializable_chromosome (line 19) | def test_unserializable_chromosome(self, tmpdir):
method test_load (line 30) | def test_load(self, tmpdir, simple_population):
method test_load_invalid_target (line 37) | def test_load_invalid_target(self, tmpdir):
method test_checkpoint_invalid_target (line 48) | def test_checkpoint_invalid_target(self, tmpdir, simple_population):
method test_override_default_path (line 58) | def test_override_default_path(self, tmpdir, simple_chromosomes, simpl...
method test_evolution (line 76) | def test_evolution(self, tmpdir, simple_population):
method test_every (line 82) | def test_every(self, tmpdir, simple_population):
class TestJsonCheckpoint (line 91) | class TestJsonCheckpoint(TestPickleCheckpoint):
FILE: tests/test_utils.py
class TestOffspringGenerator (line 8) | class TestOffspringGenerator:
method test_simple_combiner (line 10) | def test_simple_combiner(self, simple_population: Population):
method test_args (line 20) | def test_args(self, n_parents: int, simple_population: Population):
method test_simple_picker (line 30) | def test_simple_picker(self, simple_population: Population):
method test_multiple_offspring (line 41) | def test_multiple_offspring(self, simple_population: Population):
class TestSelectArguments (line 53) | class TestSelectArguments:
method test_no_kwargs (line 56) | def test_no_kwargs(self, args, kwargs, result):
method test_with_kwargs (line 63) | def test_with_kwargs(self, args, kwargs, result):
method test_all_kwargs (line 70) | def test_all_kwargs(self, args, kwargs, result):
Condensed preview — 77 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (216K chars).
[
{
"path": ".gitignore",
"chars": 1284,
"preview": "# Created by .ignore support plugin (hsz.mobi)\n### Python template\n# Byte-compiled / optimized / DLL files\n__pycache__/\n"
},
{
"path": ".gitpod.yml",
"chars": 61,
"preview": "tasks:\n - init: pyenv local 3.7.2 && pip install -e \".[dev]\""
},
{
"path": "LICENSE",
"chars": 1073,
"preview": "MIT License\n\nCopyright (c) 2017-2020 GoDataDriven\n\nPermission is hereby granted, free of charge, to any person obtaining"
},
{
"path": "Makefile",
"chars": 485,
"preview": ".PHONY: docs\n\nflake:\n\tpython setup.py flake8\n\ninstall:\n\tpip install -e .\n\ndevelop:\n\tpip install -e \".[dev]\"\n\tpython setu"
},
{
"path": "README.md",
"chars": 5674,
"preview": "[](https://evol.readthedocs.io/en/la"
},
{
"path": "azure-pipelines.yml",
"chars": 1646,
"preview": "trigger:\n- master\npr:\n- master\n\npool:\n vmImage: 'ubuntu-latest'\n\n\nstages:\n- stage: Test\n jobs:\n - job: TestJob\n st"
},
{
"path": "doc/_static/css/custom.css",
"chars": 290,
"preview": ".wy-nav-side{\n\tbackground-color: #f2f2f2;\n\tcolor: black;\n}\n\n.wy-side-nav-search{\n background-color: #404040;\n}\n\n.wy-n"
},
{
"path": "doc/api/evol.helpers.combiners.rst",
"chars": 592,
"preview": "evol.helpers.combiners package\n==============================\n\nSubmodules\n----------\n\nevol.helpers.combiners.generic mod"
},
{
"path": "doc/api/evol.helpers.mutators.rst",
"chars": 404,
"preview": "evol.helpers.mutators package\n=============================\n\nSubmodules\n----------\n\nevol.helpers.mutators.permutation mo"
},
{
"path": "doc/api/evol.helpers.rst",
"chars": 577,
"preview": "evol.helpers package\n====================\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.helpers.combiners\n evol.he"
},
{
"path": "doc/api/evol.problems.functions.rst",
"chars": 422,
"preview": "evol.problems.functions package\n===============================\n\nSubmodules\n----------\n\nevol.problems.functions.variable"
},
{
"path": "doc/api/evol.problems.routing.rst",
"chars": 759,
"preview": "evol.problems.routing package\n=============================\n\nSubmodules\n----------\n\nevol.problems.routing.coordinates mo"
},
{
"path": "doc/api/evol.problems.rst",
"chars": 438,
"preview": "evol.problems package\n=====================\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.problems.functions\n evol"
},
{
"path": "doc/api/evol.rst",
"chars": 1035,
"preview": "evol package\n============\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.helpers\n evol.problems\n\nSubmodules\n-------"
},
{
"path": "doc/api/modules.rst",
"chars": 49,
"preview": "evol\n====\n\n.. toctree::\n :maxdepth: 4\n\n evol\n"
},
{
"path": "doc/conf.py",
"chars": 5497,
"preview": "import evol\n\n# -*- coding: utf-8 -*-\n#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does onl"
},
{
"path": "doc/development.rst",
"chars": 1150,
"preview": ".. image:: https://i.imgur.com/7MHcIq1.png\n :align: center\n\nDevelopment\n===========\n\nInstalling from PyPi\n^^^^^^^^^^^^"
},
{
"path": "doc/index.rst",
"chars": 1806,
"preview": ".. evol documentation master file, created by\n sphinx-quickstart on Thu Apr 4 09:34:54 2019.\n You can adapt this fi"
},
{
"path": "doc/population.rst",
"chars": 3303,
"preview": "Population Guide\n================\n\nThe \"Population\" object in **evol** is the base container for all your\ncandidate solu"
},
{
"path": "doc/problems.rst",
"chars": 3106,
"preview": "Problem Guide\n=============\n\nCertain problems are general enough, if only for educational\npurposes, to include into our "
},
{
"path": "doc/quickstart.rst",
"chars": 7264,
"preview": "\nQuick-Start Guide\n=================\n\nThe goal is this document is to build the pipeline you see below.\n\n.. image:: _sta"
},
{
"path": "evol/__init__.py",
"chars": 3952,
"preview": "\"\"\"\n\n\n`Evol` is clear dsl for composable evolutionary algorithms that optimised"
},
{
"path": "evol/conditions.py",
"chars": 2786,
"preview": "from time import monotonic\nfrom typing import Callable, Optional, TYPE_CHECKING\n\nfrom evol.exceptions import StopEvoluti"
},
{
"path": "evol/evolution.py",
"chars": 9659,
"preview": "\"\"\"\nEvolution objects in `evol` are objects that describe how the\nevolutionary algorithm will change members of a popula"
},
{
"path": "evol/exceptions.py",
"chars": 107,
"preview": "class PopulationIsNotEvaluatedException(RuntimeError):\n pass\n\n\nclass StopEvolution(Exception):\n pass\n"
},
{
"path": "evol/helpers/__init__.py",
"chars": 138,
"preview": "\"\"\"\nHelpers in `evol` are functions that help you when you are \ndesigning algorithms. We archive these helping functions"
},
{
"path": "evol/helpers/combiners/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "evol/helpers/combiners/permutation.py",
"chars": 1942,
"preview": "from itertools import islice, tee\nfrom random import choice\nfrom typing import Any, Tuple\n\nfrom .utils import select_nod"
},
{
"path": "evol/helpers/combiners/utils.py",
"chars": 3212,
"preview": "from collections import defaultdict\nfrom itertools import tee, islice, cycle\n\nfrom random import choice\nfrom typing impo"
},
{
"path": "evol/helpers/groups.py",
"chars": 2558,
"preview": "from random import shuffle\nfrom typing import List\n\nfrom evol import Individual\nfrom evol.exceptions import PopulationIs"
},
{
"path": "evol/helpers/mutators/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "evol/helpers/mutators/permutation.py",
"chars": 1063,
"preview": "from random import sample\nfrom typing import Any, Tuple\n\nfrom ..utils import select_partition\n\n\ndef inversion(chromosome"
},
{
"path": "evol/helpers/pickers.py",
"chars": 373,
"preview": "from typing import Sequence, Tuple\n\nfrom random import choice\n\nfrom evol import Individual\n\n\ndef pick_random(parents: Se"
},
{
"path": "evol/helpers/utils.py",
"chars": 974,
"preview": "from random import randint\nfrom typing import Tuple\n\n\ndef select_partition(length: int, min_size: int = 1, max_size: int"
},
{
"path": "evol/individual.py",
"chars": 2427,
"preview": "\"\"\"\nIndividual objects in `evol` are a wrapper around a chromosome.\nInternally we work with individuals because that all"
},
{
"path": "evol/logger.py",
"chars": 4833,
"preview": "\"\"\"\nLoggers help keep track of the workings of your evolutionary algorithm. By\ndefault, each Population is initialized w"
},
{
"path": "evol/population.py",
"chars": 30750,
"preview": "\"\"\"\nPopulation objects in `evol` are a collection of chromosomes\nat some point in an evolutionary algorithm. You can app"
},
{
"path": "evol/problems/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "evol/problems/functions/__init__.py",
"chars": 308,
"preview": "\"\"\"\nThe `evol.problems.functions` part of the library contains\nsimple problem instances that do with known math function"
},
{
"path": "evol/problems/functions/variableinput.py",
"chars": 1878,
"preview": "import math\nfrom typing import Sequence\n\nfrom evol.helpers.utils import sliding_window\nfrom evol.problems.problem import"
},
{
"path": "evol/problems/problem.py",
"chars": 170,
"preview": "from abc import ABCMeta, abstractmethod\n\n\nclass Problem(metaclass=ABCMeta):\n\n @abstractmethod\n def eval_function(s"
},
{
"path": "evol/problems/routing/__init__.py",
"chars": 340,
"preview": "\"\"\"\nThe `evol.problems.routing` part of the library contains\nsimple problem instances that do with routing problems. The"
},
{
"path": "evol/problems/routing/coordinates.py",
"chars": 2581,
"preview": "united_states_capitols = [\n (32.361538, -86.279118, \"Montgomery\", \"Alabama\"),\n (58.301935, -134.419740, \"Juneau\", "
},
{
"path": "evol/problems/routing/magicsanta.py",
"chars": 3097,
"preview": "import math\nfrom collections import Counter\nfrom itertools import chain\nfrom typing import List, Union\n\nfrom evol.helper"
},
{
"path": "evol/problems/routing/tsp.py",
"chars": 2053,
"preview": "import math\nfrom typing import List, Union\n\nfrom evol.problems.problem import Problem\nfrom evol.helpers.utils import rot"
},
{
"path": "evol/serialization.py",
"chars": 4376,
"preview": "\"\"\"\nSerializers help store (checkpoint) the state of your population during or\nafter running your evolutionary algorithm"
},
{
"path": "evol/step.py",
"chars": 3761,
"preview": "from abc import ABCMeta, abstractmethod\nfrom typing import Callable, Optional, TYPE_CHECKING\n\nfrom evol.population impor"
},
{
"path": "evol/utils.py",
"chars": 2418,
"preview": "from inspect import signature\nfrom typing import List, Callable, Union, Sequence, Any, Generator\n\nfrom evol import Indiv"
},
{
"path": "examples/number_of_parents.py",
"chars": 3119,
"preview": "\"\"\"\nThere are a few worthwhile things to notice in this example:\n\n1. you can pass hyperparams into functions from the `."
},
{
"path": "examples/population_demo.py",
"chars": 1295,
"preview": "import random\nfrom evol import Population\n\n\ndef create_candidate():\n return random.random() - 0.5\n\n\ndef func_to_optim"
},
{
"path": "examples/rock_paper_scissors.py",
"chars": 6284,
"preview": "#!/usr/bin/env python\n\nfrom argparse import ArgumentParser\nfrom collections import Counter\nfrom random import choice, ra"
},
{
"path": "examples/simple_callback.py",
"chars": 2269,
"preview": "\"\"\"\nThis example demonstrates how logging works in evolutions.\n\"\"\"\n\nimport random\nfrom evol import Population, Evolution"
},
{
"path": "examples/simple_logging.py",
"chars": 2126,
"preview": "\"\"\"\nThis example demonstrates how logging works in evolutions.\n\"\"\"\n\nimport random\nfrom tempfile import NamedTemporaryFil"
},
{
"path": "examples/simple_nonlinear.py",
"chars": 2956,
"preview": "import random\nfrom random import random as r\nfrom evol import Population, Evolution\n\nrandom.seed(42)\n\n\ndef random_start("
},
{
"path": "examples/travelling_salesman.py",
"chars": 3659,
"preview": "#!/usr/bin/env python\nfrom argparse import ArgumentParser\nfrom math import sqrt\nfrom random import random, seed, shuffle"
},
{
"path": "examples/very_basic_tsp.py",
"chars": 4651,
"preview": "\"\"\"\nThere are a few things to notice with this example.\n\n1. from the command line you can re-run and see a different mat"
},
{
"path": "setup.cfg",
"chars": 149,
"preview": "[metadata]\ndescription-file=README.md\n\n[aliases]\ntest=pytest\n\n[flake8]\nmax-complexity=10\nmax-line-length=120\nexclude = *"
},
{
"path": "setup.py",
"chars": 1995,
"preview": "import codecs\nfrom os import path\nfrom re import search, M\nfrom setuptools import setup, find_packages\n\n\ndef load_readme"
},
{
"path": "test_local.sh",
"chars": 105,
"preview": "flake8 evol\nflake8 tests\nif [ -x \"$(command -v py.test-3)\" ]; then\n py.test-3\nelse\n python -m pytest\nfi"
},
{
"path": "tests/conftest.py",
"chars": 2376,
"preview": "from random import seed, shuffle\n\nfrom pytest import fixture\n\nfrom evol import Individual, Population, ContestPopulation"
},
{
"path": "tests/helpers/combiners/test_permutation_combiners.py",
"chars": 1192,
"preview": "from random import seed\n\nfrom evol.helpers.combiners.permutation import order_one_crossover, cycle_crossover\n\n\ndef test_"
},
{
"path": "tests/helpers/mutators/test_permutation_mutators.py",
"chars": 850,
"preview": "from random import seed\n\nfrom evol.helpers.mutators.permutation import inversion, swap_elements\n\n\ndef test_inversion_int"
},
{
"path": "tests/helpers/test_groups.py",
"chars": 2183,
"preview": "from random import seed\n\nfrom pytest import mark, raises\n\nfrom evol import Population\nfrom evol.helpers.groups import gr"
},
{
"path": "tests/helpers/test_helpers_utils.py",
"chars": 279,
"preview": "from evol.helpers.utils import rotating_window, sliding_window\n\n\ndef test_sliding_window():\n assert list(sliding_wind"
},
{
"path": "tests/problems/test_functions.py",
"chars": 672,
"preview": "from evol.problems.functions import Rosenbrock, Sphere, Rastrigin\n\n\ndef test_rosenbrock_optimality():\n problem = Rose"
},
{
"path": "tests/problems/test_santa.py",
"chars": 1969,
"preview": "import math\n\nimport pytest\n\nfrom evol.problems.routing import MagicSanta\n\n\n@pytest.fixture\ndef base_problem():\n retur"
},
{
"path": "tests/problems/test_tsp.py",
"chars": 1621,
"preview": "import math\nimport pytest\nfrom evol.problems.routing import TSPProblem\nfrom evol.problems.routing.coordinates import uni"
},
{
"path": "tests/test_callback.py",
"chars": 2043,
"preview": "from evol import Population, Evolution\nfrom evol.exceptions import StopEvolution\n\n\nclass PopCounter:\n def __init__(se"
},
{
"path": "tests/test_conditions.py",
"chars": 2020,
"preview": "from time import monotonic, sleep\n\nfrom pytest import raises\n\nfrom evol import Population\nfrom evol.conditions import Co"
},
{
"path": "tests/test_evolution.py",
"chars": 2028,
"preview": "from pytest import mark\n\nfrom evol import Evolution, Population\nfrom evol.helpers.groups import group_random, group_dupl"
},
{
"path": "tests/test_examples.py",
"chars": 996,
"preview": "from pytest import mark\nimport sys\n\nsys.path.append('.')\n\nfrom examples.number_of_parents import run_evolutionary # noq"
},
{
"path": "tests/test_individual.py",
"chars": 1424,
"preview": "from copy import copy\n\nfrom evol import Individual\n\n\nclass TestIndividual:\n\n def test_init(self):\n chromosome "
},
{
"path": "tests/test_logging.py",
"chars": 9518,
"preview": "import random\n\nfrom evol import Population, Evolution\nfrom evol.helpers.pickers import pick_random\nfrom evol.logger impo"
},
{
"path": "tests/test_parallel_population.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/test_population.py",
"chars": 17377,
"preview": "from time import sleep, time\n\nimport os\nfrom copy import copy\nfrom pytest import raises, mark\nfrom random import random,"
},
{
"path": "tests/test_serialization.py",
"chars": 3988,
"preview": "from os import listdir\nfrom pytest import raises\n\nfrom evol import Population, Evolution\nfrom evol.serialization import "
},
{
"path": "tests/test_utils.py",
"chars": 2827,
"preview": "from pytest import mark\n\nfrom evol import Population, Individual\nfrom evol.helpers.pickers import pick_random\nfrom evol."
}
]
About this extraction
This page contains the full source code of the godatadriven/evol GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 77 files (199.8 KB), approximately 49.5k tokens, and a symbol index with 367 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.