[ { "path": ".gitignore", "content": "# Created by .ignore support plugin (hsz.mobi)\n### Python template\n# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nenv/\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.coverage\n.coverage.*\n.cache\nnosetests.xml\ncoverage.xml\n*,cover\n.hypothesis/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\nlocal_settings.py\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# pyenv\n.python-version\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# dotenv\n.env\n\n# virtualenv\n.venv\nvenv/\nENV/\nevol-env/\n\n# Spyder project settings\n.spyderproject\n\n# Rope project settings\n.ropeproject\n\n# VSCode\n.vscode/\n\n# PyCharm\n.idea/\n\n# Mac\n.DS_Store\n\n# Pytest\n.pytest_cache/\n\n# documentation build folder\ndocs" }, { "path": ".gitpod.yml", "content": "tasks:\n - init: pyenv local 3.7.2 && pip install -e \".[dev]\"" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2017-2020 GoDataDriven\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE." }, { "path": "Makefile", "content": ".PHONY: docs\n\nflake:\n\tpython setup.py flake8\n\ninstall:\n\tpip install -e .\n\ndevelop:\n\tpip install -e \".[dev]\"\n\tpython setup.py develop\n\ntest:\n\tpython setup.py test\n\ncheck: test flake\n\ndocs:\n\tsphinx-apidoc -f -o doc/api evol\n\tsphinx-build doc docs\n\nclean:\n\trm -rf .cache\n\trm -rf .eggs\n\trm -rf .pytest_cache\n\trm -rf build\n\trm -rf dist\n\trm -rf evol.egg-info\n\trm -rf .ipynb_checkpoints\n\npush:\n\trm -rf dist\n\tpython setup.py sdist\n\tpython setup.py bdist_wheel --universal\n\ttwine upload dist/*\n" }, { "path": "README.md", "content": "[![Documentation Status](https://readthedocs.org/projects/evol/badge/?version=latest)](https://evol.readthedocs.io/en/latest/?badge=latest)[![Downloads](https://pepy.tech/badge/evol)](https://pepy.tech/project/evol)\n[![Build Status](https://dev.azure.com/godatadriven/evol/_apis/build/status/godatadriven.evol?branchName=master)](https://dev.azure.com/godatadriven/evol/_build/latest?definitionId=9&branchName=master) [![Documentation Status](https://readthedocs.org/projects/evol/badge/?version=latest)](https://evol.readthedocs.io/en/latest/?badge=latest)[![Downloads](https://pepy.tech/badge/evol)](https://pepy.tech/project/evol)\n\n\n![Imgur](https://i.imgur.com/7MHcIq1.png)\n\n`Evol` is clear dsl for composable evolutionary algorithms that optimised for joy.\n\n## Installation\n\nWe currently support python3.6 and python3.7 and you can install it via pip.\n\n```\npip install evol\n```\n\n## Documentation\n\nFor more details you can read the [docs](https://evol.readthedocs.io/en/latest/) but we advice everyone to get start by first checking out the examples in the `/examples` directory. These stand alone examples should show the spirit of usage better than the docs.\n\n## The Gist\n\nThe main idea is that you should be able to define a complex algorithm\nin a composable way. To explain what we mean by this: let's consider\ntwo evolutionary algorithms for travelling salesman problems.\n\nThe first approach takes a collections of solutions and applies:\n\n1. a survival where only the top 50% solutions survive\n2. the population reproduces using a crossover of genes\n3. certain members mutate\n4. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nWe can also think of another approach:\n\n1. pick the best solution of the population\n2. make random changes to this parent and generate new solutions\n3. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nOne could even combine the two algorithms into a new one:\n\n1. run algorithm 1 50 times\n2. run algorithm 2 10 times\n3. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nYou might notice that many parts of these algorithms are similar and it\nis the goal of this library is to automate these parts. We hope to\nprovide an API that is fun to use and easy to tweak your heuristics in.\n\nA working example of something silimar to what is depicted above is shown below. You can also find this code as an example in the `/examples/simple_nonlinear.py`. \n\n```python\nimport random\nfrom evol import Population, Evolution\n\nrandom.seed(42)\n\ndef random_start():\n \"\"\"\n This function generates a random (x,y) coordinate\n \"\"\"\n return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20\n\ndef func_to_optimise(xy):\n \"\"\"\n This is the function we want to optimise (maximize)\n \"\"\"\n x, y = xy\n return -(1-x)**2 - 2*(2-x**2)**2\n\ndef pick_random_parents(pop):\n \"\"\"\n This is how we are going to select parents from the population\n \"\"\"\n mom = random.choice(pop)\n dad = random.choice(pop)\n return mom, dad\n\ndef make_child(mom, dad):\n \"\"\"\n This function describes how two candidates combine into a new candidate\n Note that the output is a tuple, just like the output of `random_start`\n We leave it to the developer to ensure that chromosomes are of the same type\n \"\"\"\n child_x = (mom[0] + dad[0])/2\n child_y = (mom[1] + dad[1])/2\n return child_x, child_y\n\ndef add_noise(chromosome, sigma):\n \"\"\"\n This is a function that will add some noise to the chromosome.\n \"\"\"\n new_x = chromosome[0] + (random.random()-0.5) * sigma\n new_y = chromosome[1] + (random.random()-0.5) * sigma\n return new_x, new_y\n\n# We start by defining a population with candidates.\npop = Population(chromosomes=[random_start() for _ in range(200)],\n eval_function=func_to_optimise, maximize=True)\n\n# We define a sequence of steps to change these candidates\nevo1 = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(func=add_noise, sigma=1))\n\n# We define another sequence of steps to change these candidates\nevo2 = (Evolution()\n .survive(n=1)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(func=add_noise, sigma=0.2))\n\n# We are combining two evolutions into a third one. You don't have to\n# but this approach demonstrates the flexibility of the library.\nevo3 = (Evolution()\n .repeat(evo1, n=50)\n .repeat(evo2, n=10)\n .evaluate())\n\n# In this step we are telling evol to apply the evolutions\n# to the population of candidates.\npop = pop.evolve(evo3, n=5)\nprint(f\"the best score found: {max([i.fitness for i in pop])}\")\n```\n\nGetting Started\n---------------------------------------\n\nThe best place to get started is the `/examples` folder on github.\nThis folder contains self contained examples that work out of the\nbox.\n\n## How does it compare to ...\n\n- [... deap?](https://github.com/DEAP/deap) We think our library is more composable and pythonic while not removing any functionality. Our library may be a bit slower though.\n- [... hyperopt?](http://jaberg.github.io/hyperopt/) Since we force the user to make the actual algorithm we are less black boxy. Hyperopt is meant for hyperparameter tuning for machine learning and has better support for search in scikit learn.\n- [... inspyred?](https://pypi.org/project/inspyred/) The library offers a simple way to get started but it seems the project is less actively maintained than ours. \n" }, { "path": "azure-pipelines.yml", "content": "trigger:\n- master\npr:\n- master\n\npool:\n vmImage: 'ubuntu-latest'\n\n\nstages:\n- stage: Test\n jobs:\n - job: TestJob\n strategy:\n matrix:\n Python36:\n python.version: '3.6'\n Python37:\n python.version: '3.7'\n Python38:\n python.version: '3.8'\n steps:\n - task: UsePythonVersion@0\n inputs:\n versionSpec: '$(python.version)'\n\n - bash: |\n pip install --upgrade pip\n pip install -e .[dev]\n displayName: 'Install'\n\n - bash: flake8\n displayName: 'Flake'\n\n - bash: python setup.py test\n displayName: 'Tests'\n \n - bash: |\n set -e\n python examples/simple_nonlinear.py\n python examples/number_of_parents.py --n-parents=2 --workers=1\n python examples/number_of_parents.py --n-parents=3 --workers=1\n python examples/number_of_parents.py --n-parents=4 --workers=1\n python examples/number_of_parents.py --n-parents=2 --workers=2\n python examples/number_of_parents.py --n-parents=3 --workers=2\n python examples/number_of_parents.py --n-parents=4 --workers=2\n python examples/very_basic_tsp.py\n python examples/simple_logging.py\n python examples/rock_paper_scissors.py\n\n\n- stage: Docs\n condition: eq(variables['build.sourceBranch'], 'refs/heads/master')\n jobs:\n - job: DocsJob\n steps:\n - bash: |\n set -e\n pip install --upgrade pip\n pip install -e .[docs]\n sphinx-apidoc -f -o doc/api evol\n sphinx-build doc public\n\n - task: PublishBuildArtifacts@1\n inputs:\n pathToPublish: public\n artifactName: BuildOutput\n" }, { "path": "doc/_static/css/custom.css", "content": ".wy-nav-side{\n\tbackground-color: #f2f2f2;\n\tcolor: black;\n}\n\n.wy-side-nav-search{\n background-color: #404040;\n}\n\n.wy-nav-content{\n\tbackground-color: #ffffff;\n}\n\n.wy-nav-content-wrap{\n\tbackground-color: #ffffff;\n}\n\na.reference.internal{\n\tcolor: black;\n}\n\npre{\n background: #eeeeee29;\n}\n" }, { "path": "doc/api/evol.helpers.combiners.rst", "content": "evol.helpers.combiners package\n==============================\n\nSubmodules\n----------\n\nevol.helpers.combiners.generic module\n-------------------------------------\n\n.. automodule:: evol.helpers.combiners.generic\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.helpers.combiners.permutation module\n-----------------------------------------\n\n.. automodule:: evol.helpers.combiners.permutation\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.helpers.combiners\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.helpers.mutators.rst", "content": "evol.helpers.mutators package\n=============================\n\nSubmodules\n----------\n\nevol.helpers.mutators.permutation module\n----------------------------------------\n\n.. automodule:: evol.helpers.mutators.permutation\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.helpers.mutators\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.helpers.rst", "content": "evol.helpers package\n====================\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.helpers.combiners\n evol.helpers.mutators\n\nSubmodules\n----------\n\nevol.helpers.pickers module\n---------------------------\n\n.. automodule:: evol.helpers.pickers\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.helpers.utils module\n-------------------------\n\n.. automodule:: evol.helpers.utils\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.helpers\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.problems.functions.rst", "content": "evol.problems.functions package\n===============================\n\nSubmodules\n----------\n\nevol.problems.functions.variableinput module\n--------------------------------------------\n\n.. automodule:: evol.problems.functions.variableinput\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.problems.functions\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.problems.routing.rst", "content": "evol.problems.routing package\n=============================\n\nSubmodules\n----------\n\nevol.problems.routing.coordinates module\n----------------------------------------\n\n.. automodule:: evol.problems.routing.coordinates\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.problems.routing.magicsanta module\n---------------------------------------\n\n.. automodule:: evol.problems.routing.magicsanta\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.problems.routing.tsp module\n--------------------------------\n\n.. automodule:: evol.problems.routing.tsp\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.problems.routing\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.problems.rst", "content": "evol.problems package\n=====================\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.problems.functions\n evol.problems.routing\n\nSubmodules\n----------\n\nevol.problems.problem module\n----------------------------\n\n.. automodule:: evol.problems.problem\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol.problems\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/evol.rst", "content": "evol package\n============\n\nSubpackages\n-----------\n\n.. toctree::\n\n evol.helpers\n evol.problems\n\nSubmodules\n----------\n\nevol.evolution module\n---------------------\n\n.. automodule:: evol.evolution\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.individual module\n----------------------\n\n.. automodule:: evol.individual\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.logger module\n------------------\n\n.. automodule:: evol.logger\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.population module\n----------------------\n\n.. automodule:: evol.population\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.serialization module\n-------------------------\n\n.. automodule:: evol.serialization\n :members:\n :undoc-members:\n :show-inheritance:\n\nevol.step module\n----------------\n\n.. automodule:: evol.step\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: evol\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "doc/api/modules.rst", "content": "evol\n====\n\n.. toctree::\n :maxdepth: 4\n\n evol\n" }, { "path": "doc/conf.py", "content": "import evol\n\n# -*- coding: utf-8 -*-\n#\n# Configuration file for the Sphinx documentation builder.\n#\n# This file does only contain a selection of the most common options. For a\n# full list see the documentation:\n# http://www.sphinx-doc.org/en/master/config\n\n# -- Path setup --------------------------------------------------------------\n\n# If extensions (or modules to document with autodoc) are in another directory,\n# add these directories to sys.path here. If the directory is relative to the\n# documentation root, use os.path.abspath to make it absolute, like shown here.\n#\n# import os\n# import sys\n# sys.path.insert(0, os.path.abspath('.'))\n\n\n# -- Project information -----------------------------------------------------\n\nproject = 'evol'\ncopyright = '2019, Vincent D. Warmerdam & Rogier van der Geer'\nauthor = 'Vincent D. Warmerdam & Rogier van der Geer'\n\n# The short X.Y version\nversion = ''\n# The full version, including alpha/beta/rc tags\nrelease = evol.__version__\n\n\n# -- General configuration ---------------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom\n# ones.\nextensions = [\n 'sphinx.ext.autodoc',\n 'sphinx.ext.mathjax',\n 'sphinx.ext.viewcode',\n]\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of doc filenames.\n# You can specify multiple suffix as a list of string:\n#\n# source_suffix = ['.rst', '.md']\nsource_suffix = '.rst'\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to doc directory, that match files and\n# directories to ignore when looking for doc files.\n# This pattern also affects html_static_path and html_extra_path.\nexclude_patterns = []\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = None\n\n\n# -- Options for HTML output -------------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\nhtml_theme = 'sphinx_rtd_theme'\n\n# Theme options are theme-specific and customize the look and feel of a theme\n# further. For a list of options available for each theme, see the\n# documentation.\n#\n# html_theme_options = {}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\n\n# Custom sidebar templates, must be a dictionary that maps document names\n# to template names.\n#\n# The default sidebars (for documents that don't match any pattern) are\n# defined by theme itself. Builtin themes are using these templates by\n# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',\n# 'searchbox.html']``.\n#\n# html_sidebars = {}\n\n\n# -- Options for HTMLHelp output ---------------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'evoldoc'\n\n\n# -- Options for LaTeX output ------------------------------------------------\n\nlatex_elements = {\n # The paper size ('letterpaper' or 'a4paper').\n #\n # 'papersize': 'letterpaper',\n\n # The font size ('10pt', '11pt' or '12pt').\n #\n # 'pointsize': '10pt',\n\n # Additional stuff for the LaTeX preamble.\n #\n # 'preamble': '',\n\n # Latex figure (float) alignment\n #\n # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (doc start file, target name, title,\n# author, documentclass [howto, manual, or own class]).\nlatex_documents = [\n (master_doc, 'evol.tex', 'evol Documentation',\n 'Vincent D. Warmerdam \\\\& Rogier van der Geer', 'manual'),\n]\n\n\n# -- Options for manual page output ------------------------------------------\n\n# One entry per manual page. List of tuples\n# (doc start file, name, description, authors, manual section).\nman_pages = [\n (master_doc, 'evol', 'evol Documentation',\n [author], 1)\n]\n\n\n# -- Options for Texinfo output ----------------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (doc start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n (master_doc, 'evol', 'evol Documentation',\n author, 'evol', 'One line description of project.',\n 'Miscellaneous'),\n]\n\n\n# -- Options for Epub output -------------------------------------------------\n\n# Bibliographic Dublin Core info.\nepub_title = project\n\n# The unique identifier of the text. This can be a ISBN number\n# or the project homepage.\n#\n# epub_identifier = ''\n\n# A unique identification for the text.\n#\n# epub_uid = ''\n\n# A list of files that should not be packed into the epub file.\nepub_exclude_files = ['search.html']\n\n\n# -- Extension configuration -------------------------------------------------\n\ndef setup(app):\n print(\"Custom part of setup is now running...\")\n app.add_stylesheet('css/custom.css')\n print(\"Custom part of setup is now complete.\")\n" }, { "path": "doc/development.rst", "content": ".. image:: https://i.imgur.com/7MHcIq1.png\n :align: center\n\nDevelopment\n===========\n\nInstalling from PyPi\n^^^^^^^^^^^^^^^^^^^^\n\nWe currently support python3.6 and python3.7 and you can install it via pip.\n\n.. code-block:: bash\n\n pip install evol\n\nDeveloping Locally with Makefile\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nYou can also fork/clone the repository on Github_ to work on it locally. we've\nadded a `Makefile` to the project that makes it easy to install everything ready\nfor development.\n\n.. code-block:: bash\n\n make develop\n\nThere's some other helpful commands in there. For example, testing can be done via;\n\n.. code-block:: bash\n\n make test\n\nThis will pytest and possibly in the future also the docstring tests.\n\nGenerating Documentation\n^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe easiest way to generate documentation is by running:\n\n.. code-block:: bash\n\n make docs\n\nThis will populate the `/docs` folder locally. Note that we ignore the\ncontents of the this folder per git ignore because building the documentation\nis something that we outsource to the read-the-docs service.\n\n.. _Github: https://scikit-learn.org/stable/modules/compose.html\n\n" }, { "path": "doc/index.rst", "content": ".. evol documentation master file, created by\n sphinx-quickstart on Thu Apr 4 09:34:54 2019.\n You can adapt this file completely to your liking, but it should at least\n contain the root `toctree` directive.\n\n**Evol** is a clear dsl for composable evolutionary algorithms, optimised for joy.\n\n.. image:: _static/evol.png\n :align: center\n\n.. code-block:: bash\n\n pip install evol\n\nThe Gist\n********\n\nThe main idea is that you should be able to define a complex algorithm\nin a composable way. To explain what we mean by this: let's consider\ntwo evolutionary algorithms for travelling salesman problems.\n\nThe first approach takes a collections of solutions and applies:\n\n1. a survival where only the top 50% solutions survive\n2. the population reproduces using a crossover of genes\n3. certain members mutate\n4. repeat this, maybe 1000 times or more!\n\n.. image:: https://i.imgur.com/is9g07u.png\n :align: center\n\nWe can also think of another approach:\n\n1. pick the best solution of the population\n2. make random changes to this parent and generate new solutions\n3. repeat this, maybe 1000 times or more!\n\n.. image:: https://i.imgur.com/JRSWbTd.png\n :align: center\n\nOne could even combine the two algorithms into a new one:\n\n1. run algorithm 1 50 times\n2. run algorithm 2 10 times\n3. repeat this, maybe 1000 times or more!\n\n.. image:: https://i.imgur.com/SZTBWX2.png\n :align: center\n\nYou might notice that many parts of these algorithms are similar and it\nis the goal of this library is to automate these parts. We hope to\nprovide an API that is fun to use and easy to tweak your heuristics in.\n\n.. toctree::\n :maxdepth: 2\n :caption: Contents:\n\n quickstart\n population\n problems\n development\n\nIndices and tables\n==================\n\n* :ref:`genindex`\n* :ref:`modindex`\n* :ref:`search`\n" }, { "path": "doc/population.rst", "content": "Population Guide\n================\n\nThe \"Population\" object in **evol** is the base container for all your\ncandidate solutions. Each candidate solution (sometimes referred to\nas a chromosome in the literature) lives inside of the population as\nan \"Individual\" and has a fitness score attached as a property.\n\n.. image:: _static/population-1.png\n :align: center\n\nYou do not typically deal with \"Individual\" objects directly but it\nis useful to know that they are data containers that have a chromosome\nproperty as well as a fitness property.\n\nCreation\n********\n\nIn order to create a population you need an evaluation function and either:\n\n1. a collection of candidate solutions\n2. a function that can generate candidate solutions\n\nBoth methods of initialising a population are demonstrated below.\n\n.. literalinclude:: ../examples/population_demo.py\n :lines: 1-25\n\nLazy Evaluation\n***************\n\nIf we were to now query the contents of the population object\nyou can use a for loop to view some of the contents.\n\n.. code-block:: python\n\n > [i for i in pop1]\n [,\n ,\n ,\n ,\n ]\n > [i.chromosome for i in pop1]\n [ 0.13942679845788375,\n -0.47498924477733306,\n -0.22497068163088074,\n -0.27678926185117725,\n 0.2364712141640124]\n\nYou might be slightly suprised by the following result though.\n\n.. code-block:: python\n\n > [i.fitness for i in pop1]\n [None, None, None, None, None]\n\nThe fitness property seems to not exist. But if we call the \"evaluate\"\nmethod first then suddenly it does seem to make an appearance.\n\n.. code-block:: python\n\n > [i.fitness for i in pop1.evaluate()]\n [ 0.2788535969157675,\n -0.9499784895546661,\n -0.4499413632617615,\n -0.5535785237023545,\n 0.4729424283280248]\n\nThere is some logic behind this. Typically the evaluation function\ncan be very expensive to calculate so you might want to consider running\nit as late as possible and as few times as possible. The only command\nthat needs a fitness is the \"survive\" method. All other methods can apply\ntransformations to the chromosome without needing to evaluate the fitness.\n\nMore Lazyness\n*************\n\nTo demonstrate the effect of this lazyness, let's see the effect\nof the fitness of the individuals.\n\nFirst, note that after a survive method everything is evaluated.\n\n.. code-block:: python\n\n > [i.fitness for i in pop1.survive(n=3)]\n [0.4729424283280248, 0.2788535969157675, -0.4499413632617615]\n\nIf we were to add a mutate step afterwards we will see that the\nlazyness kicks in again. Only if we add an evaluate step will we\nsee fitness values again.\n\n.. code-block:: python\n\n def add_noise(x):\n return 0.1 * (random.random() - 0.5) + x\n\n > [i.fitness for i in pop1.survive(n=3).mutate(add_noise)]\n [None, None, None]\n > [i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()]\n [0.3564375534260752, 0.30990154209234466, -0.5356458732043454]\n\nIf you want to work with fitness values explicitly it is good to know\nabout this, otherwise the library will try to be as conservative as\npossible when it comes to evaluating the fitness function.\n" }, { "path": "doc/problems.rst", "content": "Problem Guide\n=============\n\nCertain problems are general enough, if only for educational\npurposes, to include into our API. This guide will demonstrate\nsome of problems that are included in evol.\n\nGeneral Idea\n------------\n\nIn general a problem in evol is nothing more than an object\nthat has `.eval_function()` implemented. This object can\nusually be initialised in different ways but the method\nmust always be implemented.\n\nFunction Problems\n-----------------\n\nThere are a few hard functions out there that can be optimised\nwith heuristics. Our library offers a few objects with this\nimplementation.\n\nThe following functions are implemented.\n\n.. code-block:: python\n\n from evol.problems.functions import Rastrigin, Sphere, Rosenbrock\n\n Rastrigin(size=1).eval_function([1])\n Sphere(size=2).eval_function([2, 1])\n Rosenbrock(size=3).eval_function([3, 2, 1])\n\nYou may notice that we pass a size parameter apon initialisation; this\nis because these functions can also be defined in higher dimensions.\nFeel free to check the wikipedia_ article for more explanation on these functions.\n\n\nRouting Problems\n----------------\n\nTraveling Salesman Problem\n**************************\n\nIt's a classic problem so we've included it here.\n\n.. code-block:: python\n\n import random\n from evol.problems.routing import TSPProblem, coordinates\n\n us_cities = coordinates.united_states_capitols\n problem = TSPProblem.from_coordinates(coordinates=us_cities)\n\n order = list(range(len(us_cities)))\n for i in range(3):\n random.shuffle(order)\n print(problem.eval_function(order))\n\nNote that you can also create an instance of a TSP problem\nfrom a distance matrix instead. Also note that you can get\nsuch a distance matrix from the object.\n\n.. code:: python\n\n same_problem = TSPProblem(problem.distance_matrix)\n print(same_problem.eval_function(order))\n\nMagic Santa\n***********\n\nThis problem was inspired by a kaggle_ competition. It involves the logistics\nof delivering gifts all around the world from the north pole. The costs of\ndelivering a gift depend on how tired santa's reindeer get while delivering\na sleigh full of gifts during a trip.\n\n\nIt is better explained on the website than here but the goal is to\nminimize the weighed reindeer weariness defined below:\n\n:math:`WRW = \\sum\\limits_{j=1}^{m} \\sum\\limits_{i=1}^{n} \\Big[ \\big( \\sum\\limits_{k=1}^{n} w_{kj} - \\sum\\limits_{k=1}^{i} w_{kj} \\big) \\cdot Dist(Loc_i, Loc_{i-1})`\n\nIn terms of setting up the problem it is very similar to a TSP except that\nwe now also need to attach the weight of a gift per location.\n\n.. code:: python\n\n import random\n from evol.problems.routing import MagicSanta, coordinates\n\n us_cities = coordinates.united_states_capitols\n problem = TSPProblem.from_coordinates(coordinates=us_cities)\n\n MagicSanta(city_coordinates=us_cities,\n home_coordinate=(0, 0),\n gift_weight=[random.random() for _ in us_cities])\n\n.. _wikipedia: https://en.wikipedia.org/wiki/Test_functions_for_optimization\n.. _kaggle: https://www.kaggle.com/c/santas-stolen-sleigh#evaluation" }, { "path": "doc/quickstart.rst", "content": "\nQuick-Start Guide\n=================\n\nThe goal is this document is to build the pipeline you see below.\n\n.. image:: _static/quickstart-step-6.png\n :align: center\n\nThis guide will offer a step by step guide on how to use evol\nto write custom heuristic solutions to problems. As an example\nwe will try to optimise the following non-linear function:\n\n:math:`f(x, y) = -(1-x)^2 - (2 - y^2)^2`\n\nStep 1: Score\n^^^^^^^^^^^^^\n\nThe first thing we need to do for evol is to describe how\n\"good\" a solution to a problem is. To facilitate this we\ncan write a simple function.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :lines: 15-20\n\nYou'll notice that this function accepts a \"solution\" to\nthe problem and it returns a value. In this case the \"solution\"\nis a list that contains two elements. Inside the function we\nunpack it but the function that we have needs to accept one\n\"candidate\"-solution and return one score.\n\nStep 2: Sample\n^^^^^^^^^^^^^^\n\nAnother thing we need is something that can create\nrandom candidates. We want our algorithm to start searching\nsomewhere and we prefer to start with different candidates\ninstead of a static set. The function below will generate\nsuch candidates.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :lines: 8-12\n\nNote that one candidate from this function will create\na tuple; one that can be unpacked by the function we've defined\nbefore.\n\nStep 3: Create\n^^^^^^^^^^^^^^\n\nWith these two functions we can create a population of\ncandidates. Below we generate a population with 200 random\ncandidates.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 54-55\n\nThis population object is merely a container for candidates.\nThe next step is to define things that we might want to do\nwith it.\n\nIf we were to draw where we currently are, it'd be here:\n\n.. image:: _static/quickstart-step-1.png\n :align: center\n\nStep 4: Survive\n^^^^^^^^^^^^^^^\n\nNow that we have a population we might add a bit of code that\ncan remove candidates that are not performing as well. This means\nthat we add a step to our \"pipeline\".\n\n.. image:: _static/quickstart-step-2.png\n :align: center\n\nTo facilitate this we merely need to call a method on our population object.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 57-58\n\nBecause the population knows what it needs to optimise for\nit is easy to halve the population size by simply calling this method.\nThis method call will return a new population object that has fewer\nmembers. A next step might be to take these remaining candidates and\nto use them to create new candidates that are similar.\n\nStep 5: Breed\n^^^^^^^^^^^^^\n\nIn order to evolve the candidates we need to start generating\nnew candites again. This adds another step to our pipeline:\n\n.. image:: _static/quickstart-step-3.png\n :align: center\n\nNote that in this view the highlighted candidates are the new ones\nthat have been created. The candidates who were already performing\nvery well are still in the population.\n\nTo generate new candidates we need to do two things:\n\n1. we need to determine what parents will be used to create a new individual\n2. we need to determine how these parent candidates create a new one\n\nBoth steps needs to be defined in functions. First, we write\na simple function that will select two random parents from\nthe population.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 23-29\n\nNext we need a function that can merge the properties of these\ntwo parents such that we create a new candidate.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 32-41\n\nWith these two functions we can expand our initial pipeline\nand expand it with a breed step.\n\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 60-63\n\nStep 6: Mutate\n^^^^^^^^^^^^^^\n\nTypically when searching for a good candidate we might want\nto add some entropy in the system. The idea being that a bit\nof random search might indeed help us explore areas that we\nmight not otherwise consider.\n\n.. image:: _static/quickstart-step-4.png\n :align: center\n\nThe idea is to add a bit of noise to every single datapoint.\nThis ensures that our population of candidates does not converge\ntoo fast towards a single datapoint and that we are able to\nexplore the search space.\n\nTo faciliate this in our pipeline we first need to create\na function that can take a candidate and apply the noise.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 44-50\n\nNext we need to add this as a step in our pipeline.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 65-69\n\nStep 7: Repeat\n^^^^^^^^^^^^^^\n\nWe're getting really close to where we want to be now but\nwe still need to discuss how to repeat our steps.\n\n.. image:: _static/quickstart-step-5.png\n :align: center\n\nOne way of getting there is to literally repeat the code\nwe saw earlier in a for loop.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 71-76\n\nThis sort of works, but there is a more elegant method.\n\nStep 8: Evolve\n^^^^^^^^^^^^^^\n\nThe problem with the previous method is that we don't\njust want to repeat but we also want to supply settings\nto our evolution steps that might change over time. To\nfacilitate this our api offers the `Evolution` object.\n\n.. image:: _static/quickstart-step-6.png\n :align: center\n\nYou can see a `Population` as a container for candidates\nand can `Evolution` as a container for changes to the\npopulation. You can use the exact same verbs in the method\nchain to specify what you'd like to see happen but it allows\nyou much more fledixbility.\n\nThe code below demonstrates an example of evolution steps.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 78-82\n\nThe code below demonstrates a slightly different set of steps.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 84-88\n\nEvolutions are kind of flexible, we can combine these two\nevolutions into a third one.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 90-96\n\nNow if you'd like to apply this evolution we've added a method\nfor that on top of our evolution object.\n\n.. literalinclude:: ../examples/simple_nonlinear.py\n :language: python\n :lines: 98-101\n\nStep 9: Dream\n^^^^^^^^^^^^^^\n\nThese steps together give us an evolution program depicted below.\n\n.. image:: _static/quickstart-step-7.png\n :align: center\n\nThe goal of evol is to make it easy to write heuristic pipelines\nthat can help search towards a solution. Note that you don't need\nto write a genetic algorithm here. You could also implement simulated\nannealing in our library just as easily but we want to help you standardise\nyour code such that testing, monitoring, parallism and checkpoint becomes\nmore joyful.\n\nEvol will help you structure your pipeline by giving a language that\ntells you *what* is happening but not *how* this is being done. For this\nyou will need to write functions yourself because our library has no\nnotion of your specific problem.\n\nWe hope this makes writing heuristic software more fun." }, { "path": "evol/__init__.py", "content": "\"\"\"\n![Imgur](https://i.imgur.com/7MHcIq1.png)\n\n`Evol` is clear dsl for composable evolutionary algorithms that optimised for joy.\n\nEvol is a library that helps you make evolutionary algorithms. The\ngoal is to have a library that is fun and clear to use, but not fast.\n\nIf you're looking at the library for the first time we recommend\nthat you first take a look at the examples in the /examples folder\non github. It is usually a better starting point to get started.\n\nAny details can be discovered on the docs. We hope that this\nlibrary makes it fun to write heuristics again.\n\nThe Gist\n---------------------------------------\n\nThe main idea is that you should be able to define a complex algorithm\nin a composable way. To explain what we mean by this: let's consider\ntwo evolutionary algorithms for travelling salesman problems.\n\nThe first approach takes a collections of solutions and applies:\n\n1. a survival where only the top 50% solutions survive\n2. the population reproduces using a crossover of genes\n3. certain members mutate\n4. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nWe can also think of another approach:\n\n1. pick the best solution of the population\n2. make random changes to this parent and generate new solutions\n3. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nOne could even combine the two algorithms into a new one:\n\n1. run algorithm 1 50 times\n2. run algorithm 2 10 times\n3. repeat this, maybe 1000 times or more!\n\n\"Drawing\"\n\nYou might notice that many parts of these algorithms are similar and it is\nthe goal of this library is to automate these parts. In fact, you can\nexpect the code for these algorithms to look something like this.\n\nA speudo-example of what is decribed about looks a bit like this:\n\n import random\n from evol import Population, Evolution\n\n population = Population(init_func=init_func, eval_func=eval_func, size=100)\n\n def pick_n_parents(population, num_parents):\n return [random.choice(population) for i in range(num_parents)]\n\n def crossover(*parents):\n ...\n\n def random_copy(parent):\n ...\n\n evo1 = (Evolution(name=\"first_algorithm\")\n .survive(fraction=0.5)\n .breed(parentpicker=pick_n_parents,\n combiner=combiner,\n num_parents=2, n_max=100)\n .mutate(lambda x: add_noise(x, 0.1)))\n\n evo2 = (Evolution(name=\"second_algorithm\")\n .survive(n=1)\n .breed(parentpicker=pick_n_parents,\n combiner=random_copy,\n num_parents=1, n_max=100))\n\n for i in range(1001):\n population.evolve(evo1, n=50).evolve(evo2, n=10)\n\nGetting Started\n---------------------------------------\n\nThe best place to get started is the `/examples` folder on github.\nThis folder contains self contained examples that work out of the\nbox.\n\nContributing Guide\n---------------------------------------\n\n### Local development\n\nPython can help you. Don't reinstall all the time, rather use a\nvirtulenv that has a link to the code.\n\n python setup.py develop\n\nWhen you submit a pull request it will be tested in travis. Once\nthe build is green the review can start. Please try to keep your\nbranch up to date to ensure you're not missing any tests. Larger\ncommits need to be discussed in github before we can accept them.\n\n### Generating New Documentation\n\nUpdating documentation is currently a manual step. From the `docs` folder:\n\n pdoc --html --overwrite evol\n cp -rf evol/* .\n rm -rf evol\n\nIf you want to confirm that it works you can open the `index.html` file.\n\"\"\"\n\nfrom .individual import Individual\nfrom .population import Population, ContestPopulation\nfrom .evolution import Evolution\nfrom .logger import BaseLogger\n\n__version__ = \"0.5.3\"\n" }, { "path": "evol/conditions.py", "content": "from time import monotonic\nfrom typing import Callable, Optional, TYPE_CHECKING\n\nfrom evol.exceptions import StopEvolution\n\nif TYPE_CHECKING:\n from evol.population import BasePopulation\n\n\nclass Condition:\n \"\"\"Stop the evolution until a condition is no longer met.\n\n :param condition: A function that accepts a Population and returns a boolean.\n If the condition does not evaluate to True, then the evolution is stopped.\n \"\"\"\n conditions = set()\n\n def __init__(self, condition: Optional[Callable[['BasePopulation'], bool]]):\n self.condition = condition\n\n def __enter__(self):\n self.conditions.add(self)\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb):\n self.conditions.remove(self)\n\n def __call__(self, population: 'BasePopulation') -> None:\n if self.condition and not self.condition(population):\n raise StopEvolution()\n\n @classmethod\n def check(cls, population: 'BasePopulation'):\n for condition in cls.conditions:\n condition(population)\n\n\nclass MinimumProgress(Condition):\n \"\"\"Stop the evolution if not enough progress is made.\n\n This condition stops the evolution if the best documented fitness\n does not improve enough within a given number of iterations.\n\n :param window: Number of iterations in which the minimum improvement must be made.\n :param change: Require more change in fitness than this value.\n Defaults to 0, meaning any change is good enough.\n \"\"\"\n\n def __init__(self, window: int, change: float = 0):\n super().__init__(condition=None)\n self._history = []\n self.change = change\n self.window = window\n\n def __call__(self, population: 'BasePopulation') -> None:\n self._history = self._history[-self.window:]\n self._history.append(population.evaluate(lazy=True).documented_best.fitness)\n if len(self._history) > self.window and abs(self._history[0] - self._history[-1]) <= self.change:\n raise StopEvolution()\n\n\nclass TimeLimit(Condition):\n \"\"\"Stop the evolution after a given amount of time.\n\n This condition stops the evolution after a given amount of time\n has elapsed. Note that the time is only checked between iterations.\n If your iterations take long, the evolution may potentially run\n for much longer than anticipated.\n\n :param seconds: The time in seconds that the evolution may run.\n \"\"\"\n\n def __init__(self, seconds: float):\n super().__init__(condition=None)\n self.time = None\n self.seconds = seconds\n\n def __call__(self, population: 'BasePopulation'):\n if self.time is None:\n self.time = monotonic()\n elif monotonic() - self.time > self.seconds:\n raise StopEvolution()\n" }, { "path": "evol/evolution.py", "content": "\"\"\"\nEvolution objects in `evol` are objects that describe how the\nevolutionary algorithm will change members of a population.\nEvolution objects contain the same methods as population objects\nbut because an evolution is separate from a population you can\nplay around with them more easily.\n\"\"\"\n\nfrom copy import copy\nfrom typing import Any, Callable, Iterator, List, Optional, Sequence\n\nfrom evol import Individual\nfrom .step import CheckpointStep, CallbackStep, EvolutionStep\nfrom .step import EvaluationStep, MapStep, FilterStep\nfrom .step import SurviveStep, BreedStep, MutateStep, RepeatStep\n\n\nclass Evolution:\n \"\"\"Describes the process a Population goes through when evolving.\"\"\"\n\n def __init__(self):\n self.chain: List[EvolutionStep] = []\n\n def __copy__(self) -> 'Evolution':\n result = Evolution()\n result.chain = copy(self.chain)\n return result\n\n def __iter__(self) -> Iterator[EvolutionStep]:\n return self.chain.__iter__()\n\n def __repr__(self) -> str:\n result = 'Evolution('\n for step in self:\n result += '\\n ' + repr(step).replace('\\n', '\\n ')\n result += ')'\n return result.strip('\\n')\n\n def evaluate(self, lazy: bool = False, name: Optional[str] = None) -> 'Evolution':\n \"\"\"Add an evaluation step to the Evolution.\n\n This evaluates the fitness of all individuals. If lazy is True, the\n fitness is only evaluated when a fitness value is not yet known. In\n most situations adding an explicit evaluation step is not needed, as\n lazy evaluation is implicitly included in the steps that need it (most\n notably in the survive step).\n\n :param lazy: If True, do no re-evaluate the fitness if the fitness is known.\n :param name: Name of the evaluation step.\n :return: This Evolution with an additional step.\n \"\"\"\n return self._add_step(EvaluationStep(name=name, lazy=lazy))\n\n def checkpoint(self,\n target: Optional[str] = None,\n method: str = 'pickle',\n name: Optional[str] = None,\n every: int = 1) -> 'Evolution':\n \"\"\"Add a checkpoint step to the Evolution.\n\n :param target: Directory to write checkpoint to. If None, the Serializer default target is taken,\n which can be provided upon initialisation. Defaults to None.\n :param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.\n Defaults to 'pickle'.\n :param name: Name of the map step.\n :param every: Checkpoint once every 'every' iterations. Defaults to 1.\n \"\"\"\n return self._add_step(CheckpointStep(name=name, target=target, method=method, every=every))\n\n def map(self, func: Callable[..., Individual], name: Optional[str] = None, **kwargs) -> 'Evolution':\n \"\"\"Add a map step to the Evolution.\n\n This applies the provided function to each individual in the\n population, in place.\n\n :param func: Function to apply to the individuals in the population.\n :param name: Name of the map step.\n :param kwargs: Arguments to pass to the function.\n :return: This Evolution with an additional step.\n \"\"\"\n return self._add_step(MapStep(name=name, func=func, **kwargs))\n\n def filter(self, func: Callable[..., bool], name: Optional[str] = None, **kwargs) -> 'Evolution':\n \"\"\"Add a filter step to the Evolution.\n\n This filters the individuals in the population using the provided function.\n\n :param func: Function to filter the individuals in the population by.\n :param name: Name of the filter step.\n :param kwargs: Arguments to pass to the function.\n :return: This Evolution with an additional step.\n \"\"\"\n return self._add_step(FilterStep(name=name, func=func, **kwargs))\n\n def survive(self,\n fraction: Optional[float] = None,\n n: Optional[int] = None,\n luck: bool = False,\n name: Optional[str] = None,\n evaluate: bool = True) -> 'Evolution':\n \"\"\"Add a survive step to the Evolution.\n\n This filters the individuals in the population according to fitness.\n\n :param fraction: Fraction of the original population that survives.\n Defaults to None.\n :param n: Number of individuals of the population that survive.\n Defaults to None.\n :param luck: If True, individuals randomly survive (with replacement!)\n with chances proportional to their fitness. Defaults to False.\n :param name: Name of the filter step.\n :param evaluate: If True, add a lazy evaluate step before the survive step.\n Defaults to True.\n :return: This Evolution with an additional step.\n \"\"\"\n if evaluate:\n after_evaluate = self.evaluate(lazy=True)\n else:\n after_evaluate = self\n return after_evaluate._add_step(SurviveStep(name=name, fraction=fraction, n=n, luck=luck))\n\n def breed(self,\n parent_picker: Callable[..., Sequence[Individual]],\n combiner: Callable,\n population_size: Optional[int] = None,\n name: Optional[str] = None,\n **kwargs) -> 'Evolution':\n \"\"\"Add a breed step to the Evolution.\n\n Create new individuals by combining existing individuals.\n\n :param parent_picker: Function that selects parents.\n :param combiner: Function that combines chromosomes into a new\n chromosome. Must be able to handle the number of chromosomes\n that the combiner returns.\n :param population_size: Intended population size after breeding.\n If None, take the previous intended population size.\n Defaults to None.\n :param name: Name of the breed step.\n :param kwargs: Kwargs to pass to the parent_picker and combiner.\n Arguments are only passed to the functions if they accept them.\n :return: self\n \"\"\"\n return self._add_step(BreedStep(name=name, parent_picker=parent_picker, combiner=combiner,\n population_size=population_size, **kwargs))\n\n def mutate(self,\n mutate_function: Callable[..., Any],\n probability: float = 1.0,\n elitist: bool = False,\n name: Optional[str] = None,\n **kwargs) -> 'Evolution':\n \"\"\"Add a mutate step to the Evolution.\n\n This mutates the chromosome of each individual.\n\n :param mutate_function: Function that accepts a chromosome and returns\n a mutated chromosome.\n :param probability: Probability that the individual mutates.\n The function is only applied in the given fraction of cases.\n Defaults to 1.0.\n :param elitist: If True, do not mutate the current best individual(s).\n Note that this only applies to evaluated individuals. Any unevaluated\n individual will be treated as normal.\n Defaults to False.\n :param name: Name of the mutate step.\n :param kwargs: Kwargs to pass to the parent_picker and combiner.\n Arguments are only passed to the functions if they accept them.\n :return: self\n \"\"\"\n return self._add_step(MutateStep(name=name, probability=probability, elitist=elitist,\n mutate_function=mutate_function, **kwargs))\n\n def repeat(self, evolution: 'Evolution', n: int = 1, name: Optional[str] = None,\n grouping_function: Optional[Callable] = None, **kwargs) -> 'Evolution':\n \"\"\"Add an evolution as a step to this evolution.\n\n This will add a step to the evolution that repeats another evolution\n several times. Optionally this step can be performed in groups.\n\n Note: if your population uses multiple concurrent workers and you use grouping,\n any callbacks inside the evolution you apply here may not have the desired effect.\n\n :param evolution: Evolution to apply.\n :param n: Number of times to perform the evolution. Defaults to 1.\n :param name: Name of the repeat step.\n :param grouping_function: Optional function to use for grouping the population.\n You can find built-in grouping functions in evol.helpers.groups.\n :param kwargs: Kwargs to pass to the grouping function, for example n_groups.\n :return: self\n \"\"\"\n return self._add_step(RepeatStep(name=name, evolution=evolution, n=n,\n grouping_function=grouping_function, **kwargs))\n\n def callback(self, callback_function: Callable[..., Any],\n every: int = 1, name: Optional[str] = None, **kwargs) -> 'Evolution':\n \"\"\"Call a function as a step in this evolution.\n\n This will call the provided function with the population as argument.\n\n Note that you can raise evol.exceptions.StopEvolution from within the\n callback to stop further evolution.\n\n :param callback_function: Function to call.\n :param every: Only call the function once per `every` iterations.\n Defaults to 1; every iteration.\n :param name: Name of the callback step.\n :return: self\n \"\"\"\n return self._add_step(CallbackStep(name=name, every=every, callback_function=callback_function, **kwargs))\n\n def _add_step(self, step: EvolutionStep) -> 'Evolution':\n result = copy(self)\n result.chain.append(step)\n return result\n" }, { "path": "evol/exceptions.py", "content": "class PopulationIsNotEvaluatedException(RuntimeError):\n pass\n\n\nclass StopEvolution(Exception):\n pass\n" }, { "path": "evol/helpers/__init__.py", "content": "\"\"\"\nHelpers in `evol` are functions that help you when you are \ndesigning algorithms. We archive these helping functions per usecase.\n\n\"\"\"" }, { "path": "evol/helpers/combiners/__init__.py", "content": "" }, { "path": "evol/helpers/combiners/permutation.py", "content": "from itertools import islice, tee\nfrom random import choice\nfrom typing import Any, Tuple\n\nfrom .utils import select_node, construct_neighbors, identify_cycles, cycle_parity\nfrom ..utils import select_partition\n\n\ndef order_one_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple:\n \"\"\"Combine two chromosomes using order-1 crossover.\n\n http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/Order1CrossoverOperator.aspx\n\n :param parent_1: First parent.\n :param parent_2: Second parent.\n :return: Child chromosome.\n \"\"\"\n start, end = select_partition(len(parent_1))\n selected_partition = parent_1[start:end + 1]\n remaining_elements = filter(lambda element: element not in selected_partition, parent_2)\n return tuple(islice(remaining_elements, 0, start)) + selected_partition + tuple(remaining_elements)\n\n\ndef edge_recombination(*parents: Tuple) -> Tuple:\n \"\"\"Combine multiple chromosomes using edge recombination.\n\n http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/EdgeRecombinationCrossoverOperator.aspx\n\n :param parents: Chromosomes to combine.\n :return: Child chromosome.\n \"\"\"\n return tuple(select_node(\n start_node=choice([chromosome[0] for chromosome in parents]),\n neighbors=construct_neighbors(*parents)\n ))\n\n\ndef cycle_crossover(parent_1: Tuple, parent_2: Tuple) -> Tuple[Tuple[Any, ...], ...]:\n \"\"\"Combine two chromosomes using cycle crossover.\n\n http://www.rubicite.com/Tutorials/GeneticAlgorithms/CrossoverOperators/CycleCrossoverOperator.aspx\n\n :param parent_1: First parent.\n :param parent_2: Second parent.\n :return: Child chromosome.\n \"\"\"\n cycles = identify_cycles(parent_1, parent_2)\n parity = cycle_parity(cycles=cycles)\n it_a, it_b = tee((b, a) if parity[i] else (a, b) for i, (a, b) in enumerate(zip(parent_1, parent_2)))\n yield tuple(x[0] for x in it_a)\n yield tuple(y[1] for y in it_b)\n" }, { "path": "evol/helpers/combiners/utils.py", "content": "from collections import defaultdict\nfrom itertools import tee, islice, cycle\n\nfrom random import choice\nfrom typing import Iterable, Generator, Any, Set, List, Dict, Tuple\n\n\ndef construct_neighbors(*chromosome: Tuple[Any]) -> defaultdict:\n result = defaultdict(set)\n for element in chromosome:\n for x, y in _neighbors_in(element):\n result[x].add(y)\n result[y].add(x)\n return result\n\n\ndef _neighbors_in(x: Tuple[Any], cyclic=True) -> Iterable[Tuple[Any, Any]]:\n a, b = tee(islice(cycle(x), 0, len(x) + (1 if cyclic else 0)))\n next(b, None)\n return zip(a, b)\n\n\ndef _remove_from_neighbors(neighbors, node):\n del neighbors[node]\n for _, element in neighbors.items():\n element.difference_update({node})\n\n\ndef select_node(start_node: Any, neighbors: defaultdict) -> Generator[Any, None, None]:\n node = start_node\n yield node\n while len(neighbors) > 1:\n options = neighbors[node]\n _remove_from_neighbors(neighbors, node)\n if len(options) > 0:\n min_len = min([len(neighbors[option]) for option in options])\n node = choice([option for option in options if len(neighbors[option]) == min_len])\n else:\n node = choice(list(neighbors.keys()))\n yield node\n\n\ndef identify_cycles(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any]) -> List[Set[int]]:\n \"\"\"Identify all cycles between the chromosomes.\n\n A cycle is found by following this procedure: given an index, look up the\n value in the first chromosome. Then find the index of that value in the\n second chromosome. Repeat, until one returns to the original index.\n\n :param chromosome_1: First chromosome.\n :param chromosome_2: Second chromosome.\n :return: A list of cycles.\n \"\"\"\n indices = set(range(len(chromosome_1)))\n cycles = []\n while len(indices) > 0:\n next_cycle = _identify_cycle(chromosome_1=chromosome_1, chromosome_2=chromosome_2, start_index=min(indices))\n indices.difference_update(next_cycle)\n cycles.append(next_cycle)\n return cycles\n\n\ndef _identify_cycle(chromosome_1: Tuple[Any], chromosome_2: Tuple[Any], start_index: int = 0) -> Set[int]:\n \"\"\"Identify a cycle between the chromosomes starting at the provided index.\n\n A cycle is found by following this procedure: given an index, look up the\n value in the first chromosome. Then find the index of that value in the\n second chromosome. Repeat, until one returns to the original index.\n\n :param chromosome_1: First chromosome.\n :param chromosome_2: Second chromosome.\n :param start_index: Index to start. Defaults to 0.\n :return: The set of indices in the identified cycle.\n \"\"\"\n indices = set()\n index = start_index\n while index not in indices:\n indices.add(index)\n value = chromosome_1[index]\n index = chromosome_2.index(value)\n return indices\n\n\ndef cycle_parity(cycles: List[Set[int]]) -> Dict[int, bool]:\n \"\"\"Create a dictionary with the cycle parity of each index.\n\n Indices in all odd cycles have parity False, while\n indices in even cycles have parity True.\"\"\"\n return {index: bool(i % 2) for i, c in enumerate(cycles) for index in c}\n" }, { "path": "evol/helpers/groups.py", "content": "from random import shuffle\nfrom typing import List\n\nfrom evol import Individual\nfrom evol.exceptions import PopulationIsNotEvaluatedException\n\n\"\"\"\nBelow are functions that allocate individuals to the\nisland populations. It will be passed a list of individuals plus\nthe kwargs passed to this method, and must return a list of lists\nof integers, each sub-list representing an island and the integers\nrepresenting the index of an individual in the list. Each island\nmust contain at least one individual, and individual may be copied\nto multiple islands.\n\"\"\"\n\n\ndef group_duplicate(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:\n \"\"\"\n Group individuals into groups that each contain all individuals.\n\n :param individuals: List of individuals to group.\n :param n_groups: Number of groups to make.\n :return: List of lists of ints\n \"\"\"\n return [list(range(len(individuals))) for _ in range(n_groups)]\n\n\ndef group_random(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:\n \"\"\"\n Group individuals randomly into groups of roughly equal size.\n\n :param individuals: List of individuals to group.\n :param n_groups: Number of groups to make.\n :return: List of lists of ints\n \"\"\"\n indexes = list(range(len(individuals)))\n shuffle(indexes)\n return [indexes[i::n_groups] for i in range(n_groups)]\n\n\ndef group_stratified(individuals: List[Individual], n_groups: int = 4) -> List[List[int]]:\n \"\"\"\n Group individuals into groups of roughly equal size in a stratified manner.\n\n This function groups such that each group contains individuals of\n higher as well as lower fitness. This requires the individuals to have a fitness.\n\n :param individuals: List of individuals to group.\n :param n_groups: Number of groups to make.\n :return: List of lists of ints\n \"\"\"\n _ensure_evaluated(individuals)\n indexes = list(map(\n lambda index_and_individual: index_and_individual[0],\n sorted(enumerate(individuals), key=lambda index_and_individual: index_and_individual[1].fitness)\n ))\n return [indexes[i::n_groups] for i in range(n_groups)]\n\n\ndef _ensure_evaluated(individuals: List[Individual]):\n \"\"\"\n Helper function to ensure individuals are evaluated.\n\n :param individuals: List of individuals\n :raises RuntimeError: When at least one of the individuals is not evaluated.\n \"\"\"\n for individual in individuals:\n if individual.fitness is None:\n raise PopulationIsNotEvaluatedException('Population must be evaluated.')\n" }, { "path": "evol/helpers/mutators/__init__.py", "content": "" }, { "path": "evol/helpers/mutators/permutation.py", "content": "from random import sample\nfrom typing import Any, Tuple\n\nfrom ..utils import select_partition\n\n\ndef inversion(chromosome: Tuple[Any, ...], min_size: int = 2, max_size: int = None) -> Tuple[Any, ...]:\n \"\"\"Mutate a chromosome using inversion.\n\n Inverts a random partition of the chromosome.\n\n :param chromosome: Original chromosome.\n :param min_size: Minimum partition size. Defaults to 2.\n :param max_size: Maximum partition size. Defaults to length - 1.\n :return: Mutated chromosome.\n \"\"\"\n start, end = select_partition(len(chromosome), min_size, max_size)\n return chromosome[:start] + tuple(reversed(chromosome[start:end])) + chromosome[end:]\n\n\ndef swap_elements(chromosome: Tuple[Any, ...]) -> Tuple[Any, ...]:\n \"\"\"Randomly swap two elements of the chromosome.\n\n :param chromosome: Original chromosome.\n :return: Mutated chromosome.\n \"\"\"\n result = list(chromosome)\n index_1, index_2 = sample(range(len(chromosome)), 2)\n result[index_1], result[index_2] = result[index_2], result[index_1]\n return tuple(result)\n" }, { "path": "evol/helpers/pickers.py", "content": "from typing import Sequence, Tuple\n\nfrom random import choice\n\nfrom evol import Individual\n\n\ndef pick_random(parents: Sequence[Individual], n_parents: int = 2) -> Tuple:\n \"\"\"Randomly selects parents with replacement\n\n Accepted arguments:\n n_parents: Number of parents to select. Defaults to 2.\n \"\"\"\n return tuple(choice(parents) for _ in range(n_parents))\n" }, { "path": "evol/helpers/utils.py", "content": "from random import randint\nfrom typing import Tuple\n\n\ndef select_partition(length: int, min_size: int = 1, max_size: int = None) -> Tuple[int, int]:\n \"\"\"Select a partition of a chromosome.\n\n :param length: Length of the chromosome.\n :param min_size: Minimum length of the partition. Defaults to 1.\n :param max_size: Maximum length of the partition. Defaults to length - 1.\n :return: Start and end index of the partition.\n \"\"\"\n partition_size = randint(min_size, length - 1 if max_size is None else max_size)\n partition_start = randint(0, length - partition_size)\n return partition_start, partition_start + partition_size\n\n\ndef rotating_window(arr):\n \"\"\"rotating_window([1,2,3,4]) -> [(4,1), (1,2), (2,3), (3,4)]\"\"\"\n for i, city in enumerate(arr):\n yield arr[i - 1], arr[i]\n\n\ndef sliding_window(arr):\n \"\"\"sliding_window([1,2,3,4]) -> [(1,2), (2,3), (3,4)]\"\"\"\n for i, city in enumerate(arr[:-1]):\n yield arr[i], arr[i + 1]\n" }, { "path": "evol/individual.py", "content": "\"\"\"\nIndividual objects in `evol` are a wrapper around a chromosome.\nInternally we work with individuals because that allows us to\nseparate the fitness calculation from the data structure. This\nsaves a lot of CPU power.\n\"\"\"\n\nfrom random import random\nfrom typing import Any, Callable, Optional\nfrom uuid import uuid4\n\n\nclass Individual:\n \"\"\"Represents an individual in a population. The individual has a chromosome.\n\n :param chromosome: The chromosome of the individual.\n :param fitness: The fitness of the individual, or None.\n Defaults to None.\n \"\"\"\n\n def __init__(self, chromosome: Any, fitness: Optional[float] = None):\n self.age = 0\n self.chromosome = chromosome\n self.fitness = fitness\n self.id = f\"{str(uuid4())[:6]}\"\n\n def __repr__(self):\n return f\"\"\n\n @classmethod\n def from_dict(cls, data: dict) -> 'Individual':\n \"\"\"Load an Individual from a dictionary.\n\n :param data: Dictionary containing the keys 'age', 'chromosome', 'fitness' and 'id'.\n :return: Individual\n \"\"\"\n result = cls(chromosome=data['chromosome'], fitness=data['fitness'])\n result.age = data['age']\n result.id = data['id']\n return result\n\n def __post_evaluate(self, result):\n self.fitness = result\n\n def evaluate(self, eval_function: Callable[..., float], lazy: bool = False):\n \"\"\"Evaluate the fitness of the individual.\n\n :param eval_function: Function that reduces a chromosome to a fitness.\n :param lazy: If True, do no re-evaluate the fitness if the fitness is known.\n \"\"\"\n if self.fitness is None or not lazy:\n self.fitness = eval_function(self.chromosome)\n\n def mutate(self, mutate_function: Callable[..., Any], probability: float = 1.0, **kwargs):\n \"\"\"Mutate the chromosome of the individual.\n\n :param mutate_function: Function that accepts a chromosome and returns a mutated chromosome.\n :param probability: Probability that the individual mutates.\n The function is only applied in the given fraction of cases.\n Defaults to 1.0.\n :param kwargs: Arguments to pass to the mutation function.\n \"\"\"\n if probability == 1.0 or random() < probability:\n self.chromosome = mutate_function(self.chromosome, **kwargs)\n self.fitness = None\n" }, { "path": "evol/logger.py", "content": "\"\"\"\nLoggers help keep track of the workings of your evolutionary algorithm. By\ndefault, each Population is initialized with a BaseLogger, which you can use\nby using the .log() method of the population. If you want more complex\nbehaviour, you can supply another logger to the Population on initialisation.\n\"\"\"\nimport datetime as dt\nimport os\nimport json\nimport logging\nimport sys\nimport uuid\n\nfrom evol.exceptions import PopulationIsNotEvaluatedException\nfrom evol.population import BasePopulation\n\n\nclass BaseLogger:\n \"\"\"\n The `evol.BaseLogger` is the most basic logger in evol.\n You can supply it to a population so that the population\n knows how to handle the `.log()` verb.\n \"\"\"\n\n def __init__(self, target=None, stdout=False, fmt='%(asctime)s,%(message)s'):\n self.file = target\n if target is not None:\n if not os.path.exists(os.path.split(target)[0]):\n raise RuntimeError(f\"path to target {os.path.split(target)[0]} does not exist!\")\n formatter = logging.Formatter(fmt=fmt, datefmt='%Y-%m-%d %H:%M:%S')\n self.logger = logging.getLogger(name=f\"{uuid.uuid4()}\")\n if not self.logger.handlers:\n # we do this extra step because loggers can behave in strange ways otherwise\n # https://navaspot.wordpress.com/2015/09/22/same-log-messages-multiple-times-in-python-issue/\n if target:\n file_handler = logging.FileHandler(filename=target)\n file_handler.setFormatter(fmt=formatter)\n self.logger.addHandler(file_handler)\n if stdout:\n stream_handler = logging.StreamHandler(stream=sys.stdout)\n stream_handler.setFormatter(fmt=formatter)\n self.logger.addHandler(stream_handler)\n self.logger.setLevel(level=logging.INFO)\n\n @staticmethod\n def check_population(population: BasePopulation) -> None:\n if not population.is_evaluated:\n raise PopulationIsNotEvaluatedException('Population must be evaluated when logging.')\n\n def log(self, population, **kwargs):\n \"\"\"\n The logger method of the Logger object determines what will be logged.\n :param population: `evol.Population` object\n :return: nothing, it merely logs to a file and perhaps stdout\n \"\"\"\n self.check_population(population)\n values = ','.join([str(item) for item in kwargs.values()])\n if values != '':\n values = f',{values}'\n for i in population:\n self.logger.info(f'{population.id},{i.id},{i.fitness}' + values)\n\n\nclass SummaryLogger(BaseLogger):\n \"\"\"\n The `evol.SummaryLogger` merely logs statistics per population and nothing else.\n You are still able to log to stdout as well.\n \"\"\"\n\n def log(self, population, **kwargs):\n self.check_population(population)\n values = ','.join([str(item) for item in kwargs.values()])\n if values != '':\n values = f',{values}'\n fitnesses = [i.fitness for i in population]\n self.logger.info(f'{min(fitnesses)},{sum(fitnesses) / len(fitnesses)},{max(fitnesses)}' + values)\n\n\nclass MultiLogger:\n \"\"\"\n The `evol.Multilogger` is a logger object that can handle writing to two files.\n It is here for demonstration purposes to show how you could customize the logging.\n The only thing that matters is that all logging is handled by the `.log()`\n call. So we are free to record to multiple files if we want as well. This is\n not per se best practice but it would work.\n \"\"\"\n\n def __init__(self, file_individuals, file_population):\n self.file_individuals = file_individuals\n self.file_population = file_population\n\n def log(self, population, **kwargs):\n \"\"\"\n The logger method of the Logger object determines what will be logged.\n :param population: population to log\n :return: generator of strings to be handled\n \"\"\"\n ind_generator = (f'{dt.datetime.now()},{population.id},{i.id},{i.fitness}' for i in population)\n fitnesses = [i.fitness for i in population]\n data = {\n 'ts': str(dt.datetime.now()),\n 'mean_ind': sum(fitnesses) / len(fitnesses),\n 'min_ind': min(fitnesses),\n 'max_ind': max(fitnesses)\n }\n dict_to_log = {**kwargs, **data}\n self.handle(ind_generator, dict_to_log)\n\n def handle(self, ind_generator, dict_to_log):\n \"\"\"\n The handler method of the Logger object determines how it will be logged.\n In this case we print if there is no file and we append to a file otherwise.\n \"\"\"\n with open(self.file_population, 'a') as f:\n f.write(json.dumps(dict_to_log))\n with open(self.file_population, 'a') as f:\n f.writelines(ind_generator)\n" }, { "path": "evol/population.py", "content": "\"\"\"\nPopulation objects in `evol` are a collection of chromosomes\nat some point in an evolutionary algorithm. You can apply\nevolutionary steps by directly calling methods on the population\nor by applying an `evol.Evolution` object.\n\"\"\"\nfrom abc import ABCMeta, abstractmethod\nfrom copy import copy\nfrom itertools import cycle, islice\nfrom math import ceil\nfrom random import choices, randint\nfrom typing import Any, Callable, Generator, Iterable, Iterator, List, Optional, Sequence, TYPE_CHECKING\nfrom uuid import uuid4\n\nfrom multiprocess.pool import Pool\n\nfrom evol import Individual\nfrom evol.conditions import Condition\nfrom evol.exceptions import StopEvolution\nfrom evol.helpers.groups import group_random\nfrom evol.utils import offspring_generator, select_arguments\nfrom evol.serialization import SimpleSerializer\n\nif TYPE_CHECKING:\n from .evolution import Evolution\n\n\nclass BasePopulation(metaclass=ABCMeta):\n\n def __init__(self,\n chromosomes: Iterable[Any],\n eval_function: Callable,\n checkpoint_target: Optional[str] = None,\n concurrent_workers: Optional[int] = 1,\n maximize: bool = True,\n generation: int = 0,\n intended_size: Optional[int] = None,\n serializer=None):\n self.concurrent_workers = concurrent_workers\n self.documented_best = None\n self.eval_function = eval_function\n self.generation = generation\n self.id = str(uuid4())[:6]\n self.individuals = [Individual(chromosome=chromosome) for chromosome in chromosomes]\n self.intended_size = intended_size or len(self.individuals)\n self.maximize = maximize\n self.serializer = serializer or SimpleSerializer(target=checkpoint_target)\n self.pool = None if concurrent_workers == 1 else Pool(concurrent_workers)\n\n def __iter__(self) -> Iterator[Individual]:\n return self.individuals.__iter__()\n\n def __getitem__(self, i) -> Individual:\n return self.individuals[i]\n\n def __len__(self):\n return len(self.individuals)\n\n def __repr__(self):\n return f\"\"\n\n @property\n def current_best(self) -> Individual:\n evaluated_individuals = tuple(filter(lambda x: x.fitness is not None, self.individuals))\n if len(evaluated_individuals) > 0:\n return max(evaluated_individuals, key=lambda x: x.fitness if self.maximize else -x.fitness)\n\n @property\n def current_worst(self) -> Individual:\n evaluated_individuals = tuple(filter(lambda x: x.fitness is not None, self.individuals))\n if len(evaluated_individuals) > 0:\n return min(evaluated_individuals, key=lambda x: x.fitness if self.maximize else -x.fitness)\n\n @property\n def chromosomes(self) -> Generator[Any, None, None]:\n for individual in self.individuals:\n yield individual.chromosome\n\n @property\n def is_evaluated(self) -> bool:\n return all(individual.fitness is not None for individual in self)\n\n @classmethod\n def generate(cls,\n init_function: Callable[[], Any],\n eval_function: Callable[..., float],\n size: int = 100,\n **kwargs) -> 'BasePopulation':\n \"\"\"Generate a population from an initialisation function.\n\n :param init_function: Function that returns a chromosome.\n :param eval_function: Function that reduces a chromosome to a fitness.\n :param size: Number of individuals to generate. Defaults to 100.\n :return: BasePopulation\n \"\"\"\n chromosomes = [init_function() for _ in range(size)]\n return cls(chromosomes=chromosomes, eval_function=eval_function, **kwargs)\n\n @classmethod\n def load(cls,\n target: str,\n eval_function: Callable[..., float],\n **kwargs) -> 'BasePopulation':\n \"\"\"Load a population from a checkpoint.\n\n :param target: Path to checkpoint directory or file.\n :param eval_function: Function that reduces a chromosome to a fitness.\n :param kwargs: Any argument the init method accepts.\n :return: Population\n \"\"\"\n result = cls(chromosomes=[], eval_function=eval_function, **kwargs)\n result.individuals = result.serializer.load(target=target)\n return result\n\n def checkpoint(self, target: Optional[str] = None, method: str = 'pickle') -> 'BasePopulation':\n \"\"\"Checkpoint the population.\n\n :param target: Directory to write checkpoint to. If None, the Serializer default target is taken,\n which can be provided upon initialisation. Defaults to None.\n :param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.\n Defaults to 'pickle'.\n :return: Population\n \"\"\"\n self.serializer.checkpoint(individuals=self.individuals, target=target, method=method)\n return self\n\n @property\n def _individual_weights(self):\n try:\n min_fitness = min(individual.fitness for individual in self)\n max_fitness = max(individual.fitness for individual in self)\n except TypeError:\n raise RuntimeError('Individual weights can not be computed if the individuals are not evaluated.')\n if min_fitness == max_fitness:\n return [1] * len(self)\n elif self.maximize:\n return [(individual.fitness - min_fitness) / (max_fitness - min_fitness) for individual in self]\n else:\n return [1 - (individual.fitness - min_fitness) / (max_fitness - min_fitness) for individual in self]\n\n def evolve(self, evolution: 'Evolution', n: int = 1) -> 'BasePopulation': # noqa: F821\n \"\"\"Evolve the population according to an Evolution.\n\n :param evolution: Evolution to follow\n :param n: Times to apply the evolution. Defaults to 1.\n :return: Population\n \"\"\"\n result = copy(self)\n try:\n for _ in range(n):\n Condition.check(result)\n for step in evolution:\n result = step.apply(result)\n except StopEvolution:\n pass\n return result\n\n @abstractmethod\n def evaluate(self, lazy: bool = False) -> 'BasePopulation':\n pass\n\n def breed(self,\n parent_picker: Callable[..., Sequence[Individual]],\n combiner: Callable,\n population_size: Optional[int] = None,\n **kwargs) -> 'BasePopulation':\n \"\"\"Create new individuals by combining existing individuals.\n\n This increments the generation of the Population.\n\n :param parent_picker: Function that selects parents from a collection of individuals.\n :param combiner: Function that combines chromosomes into a new\n chromosome. Must be able to handle the number of chromosomes\n that the combiner returns.\n :param population_size: Intended population size after breeding.\n If None, take the previous intended population size.\n Defaults to None.\n :param kwargs: Kwargs to pass to the parent_picker and combiner.\n Arguments are only passed to the functions if they accept them.\n :return: self\n \"\"\"\n if population_size:\n self.intended_size = population_size\n offspring = offspring_generator(parents=self.individuals,\n parent_picker=select_arguments(parent_picker),\n combiner=select_arguments(combiner),\n **kwargs)\n self.individuals += list(islice(offspring, self.intended_size - len(self.individuals)))\n self.generation += 1\n return self\n\n def mutate(self,\n mutate_function: Callable[..., Any],\n probability: float = 1.0,\n elitist: bool = False, **kwargs) -> 'BasePopulation':\n \"\"\"Mutate the chromosome of each individual.\n\n :param mutate_function: Function that accepts a chromosome and returns\n a mutated chromosome.\n :param probability: Probability that the individual mutates.\n The function is only applied in the given fraction of cases.\n Defaults to 1.0.\n :param elitist: If True, do not mutate the current best individual(s).\n Note that this only applies to evaluated individuals. Any unevaluated\n individual will be treated as normal.\n Defaults to False.\n :param kwargs: Arguments to pass to the mutation function.\n :return: self\n \"\"\"\n elite_fitness: Optional[float] = self.current_best.fitness if elitist else None\n for individual in self.individuals:\n if elite_fitness is None or individual.fitness != elite_fitness:\n individual.mutate(mutate_function, probability=probability, **kwargs)\n return self\n\n def map(self, func: Callable[..., Individual], **kwargs) -> 'BasePopulation':\n \"\"\"Apply the provided function to each individual in the population.\n\n :param func: A function to apply to each individual in the population,\n which when called returns a modified individual.\n :param kwargs: Arguments to pass to the function.\n :return: self\n \"\"\"\n self.individuals = [func(individual, **kwargs) for individual in self.individuals]\n return self\n\n def filter(self, func: Callable[..., bool], **kwargs) -> 'BasePopulation':\n \"\"\"Add a filter step to the Evolution.\n\n Filters the individuals in the population using the provided function.\n\n :param func: Function to filter the individuals in the population by,\n which returns a boolean when called on an individual.\n :param kwargs: Arguments to pass to the function.\n :return: self\n \"\"\"\n self.individuals = [individual for individual in self.individuals if func(individual, **kwargs)]\n return self\n\n def survive(self, fraction: Optional[float] = None,\n n: Optional[int] = None, luck: bool = False) -> 'BasePopulation':\n \"\"\"Let part of the population survive.\n\n Remove part of the population. If both fraction and n are specified,\n the minimum resulting population size is taken.\n\n :param fraction: Fraction of the original population that survives.\n Defaults to None.\n :param n: Number of individuals of the population that survive.\n Defaults to None.\n :param luck: If True, individuals randomly survive (with replacement!)\n with chances proportional to their fitness. Defaults to False.\n :return: self\n \"\"\"\n if fraction is None:\n if n is None:\n raise ValueError('everyone survives! must provide either \"fraction\" and/or \"n\".')\n resulting_size = n\n elif n is None:\n resulting_size = round(fraction * len(self.individuals))\n else:\n resulting_size = min(round(fraction * len(self.individuals)), n)\n self.evaluate(lazy=True)\n if resulting_size == 0:\n raise RuntimeError(f'No individual out of {len(self.individuals)} survived!')\n if resulting_size > len(self.individuals):\n raise ValueError(f'everyone survives in population {self.id}: '\n f'{resulting_size} out of {len(self.individuals)} must survive.')\n if luck:\n self.individuals = choices(self.individuals, k=resulting_size, weights=self._individual_weights)\n else:\n sorted_individuals = sorted(self.individuals, key=lambda x: x.fitness, reverse=self.maximize)\n self.individuals = sorted_individuals[:resulting_size]\n return self\n\n def callback(self, callback_function: Callable[..., None],\n **kwargs) -> 'BasePopulation':\n \"\"\"\n Performs a callback function on the population. Can be used for\n custom logging/checkpointing.\n :param callback_function: Function that accepts the population\n as a first argument.\n :return:\n \"\"\"\n self.evaluate(lazy=True)\n callback_function(self, **kwargs)\n return self\n\n def group(self, grouping_function: Callable[..., List[List[int]]] = group_random,\n **kwargs) -> List['BasePopulation']:\n \"\"\"\n Group a population into islands.\n\n Divides the population into multiple island populations, each of which\n contains a subset of the original population. An individual from the\n original population may end up in multiple (>= 0) island populations.\n\n :param grouping_function: Function that allocates individuals to the\n island populations. It will be passed a list of individuals plus\n the kwargs passed to this method, and must return a list of lists\n of integers, each sub-list representing an island and the integers\n representing the index of an individual in the list. Each island\n must contain at least one individual, and individual may be copied\n to multiple islands.\n :param kwargs: Additional keyworded arguments are passed to the\n grouping function.\n :return: List[Population]\n \"\"\"\n group_indexes = grouping_function(self.individuals, **kwargs)\n if len(group_indexes) == 0:\n raise ValueError('Group yielded zero islands.')\n result = [self._subset(index=index, subset_id=str(i)) for i, index in enumerate(group_indexes)]\n return result\n\n @classmethod\n def combine(cls, *populations: 'BasePopulation',\n intended_size: Optional[int] = None,\n pool: Optional[Pool] = None) -> 'BasePopulation':\n \"\"\"\n Combine multiple island populations into a single population.\n\n The resulting population is reduced to its intended size.\n\n :param populations: Populations to combine.\n :param intended_size: Intended size of the resulting population.\n Defaults to the sum of the intended sizes of the islands.\n :param pool: Optionally provide a multiprocessing pool to be\n used by the population.\n :return: Population\n \"\"\"\n if len(populations) == 0:\n raise ValueError('Cannot combine zero islands into one.')\n result = copy(populations[0])\n for pop in populations[1:]:\n result.individuals += pop.individuals\n result.intended_size = intended_size or sum([pop.intended_size for pop in populations])\n result.pool = pool\n result.id = result.id.split('-')[0]\n return result.survive(n=result.intended_size)\n\n def _subset(self, index: List[int], subset_id: str) -> 'BasePopulation':\n \"\"\"Create a new population that is a subset of this population.\"\"\"\n if len(index) == 0:\n raise ValueError('Grouping yielded an empty island.')\n result = copy(self)\n result.individuals = [result.individuals[i] for i in index]\n result.intended_size = len(result.individuals)\n result.pool = None # Subsets shouldn't parallelize anything\n result.id += '-' + subset_id\n return result\n\n def _update_documented_best(self):\n \"\"\"Update the documented best\"\"\"\n current_best = self.current_best\n if (self.documented_best is None or\n (self.maximize and current_best.fitness > self.documented_best.fitness) or\n (not self.maximize and current_best.fitness < self.documented_best.fitness)):\n self.documented_best = copy(current_best)\n\n\nclass Population(BasePopulation):\n \"\"\"Population of Individuals\n\n :param chromosomes: Iterable of initial chromosomes of the Population.\n :param eval_function: Function that reduces a chromosome to a fitness.\n :param maximize: If True, fitness will be maximized, otherwise minimized.\n Defaults to True.\n :param generation: Generation of the Population. This is incremented after\n each breed call. Defaults to 0.\n :param intended_size: Intended size of the Population. The population will\n be replenished to this size by .breed(). Defaults to the number of\n chromosomes provided.\n :param checkpoint_target: Target for the serializer of the Population. If\n a serializer is provided, this target is ignored. Defaults to None.\n :param serializer: Serializer for the Population. If None, a new\n SimpleSerializer is created. Defaults to None.\n :param concurrent_workers: If > 1, evaluate individuals in {concurrent_workers}\n separate processes. If None, concurrent_workers is set to n_cpus. Defaults to 1.\n \"\"\"\n\n def __init__(self,\n chromosomes: Iterable,\n eval_function: Callable[..., float],\n maximize: bool = True,\n generation: int = 0,\n intended_size: Optional[int] = None,\n checkpoint_target: Optional[str] = None,\n serializer=None,\n concurrent_workers: Optional[int] = 1):\n super().__init__(chromosomes=chromosomes,\n eval_function=eval_function,\n checkpoint_target=checkpoint_target,\n concurrent_workers=concurrent_workers,\n maximize=maximize,\n generation=generation,\n intended_size=intended_size,\n serializer=serializer)\n\n def __copy__(self):\n result = self.__class__(chromosomes=[],\n eval_function=self.eval_function,\n maximize=self.maximize,\n serializer=self.serializer,\n intended_size=self.intended_size,\n generation=self.generation,\n concurrent_workers=1) # Prevent new pool from being made\n result.individuals = [copy(individual) for individual in self.individuals]\n result.concurrent_workers = self.concurrent_workers\n result.pool = self.pool\n result.documented_best = self.documented_best\n result.id = self.id\n return result\n\n def evaluate(self, lazy: bool = False) -> 'Population':\n \"\"\"Evaluate the individuals in the population.\n\n This evaluates the fitness of all individuals. If lazy is True, the\n fitness is only evaluated when a fitness value is not yet known. In\n most situations adding an explicit evaluation step is not needed, as\n lazy evaluation is implicitly included in the operations that need it\n (most notably in the survive operation).\n\n :param lazy: If True, do no re-evaluate the fitness if the fitness is known.\n :return: self\n \"\"\"\n if self.pool:\n f = self.eval_function # We cannot refer to self in the map\n scores = self.pool.map(lambda i: i.fitness if (i.fitness and lazy) else f(i.chromosome), self.individuals)\n for individual, fitness in zip(self.individuals, scores):\n individual.fitness = fitness\n else:\n for individual in self.individuals:\n individual.evaluate(eval_function=self.eval_function, lazy=lazy)\n self._update_documented_best()\n return self\n\n\nclass Contest:\n \"\"\"A single contest among a group of competitors.\n\n This is encapsulated in an object so that scores for many sets of\n competitors can be evaluated concurrently without resorting to a\n dict or some similar madness to correlate score vectors with an\n ever-widening matrix of contests and competitors.\n\n :param competitors: Iterable of Individuals in this Contest.\n \"\"\"\n\n def __init__(self, competitors: Iterable[Individual]):\n self.competitors = list(competitors)\n\n def assign_scores(self, scores: Sequence[float]) -> None:\n for competitor, score in zip(self.competitors, scores):\n competitor.fitness += score\n\n @property\n def competitor_chromosomes(self):\n return [competitor.chromosome for competitor in self.competitors]\n\n @classmethod\n def generate(cls, individuals: Sequence[Individual],\n individuals_per_contest: int, contests_per_round: int) -> List['Contest']:\n \"\"\"Generate contests for a round of evaluations.\n\n :param individuals: A sequence of competing Individuals.\n :param individuals_per_contest: Number of Individuals participating in each Contest.\n :param contests_per_round: Minimum number of contests each individual\n takes part in for each evaluation round. The actual number of contests\n per round is a multiple of individuals_per_contest.\n :return: List of Contests\n \"\"\"\n contests = []\n n_rounds = ceil(contests_per_round / individuals_per_contest)\n for _ in range(n_rounds):\n offsets = [0] + [randint(0, len(individuals) - 1) for _ in range(individuals_per_contest - 1)]\n generators = [islice(cycle(individuals), offset, None) for offset in offsets]\n for competitors in islice(zip(*generators), len(individuals)):\n contests.append(Contest(competitors))\n return contests\n\n\nclass ContestPopulation(BasePopulation):\n \"\"\"Population which is evaluated through contests.\n\n This variant of the Population is used when individuals cannot be\n evaluated on a one-by-one basis, but instead can only be compared to\n each other. This is typically the case for AI that performs some task\n (i.e. plays a game), but can be useful in many other cases.\n\n For each round of evaluation, each individual participates in a given\n number of contests, in which a given number of individuals take part.\n The resulting scores of these contests are summed to form the fitness.\n\n Since the fitness of an individual is dependent on the other individuals\n in the population, the fitness of all individuals is recalculated when\n new individuals are present, and the fitness of all individuals is reset\n when the population is modified (e.g. by calling survive, mutate etc).\n\n :param chromosomes: Iterable of initial chromosomes of the Population.\n :param eval_function: Function that reduces a chromosome to a fitness.\n :param maximize: If True, fitness will be maximized, otherwise minimized.\n Defaults to True.\n :param individuals_per_contest: Number of individuals that take part in\n each contest. The size of the population must be divisible by this\n number. Defaults to 2.\n :param contests_per_round: Minimum number of contests each individual\n takes part in for each evaluation round. The actual number of contests\n per round is a multiple of individuals_per_contest. Defaults to 10.\n :param generation: Generation of the Population. This is incremented after\n echo survive call. Defaults to 0.\n :param intended_size: Intended size of the Population. The population will\n be replenished to this size by .breed(). Defaults to the number of\n chromosomes provided.\n :param checkpoint_target: Target for the serializer of the Population. If\n a serializer is provided, this target is ignored. Defaults to None.\n :param serializer: Serializer for the Population. If None, a new\n SimpleSerializer is created. Defaults to None.\n :param concurrent_workers: If > 1, evaluate individuals in {concurrent_workers}\n separate processes. If None, concurrent_workers is set to n_cpus. Defaults to 1.\n \"\"\"\n eval_function: Callable[..., Sequence[float]] # This population expects a different eval signature\n\n def __init__(self,\n chromosomes: Iterable,\n eval_function: Callable[..., Sequence[float]],\n maximize: bool = True,\n individuals_per_contest=2,\n contests_per_round=10,\n generation: int = 0,\n intended_size: Optional[int] = None,\n checkpoint_target: Optional[int] = None,\n serializer=None,\n concurrent_workers: Optional[int] = 1):\n super().__init__(chromosomes=chromosomes,\n eval_function=eval_function,\n maximize=maximize,\n generation=generation,\n intended_size=intended_size,\n checkpoint_target=checkpoint_target,\n serializer=serializer,\n concurrent_workers=concurrent_workers)\n self.contests_per_round = contests_per_round\n self.individuals_per_contest = individuals_per_contest\n\n def __copy__(self):\n result = self.__class__(chromosomes=[],\n eval_function=self.eval_function,\n maximize=self.maximize,\n contests_per_round=self.contests_per_round,\n individuals_per_contest=self.individuals_per_contest,\n serializer=self.serializer,\n intended_size=self.intended_size,\n generation=self.generation,\n concurrent_workers=1)\n result.individuals = [copy(individual) for individual in self.individuals]\n result.pool = self.pool\n result.concurrent_workers = self.concurrent_workers\n result.documented_best = None\n result.id = self.id\n return result\n\n def evaluate(self,\n lazy: bool = False,\n contests_per_round: Optional[int] = None,\n individuals_per_contest: Optional[int] = None) -> 'ContestPopulation':\n \"\"\"Evaluate the individuals in the population.\n\n This evaluates the fitness of all individuals. For each round of\n evaluation, each individual participates in a given number of\n contests, in which a given number of individuals take part.\n The resulting scores of these contests are summed to form the fitness.\n This means that the score of the individual is influenced by other\n chromosomes in the population.\n\n Note that in the `ContestPopulation` two settings are passed at\n initialisation which affect how we are evaluating individuals:\n contests_per_round and individuals_per_contest. You may overwrite them\n here if you wish.\n\n If lazy is True, the fitness is only evaluated when a fitness value\n is not yet known for all individuals.\n In most situations adding an explicit evaluation step is not needed, as\n lazy evaluation is implicitly included in the operations that need it\n (most notably in the survive operation).\n\n :param lazy: If True, do no re-evaluate the fitness if the fitness is known.\n :param contests_per_round: If set, overwrites the population setting for the\n number of contests there will be every round.\n :param individuals_per_contest: If set, overwrites the population setting for\n number of individuals to have in a contest during the evaluation.\n :return: self\n \"\"\"\n if contests_per_round is None:\n contests_per_round = self.contests_per_round\n if individuals_per_contest is None:\n individuals_per_contest = self.individuals_per_contest\n if lazy and all(individual.fitness is not None for individual in self):\n return self\n for individual in self.individuals:\n individual.fitness = 0\n contests = Contest.generate(individuals=self.individuals, individuals_per_contest=individuals_per_contest,\n contests_per_round=contests_per_round)\n if self.pool is None:\n for contest in contests:\n contest.assign_scores(self.eval_function(*contest.competitor_chromosomes))\n else:\n f = self.eval_function # We cannot refer to self in the map\n results = self.pool.map(lambda c: f(*c.competitor_chromosomes), contests)\n for result, contest in zip(results, contests):\n contest.assign_scores(result)\n return self\n\n def map(self, func: Callable[..., Individual], **kwargs) -> 'ContestPopulation':\n \"\"\"Apply the provided function to each individual in the population.\n\n Resets the fitness of all individuals.\n\n :param func: A function to apply to each individual in the population,\n which when called returns a modified individual.\n :param kwargs: Arguments to pass to the function.\n :return: self\n \"\"\"\n BasePopulation.map(self, func=func, **kwargs)\n self.reset_fitness()\n return self\n\n def filter(self, func: Callable[..., bool], **kwargs) -> 'ContestPopulation':\n \"\"\"Add a filter step to the Evolution.\n\n Filters the individuals in the population using the provided function.\n Resets the fitness of all individuals.\n\n :param func: Function to filter the individuals in the population by,\n which returns a boolean when called on an individual.\n :param kwargs: Arguments to pass to the function.\n :return: self\n \"\"\"\n BasePopulation.filter(self, func=func, **kwargs)\n self.reset_fitness()\n return self\n\n def survive(self,\n fraction: Optional[float] = None,\n n: Optional[int] = None,\n luck: bool = False) -> 'ContestPopulation':\n \"\"\"Let part of the population survive.\n\n Remove part of the population. If both fraction and n are specified,\n the minimum resulting population size is taken. Resets the fitness\n of all individuals.\n\n :param fraction: Fraction of the original population that survives.\n Defaults to None.\n :param n: Number of individuals of the population that survive.\n Defaults to None.\n :param luck: If True, individuals randomly survive (with replacement!)\n with chances proportional to their fitness. Defaults to False.\n :return: self\n \"\"\"\n BasePopulation.survive(self, fraction=fraction, n=n, luck=luck)\n self.reset_fitness()\n return self\n\n def reset_fitness(self):\n \"\"\"Reset the fitness of all individuals.\"\"\"\n for individual in self:\n individual.fitness = None\n" }, { "path": "evol/problems/__init__.py", "content": "" }, { "path": "evol/problems/functions/__init__.py", "content": "\"\"\"\nThe `evol.problems.functions` part of the library contains\nsimple problem instances that do with known math functions.\n\nThe functions in here are typically inspired from wikipedia:\nhttps://en.wikipedia.org/wiki/Test_functions_for_optimization\n\"\"\"\n\nfrom .variableinput import Rosenbrock, Sphere, Rastrigin" }, { "path": "evol/problems/functions/variableinput.py", "content": "import math\nfrom typing import Sequence\n\nfrom evol.helpers.utils import sliding_window\nfrom evol.problems.problem import Problem\n\n\nclass FunctionProblem(Problem):\n def __init__(self, size=2):\n self.size = size\n\n def check_solution(self, solution: Sequence[float]) -> Sequence[float]:\n if len(solution) > self.size:\n raise ValueError(f\"{self.__class__.__name__} has size {self.size}, \\\n got solution of size: {len(solution)}\")\n return solution\n\n def value(self, solution):\n return sum(solution)\n\n def eval_function(self, solution: Sequence[float]) -> float:\n self.check_solution(solution)\n return self.value(solution)\n\n\nclass Sphere(FunctionProblem):\n def value(self, solution: Sequence[float]) -> float:\n \"\"\"\n The optimal value can be found when a sequence of zeros is given.\n :param solution: a sequence of x_i values\n :return: the value of the Sphere function\n \"\"\"\n return sum([_**2 for _ in solution])\n\n\nclass Rosenbrock(FunctionProblem):\n def value(self, solution: Sequence[float]) -> float:\n \"\"\"\n The optimal value can be found when a sequence of ones is given.\n :param solution: a sequence of x_i values\n :return: the value of the Rosenbrock function\n \"\"\"\n result = 0\n for x_i, x_j in sliding_window(solution):\n result += 100*(x_j - x_i**2)**2 + (1 - x_i)**2\n return result\n\n\nclass Rastrigin(FunctionProblem):\n def value(self, solution: Sequence[float]) -> float:\n \"\"\"\n The optimal value can be found when a sequence of zeros is given.\n :param solution: a sequence of x_i values\n :return: the value of the Rosenbrock function\n \"\"\"\n return (10 * self.size) + sum([_**2 - 10 * math.cos(2*math.pi*_) for _ in solution])\n" }, { "path": "evol/problems/problem.py", "content": "from abc import ABCMeta, abstractmethod\n\n\nclass Problem(metaclass=ABCMeta):\n\n @abstractmethod\n def eval_function(self, solution):\n raise NotImplementedError\n" }, { "path": "evol/problems/routing/__init__.py", "content": "\"\"\"\nThe `evol.problems.routing` part of the library contains\nsimple problem instances that do with routing problems. These\nare meant to be used for education and training purposes and\nthese problems are typically good starting points if you want\nto play with the library.\n\"\"\"\n\nfrom .tsp import TSPProblem\nfrom .magicsanta import MagicSanta\n" }, { "path": "evol/problems/routing/coordinates.py", "content": "united_states_capitols = [\n (32.361538, -86.279118, \"Montgomery\", \"Alabama\"),\n (58.301935, -134.419740, \"Juneau\", \"Alaska\"),\n (33.448457, -112.073844, \"Phoenix\", \"Arizona\"),\n (34.736009, -92.331122, \"Little Rock\", \"Arkansas\"),\n (38.555605, -121.468926, \"Sacramento\", \"California\"),\n (39.7391667, -104.984167, \"Denver\", \"Colorado\"),\n (41.767, -72.677, \"Hartford\", \"Connectic\"),\n (39.161921, -75.526755, \"Dover\", \"Delaware\"),\n (30.4518, -84.27277, \"Tallahassee\", \"Florida\"),\n (33.76, -84.39, \"Atlanta\", \"Georgia\"),\n (21.30895, -157.826182, \"Honolulu\", \"Hawaii\"),\n (43.613739, -116.237651, \"Boise\", \"Idaho\"),\n (39.783250, -89.650373, \"Springfield\", \"Illinois\"),\n (39.790942, -86.147685, \"Indianapolis\", \"Indiana\"),\n (41.590939, -93.620866, \"Des Moines\", \"Iowa\"),\n (39.04, -95.69, \"Topeka\", \"Kansas\"),\n (38.197274, -84.86311, \"Frankfort\", \"Kentucky\"),\n (30.45809, -91.140229, \"Baton Rouge\", \"Louisiana\"),\n (44.323535, -69.765261, \"Augusta\", \"Maine\"),\n (38.972945, -76.501157, \"Annapolis\", \"Maryland\"),\n (42.2352, -71.0275, \"Boston\", \"Massachuset\"),\n (42.7335, -84.5467, \"Lansing\", \"Michigan\"),\n (44.95, -93.094, \"Saint Paul\", \"Minnesot\"),\n (32.320, -90.207, \"Jackson\", \"Mississip\"),\n (38.572954, -92.189283, \"Jefferson City\", \"Missouri\"),\n (46.595805, -112.027031, \"Helana\", \"Montana\"),\n (40.809868, -96.675345, \"Lincoln\", \"Nebraska\"),\n (39.160949, -119.753877, \"Carson City\", \"Nevada\"),\n (43.220093, -71.549127, \"Concord\", \"Hampshire\"),\n (40.221741, -74.756138, \"Trenton\", \"Jersey\"),\n (35.667231, -105.964575, \"Santa Fe\", \"Mexico\"),\n (42.659829, -73.781339, \"Albany\", \"York\"),\n (35.771, -78.638, \"Raleigh\", \"Car\"),\n (48.813343, -100.779004, \"Bismarck\", \"Dakota\"),\n (39.962245, -83.000647, \"Columbus\", \"Ohio\"),\n (35.482309, -97.534994, \"Oklahoma City\", \"Oklahoma\"),\n (44.931109, -123.029159, \"Salem\", \"Oregon\"),\n (40.269789, -76.875613, \"Harrisburg\", \"Pennsylvania\"),\n (41.82355, -71.422132, \"Providence\", \"Island\"),\n (34.000, -81.035, \"Columbia\", \"Car\"),\n (44.367966, -100.336378, \"Pierre\", \"Dakota\"),\n (36.165, -86.784, \"Nashville\", \"Tennessee\"),\n (30.266667, -97.75, \"Austin\", \"Texas\"),\n (40.7547, -111.892622, \"Salt Lake City\", \"Utah\"),\n (44.26639, -72.57194, \"Montpelier\", \"Vermont\"),\n (37.54, -77.46, \"Richmond\", \"Virgini\"),\n (47.042418, -122.893077, \"Olympia\", \"Washington\"),\n (38.349497, -81.633294, \"Charleston\", \"Virginia\"),\n (43.074722, -89.384444, \"Madison\", \"Wisconsin\"),\n (41.145548, -104.802042, \"Cheyenne\", \"Wyoming\")]\n" }, { "path": "evol/problems/routing/magicsanta.py", "content": "import math\nfrom collections import Counter\nfrom itertools import chain\nfrom typing import List, Union\n\nfrom evol.helpers.utils import sliding_window\nfrom evol.problems.problem import Problem\n\n\nclass MagicSanta(Problem):\n def __init__(self, city_coordinates, home_coordinate, gift_weight=None, sleigh_weight=1):\n \"\"\"\n This problem is based on this kaggle competition:\n https://www.kaggle.com/c/santas-stolen-sleigh#evaluation.\n :param city_coordinates: List of tuples containing city coordinates.\n :param home_coordinate: Tuple containing coordinate of home base.\n :param gift_weight: Vector of weights per gift associated with cities.\n :param sleigh_weight: Weight of the sleight.\n \"\"\"\n self.coordinates = city_coordinates\n self.home_coordinate = home_coordinate\n self.gift_weight = gift_weight\n if gift_weight is None:\n self.gift_weight = [1 for _ in city_coordinates]\n self.sleigh_weight = sleigh_weight\n\n @staticmethod\n def distance(coord_a, coord_b):\n return math.sqrt(sum([(z[0] - z[1]) ** 2 for z in zip(coord_a, coord_b)]))\n\n def check_solution(self, solution: List[List[int]]):\n \"\"\"\n Check if the solution for the problem is valid.\n :param solution: List of lists containing integers representing visited cities.\n :return: None, unless errors are raised.\n \"\"\"\n set_visited = set(chain.from_iterable(solution))\n set_problem = set(range(len(self.coordinates)))\n if set_visited != set_problem:\n missing = set_problem.difference(set_visited)\n extra = set_visited.difference(set_problem)\n raise ValueError(f\"Not all cities are visited! Missing: {missing} Extra: {extra}\")\n city_counter = Counter(chain.from_iterable(solution))\n if max(city_counter.values()) > 1:\n double_cities = {key for key, value in city_counter.items() if value > 1}\n raise ValueError(f\"Multiple occurrences found for cities: {double_cities}\")\n\n def eval_function(self, solution: List[List[int]]) -> Union[float, int]:\n \"\"\"\n Calculates the cost of the current solution for the TSP problem.\n :param solution: List of integers which refer to cities.\n :return:\n \"\"\"\n self.check_solution(solution=solution)\n cost = 0\n for route in solution:\n total_route_weight = sum([self.gift_weight[t] for t in route]) + self.sleigh_weight\n distance = self.distance(self.home_coordinate, self.coordinates[route[0]])\n cost += distance * total_route_weight\n for t1, t2 in sliding_window(route):\n total_route_weight -= self.gift_weight[t1]\n city1 = self.coordinates[t1]\n city2 = self.coordinates[t2]\n cost += self.distance(city1, city2) * total_route_weight\n last_leg_distance = self.distance(self.coordinates[route[-1]], self.home_coordinate)\n cost += self.sleigh_weight * last_leg_distance\n return cost\n" }, { "path": "evol/problems/routing/tsp.py", "content": "import math\nfrom typing import List, Union\n\nfrom evol.problems.problem import Problem\nfrom evol.helpers.utils import rotating_window\n\n\nclass TSPProblem(Problem):\n def __init__(self, distance_matrix):\n self.distance_matrix = distance_matrix\n\n @classmethod\n def from_coordinates(cls, coordinates: List[Union[tuple, list]]) -> 'TSPProblem':\n \"\"\"\n Creates a distance matrix from a list of city coordinates.\n :param coordinates: An iterable that contains tuples or lists representing a x,y coordinate.\n :return: A list of lists containing the distances between cities.\n \"\"\"\n res = [[0 for i in coordinates] for j in coordinates]\n for i, coord_i in enumerate(coordinates):\n for j, coord_j in enumerate(coordinates):\n dist = math.sqrt(sum([(z[0] - z[1])**2 for z in zip(coord_i[:2], coord_j[:2])]))\n res[i][j] = dist\n res[j][i] = dist\n return TSPProblem(distance_matrix=res)\n\n def check_solution(self, solution: List[int]):\n \"\"\"\n Check if the solution for the TSP problem is valid.\n :param solution: List of integers which refer to cities.\n :return: None, unless errors are raised.\n \"\"\"\n set_solution = set(solution)\n set_problem = set(range(len(self.distance_matrix)))\n if len(solution) > len(self.distance_matrix):\n raise ValueError(\"Solution is longer than number of towns!\")\n if set_solution != set_problem:\n raise ValueError(f\"Not all towns are visited! Am missing {set_problem.difference(set_solution)}\")\n\n def eval_function(self, solution: List[int]) -> Union[float, int]:\n \"\"\"\n Calculates the cost of the current solution for the TSP problem.\n :param solution: List of integers which refer to cities.\n :return:\n \"\"\"\n self.check_solution(solution=solution)\n cost = 0\n for t1, t2 in rotating_window(solution):\n cost += self.distance_matrix[t1][t2]\n return cost\n" }, { "path": "evol/serialization.py", "content": "\"\"\"\nSerializers help store (checkpoint) the state of your population during or\nafter running your evolutionary algorithm. By default, each Population is\ninitialized with a SimpleSerializer, which you can use to store the individuals\nin your population in pickle or json format using the .checkpoint() method of\nthe population. Currently no other serializers are available.\n\"\"\"\nimport json\nimport pickle\nfrom datetime import datetime\nfrom typing import List, Optional\n\nfrom os import listdir\nfrom os.path import isdir, exists, join\n\nfrom evol import Individual\n\n\nclass SimpleSerializer:\n \"\"\"The SimpleSerializer handles serialization to and from pickle and json.\n\n :param target: Default location (directory) to store checkpoint.\n This may be overridden in the `checkpoint` method. Defaults to None.\n \"\"\"\n\n def __init__(self, target: Optional[str] = None):\n self.target = target\n\n def checkpoint(self, individuals: List[Individual], target: Optional[str] = None, method: str = 'pickle') -> None:\n \"\"\"Checkpoint a list of individuals.\n\n :param individuals: List of individuals to checkpoint.\n :param target: Directory to write checkpoint to. If None, the Serializer default target is taken,\n which can be provided upon initialisation. Defaults to None.\n :param method: One of 'pickle' or 'json'. When 'json', the chromosomes need to be json-serializable.\n Defaults to 'pickle'.\n \"\"\"\n filename = self._new_checkpoint_file(target=self.target if target is None else target, method=method)\n if method == 'pickle':\n with open(filename, 'wb') as pickle_file:\n pickle.dump(individuals, pickle_file)\n elif method == 'json':\n with open(filename, 'w') as json_file:\n json.dump([individual.__dict__ for individual in individuals], json_file)\n else:\n raise ValueError('Invalid checkpointing method \"{}\". Choose \"pickle\" or \"json\".'.format(method))\n\n def load(self, target: Optional[str] = None) -> List[Individual]:\n \"\"\"Load a checkpoint.\n\n If path is a file, load that file. If it is a directory, load the most recent checkpoint.\n The checkpoint file must end with a '.json' or '.pkl' extension.\n\n :param target: Path to checkpoint directory or file.\n :return: List of individuals from checkpoint.\n \"\"\"\n filename = self._find_checkpoint(self.target if target is None else target)\n if filename.endswith('.json'):\n with open(filename, 'r') as json_file:\n return [Individual.from_dict(d) for d in json.load(json_file)]\n elif filename.endswith('.pkl'):\n with open(filename, 'rb') as pickle_file:\n return pickle.load(pickle_file)\n\n @staticmethod\n def _new_checkpoint_file(target: str, method: str):\n \"\"\"Generate a filename for a new checkpoint.\"\"\"\n if target is None:\n raise ValueError('Serializer requires a target to checkpoint to.')\n if not isdir(target):\n raise FileNotFoundError('Cannot checkpoint to \"{}\": is not a directory.'.format(target))\n result = join(target, datetime.now().strftime(\"%Y%m%d-%H%M%S.%f\") + ('.pkl' if method == 'pickle' else '.json'))\n if exists(result):\n raise FileExistsError('Cannot checkpoint to \"{}\": file exists.'.format(result))\n return result\n\n @classmethod\n def _find_checkpoint(cls, target: str):\n \"\"\"Find the most recent checkpoint file.\"\"\"\n if not exists(target):\n raise FileNotFoundError('Cannot load from \"{}\": file or directory does not exists.'.format(target))\n elif isdir(target):\n try:\n return join(target, max(filter(cls._has_valid_extension, listdir(path=target))))\n except ValueError:\n raise FileNotFoundError('Cannot load from \"{}\": directory contains no checkpoints.'.format(target))\n else:\n if not cls._has_valid_extension(target):\n raise ValueError('Invalid extension \"{}\": Was expecting \".pkl\" or \".json\".'.format(target))\n return target\n\n @staticmethod\n def _has_valid_extension(filename: str):\n \"\"\"Check if a filename has a valid extension.\"\"\"\n return filename.endswith('.pkl') or filename.endswith('.json')\n" }, { "path": "evol/step.py", "content": "from abc import ABCMeta, abstractmethod\nfrom typing import Callable, Optional, TYPE_CHECKING\n\nfrom evol.population import BasePopulation\n\nif TYPE_CHECKING:\n from evol.evolution import Evolution\n\n\nclass EvolutionStep(metaclass=ABCMeta):\n\n def __init__(self, name: Optional[str], **kwargs):\n self.name = name\n self.kwargs = kwargs\n\n def __repr__(self):\n return f\"{self.__class__.__name__}({self.name or ''})\"\n\n @abstractmethod\n def apply(self, population: BasePopulation) -> BasePopulation:\n pass\n\n\nclass EvaluationStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.evaluate(**self.kwargs)\n\n\nclass CheckpointStep(EvolutionStep):\n\n def __init__(self, name, every=1, **kwargs):\n EvolutionStep.__init__(self, name, **kwargs)\n self.count = 0\n self.every = every\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n self.count += 1\n if self.count >= self.every:\n self.count = 0\n return population.checkpoint(**self.kwargs)\n return population\n\n\nclass MapStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.map(**self.kwargs)\n\n\nclass FilterStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.filter(**self.kwargs)\n\n\nclass SurviveStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.survive(**self.kwargs)\n\n\nclass BreedStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.breed(**self.kwargs)\n\n\nclass MutateStep(EvolutionStep):\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n return population.mutate(**self.kwargs)\n\n\nclass RepeatStep(EvolutionStep):\n\n def __init__(self, name: str, evolution: 'Evolution', n: int,\n grouping_function: Optional[Callable] = None, **kwargs):\n super().__init__(name=name, **kwargs)\n self.evolution = evolution\n self.n = n\n self.grouping_function = grouping_function\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n if self.grouping_function is None:\n if len(self.kwargs) > 0:\n raise ValueError(f'Unexpected argument(s) for non-grouped repeat step: {self.kwargs}')\n return population.evolve(evolution=self.evolution, n=self.n)\n else:\n return self._apply_grouped(population=population)\n\n def _apply_grouped(self, population: BasePopulation) -> BasePopulation:\n groups = population.group(grouping_function=self.grouping_function, **self.kwargs)\n if population.pool:\n results = population.pool.map(lambda group: group.evolve(evolution=self.evolution, n=self.n), groups)\n else:\n results = [group.evolve(evolution=self.evolution, n=self.n) for group in groups]\n return population.combine(*results, intended_size=population.intended_size, pool=population.pool)\n\n def __repr__(self):\n result = f\"{self.__class__.__name__}({self.name or ''}) with evolution ({self.n}x):\\n \"\n result += repr(self.evolution).replace('\\n', '\\n ')\n return result\n\n\nclass CallbackStep(EvolutionStep):\n def __init__(self, name, every: int = 1, **kwargs):\n EvolutionStep.__init__(self, name, **kwargs)\n self.count = 0\n self.every = every\n\n def apply(self, population: BasePopulation) -> BasePopulation:\n self.count += 1\n if self.count >= self.every:\n self.count = 0\n return population.callback(**self.kwargs)\n return population\n" }, { "path": "evol/utils.py", "content": "from inspect import signature\nfrom typing import List, Callable, Union, Sequence, Any, Generator\n\nfrom evol import Individual\n\n\ndef offspring_generator(parents: List[Individual],\n parent_picker: Callable[..., Union[Individual, Sequence]],\n combiner: Callable[..., Any],\n **kwargs) -> Generator[Individual, None, None]:\n \"\"\"Generator for offspring.\n\n This helps create the right number of offspring,\n especially in the case of of multiple offspring.\n\n :param parents: List of parents.\n :param parent_picker: Function that selects parents. Must accept a sequence of\n individuals and must return a single individual or a sequence of individuals.\n Must accept all kwargs passed (i.e. must be decorated by select_arguments).\n :param combiner: Function that combines chromosomes. Must accept a tuple of\n chromosomes and either return a single chromosome or yield multiple chromosomes.\n Must accept all kwargs passed (i.e. must be decorated by select_arguments).\n :param kwargs: Arguments\n :returns: Children\n \"\"\"\n while True:\n # Obtain parent chromosomes\n selected_parents = parent_picker(parents, **kwargs)\n if isinstance(selected_parents, Individual):\n chromosomes = (selected_parents.chromosome,)\n else:\n chromosomes = tuple(individual.chromosome for individual in selected_parents)\n # Create children\n combined = combiner(*chromosomes, **kwargs)\n if isinstance(combined, Generator):\n for child in combined:\n yield Individual(chromosome=child)\n else:\n yield Individual(chromosome=combined)\n\n\ndef select_arguments(func: Callable) -> Callable:\n \"\"\"Decorate a function such that it accepts any keyworded arguments.\n\n The resulting function accepts any arguments, but only arguments that\n the original function accepts are passed. This allows keyworded\n arguments to be passed to multiple (decorated) functions, even if they\n do not (all) accept these arguments.\n\n :param func: Function to decorate.\n :return: Callable\n \"\"\"\n def result(*args, **kwargs):\n try:\n return func(*args, **kwargs)\n except TypeError:\n return func(*args, **{k: v for k, v in kwargs.items() if k in signature(func).parameters})\n\n return result\n" }, { "path": "examples/number_of_parents.py", "content": "\"\"\"\nThere are a few worthwhile things to notice in this example:\n\n1. you can pass hyperparams into functions from the `.breed` and `.mutate` step\n2. the algorithm does not care how many parents it will use in the `breed` step\n\"\"\"\n\nimport random\nimport math\nimport argparse\n\nfrom evol import Population, Evolution\n\n\ndef run_evolutionary(opt_value=1, population_size=100, n_parents=2, workers=1,\n num_iter=200, survival=0.5, noise=0.1, seed=42, verbose=True):\n random.seed(seed)\n\n def init_func():\n return (random.random() - 0.5) * 20 + 10\n\n def eval_func(x, opt_value=opt_value):\n return -((x - opt_value) ** 2) + math.cos(x - opt_value)\n\n def random_parent_picker(pop, n_parents):\n return [random.choice(pop) for i in range(n_parents)]\n\n def mean_parents(*parents):\n return sum(parents) / len(parents)\n\n def add_noise(chromosome, sigma):\n return chromosome + (random.random() - 0.5) * sigma\n\n pop = Population(chromosomes=[init_func() for _ in range(population_size)],\n eval_function=eval_func, maximize=True, concurrent_workers=workers).evaluate()\n\n evo = (Evolution()\n .survive(fraction=survival)\n .breed(parent_picker=random_parent_picker, combiner=mean_parents, n_parents=n_parents)\n .mutate(mutate_function=add_noise, sigma=noise)\n .evaluate())\n\n for i in range(num_iter):\n pop = pop.evolve(evo)\n\n if verbose:\n print(f\"iteration:{i} best: {pop.current_best.fitness} worst: {pop.current_worst.fitness}\")\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(description='Run an example evol algorithm against a simple continuous function.')\n parser.add_argument('--opt-value', type=int, default=0,\n help='the true optimal value of the problem')\n parser.add_argument('--population-size', type=int, default=20,\n help='the number of candidates to start the algorithm with')\n parser.add_argument('--n-parents', type=int, default=2,\n help='the number of parents the algorithm with use to generate new indivuals')\n parser.add_argument('--num-iter', type=int, default=20,\n help='the number of evolutionary cycles to run')\n parser.add_argument('--survival', type=float, default=0.7,\n help='the fraction of individuals who will survive a generation')\n parser.add_argument('--noise', type=float, default=0.5,\n help='the amount of noise the mutate step will add to each individual')\n parser.add_argument('--seed', type=int, default=42,\n help='the random seed for all this')\n parser.add_argument('--workers', type=int, default=1,\n help='the number of workers to run the command in')\n\n args = parser.parse_args()\n run_evolutionary(opt_value=args.opt_value, population_size=args.population_size,\n n_parents=args.n_parents, num_iter=args.num_iter,\n noise=args.noise, seed=args.seed, workers=args.workers)\n" }, { "path": "examples/population_demo.py", "content": "import random\nfrom evol import Population\n\n\ndef create_candidate():\n return random.random() - 0.5\n\n\ndef func_to_optimise(x):\n return x*2\n\n\ndef pick_random_parents(pop):\n return random.choice(pop)\n\n\nrandom.seed(42)\n\npop1 = Population(chromosomes=[create_candidate() for _ in range(5)],\n eval_function=func_to_optimise, maximize=True)\n\npop2 = Population.generate(init_function=create_candidate,\n eval_function=func_to_optimise,\n size=5,\n maximize=False)\n\n\nprint(\"[i for i in pop1]:\")\nprint([i for i in pop1])\nprint(\"[i.chromosome for i in pop1]:\")\nprint([i.chromosome for i in pop1])\nprint(\"[i.fitness for i in pop1]:\")\nprint([i.fitness for i in pop1])\nprint(\"[i.fitness for i in pop1.evaluate()]:\")\n\n\ndef produce_clone(parent):\n return parent\n\n\ndef add_noise(x):\n return 0.1 * (random.random() - 0.5) + x\n\n\nprint(\"[i.fitness for i in pop1.survive(n=3)]:\")\nprint([i.fitness for i in pop1.survive(n=3)])\nprint(\"[i.fitness for i in pop1.survive(n=3).mutate(add_noise)]:\")\nprint([i.fitness for i in pop1.survive(n=3).mutate(add_noise)])\nprint(\"[i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()]:\")\nprint([i.fitness for i in pop1.survive(n=3).mutate(add_noise).evaluate()])\n" }, { "path": "examples/rock_paper_scissors.py", "content": "#!/usr/bin/env python\n\nfrom argparse import ArgumentParser\nfrom collections import Counter\nfrom random import choice, random, seed\nfrom typing import List\n\nfrom evol import Evolution, ContestPopulation\nfrom evol.helpers.pickers import pick_random\nfrom evol.helpers.groups import group_duplicate\n\n\nclass RockPaperScissorsPlayer:\n arbitrariness = 0.0\n elements = ('rock', 'paper', 'scissors')\n\n def __init__(self, preference=None):\n self.preference = preference if preference else choice(self.elements)\n\n def __repr__(self):\n return '{}({})'.format(self.__class__.__name__, self.preference)\n\n def play(self):\n if random() >= self.arbitrariness:\n return self.preference\n else:\n return choice(self.elements)\n\n def mutate(self, volatility=0.1):\n if random() < volatility:\n return self.__class__(choice(self.elements))\n else:\n return self.__class__(self.preference)\n\n def combine(self, other):\n return self.__class__(choice([self.preference, other.preference]))\n\n\nclass RockPaperScissorsLizardSpockPlayer(RockPaperScissorsPlayer):\n elements = ('rock', 'paper', 'scissors', 'lizard', 'spock')\n\n\nCOMBINATIONS = [\n ('scissors', 'paper'),\n ('paper', 'rock'),\n ('rock', 'scissors'),\n ('rock', 'lizard'),\n ('lizard', 'spock'),\n ('spock', 'scissors'),\n ('scissors', 'lizard'),\n ('lizard', 'paper'),\n ('paper', 'spock'),\n ('spock', 'rock'),\n]\n\n\ndef evaluate(player_1: RockPaperScissorsPlayer, player_2: RockPaperScissorsPlayer) -> List[float]:\n choice_1, choice_2 = player_1.play(), player_2.play()\n player_choices = {choice_1, choice_2}\n if len(player_choices) == 1:\n return [0, 0]\n for combination in COMBINATIONS:\n if set(combination) == player_choices:\n return [1, -1] if choice_1 == combination[0] else [-1, 1]\n\n\nclass History:\n\n def __init__(self):\n self.history = []\n\n def log(self, population: ContestPopulation):\n preferences = Counter()\n for individual in population:\n preferences.update([individual.chromosome.preference])\n self.history.append(dict(**preferences, id=population.id, generation=population.generation))\n\n def plot(self):\n try:\n import pandas as pd\n import matplotlib.pylab as plt\n df = pd.DataFrame(self.history).set_index(['id', 'generation']).fillna(0)\n population_size = sum(df.iloc[0].values)\n n_populations = df.reset_index()['id'].nunique()\n fig, axes = plt.subplots(nrows=n_populations, figsize=(12, 2*n_populations),\n sharex='all', sharey='all', squeeze=False)\n for row, (_, pop) in zip(axes, df.groupby('id')):\n ax = row[0]\n pop.reset_index(level='id', drop=True).plot(ax=ax)\n ax.set_ylim([0, population_size])\n ax.set_xlabel('iteration')\n ax.set_ylabel('# w/ preference')\n if n_populations > 1:\n for i in range(0, df.reset_index().generation.max(), 50):\n ax.axvline(i)\n plt.show()\n except ImportError:\n print(\"If you install matplotlib and pandas you will get a pretty plot.\")\n\n\ndef run_rock_paper_scissors(population_size: int = 100,\n n_iterations: int = 200,\n random_seed: int = 42,\n survive_fraction: float = 0.8,\n arbitrariness: float = 0.2,\n concurrent_workers: int = 1,\n lizard_spock: bool = False,\n grouped: bool = False,\n silent: bool = False):\n seed(random_seed)\n\n RockPaperScissorsPlayer.arbitrariness = arbitrariness\n\n player_class = RockPaperScissorsLizardSpockPlayer if lizard_spock else RockPaperScissorsPlayer\n pop = ContestPopulation(chromosomes=[player_class() for _ in range(population_size)],\n eval_function=evaluate, maximize=True,\n concurrent_workers=concurrent_workers).evaluate()\n history = History()\n\n evo = Evolution().repeat(\n evolution=(Evolution()\n .survive(fraction=survive_fraction)\n .breed(parent_picker=pick_random, combiner=lambda x, y: x.combine(y), n_parents=2)\n .mutate(lambda x: x.mutate())\n .evaluate()\n .callback(history.log)),\n n=n_iterations // 4,\n grouping_function=group_duplicate if grouped else None\n )\n\n pop.evolve(evo, n=4)\n\n if not silent:\n history.plot()\n return history\n\n\ndef parse_arguments():\n parser = ArgumentParser(description='Run the rock-paper-scissors example.')\n parser.add_argument('--population-size', dest='population_size', type=int, default=100,\n help='the number of candidates to start the algorithm with')\n parser.add_argument('--n-iterations', dest='n_iterations', type=int, default=200,\n help='the number of iterations to run')\n parser.add_argument('--random-seed', dest='random_seed', type=int, default=42,\n help='the random seed to set')\n parser.add_argument('--survive-fraction', dest='survive_fraction', type=float, default=0.8,\n help='the fraction of the population to survive each iteration')\n parser.add_argument('--arbitrariness', type=float, default=0.2,\n help='arbitrariness of the players. if zero, player will always choose its preference')\n parser.add_argument('--concurrent_workers', type=int, default=1,\n help='Concurrent workers to use to evaluate the population.')\n parser.add_argument('--lizard-spock', action='store_true', default=False,\n help='Play rock-paper-scissors-lizard-spock.')\n parser.add_argument('--grouped', action='store_true', default=False,\n help='Run the evolution in four groups.')\n return parser.parse_args()\n\n\nif __name__ == \"__main__\":\n args = parse_arguments()\n run_rock_paper_scissors(**args.__dict__)\n" }, { "path": "examples/simple_callback.py", "content": "\"\"\"\nThis example demonstrates how logging works in evolutions.\n\"\"\"\n\nimport random\nfrom evol import Population, Evolution\n\n\ndef random_start():\n \"\"\"\n This function generates a random (x,y) coordinate in the searchspace\n \"\"\"\n return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20\n\n\ndef func_to_optimise(xy):\n \"\"\"\n This is the function we want to optimise (maximize)\n \"\"\"\n x, y = xy\n return -(1-x)**2 - 2*(2-x**2)**2\n\n\ndef pick_random_parents(pop):\n \"\"\"\n This is how we are going to select parents from the population\n \"\"\"\n mom = random.choice(pop)\n dad = random.choice(pop)\n return mom, dad\n\n\ndef make_child(mom, dad):\n \"\"\"\n This is how two parents are going to make a child.\n Note that the output of a tuple, just like the output of `random_start`\n \"\"\"\n child_x = (mom[0] + dad[0])/2\n child_y = (mom[1] + dad[1])/2\n return child_x, child_y\n\n\ndef add_noise(chromosome, sigma):\n \"\"\"\n This is a function that will add some noise to the chromosome.\n \"\"\"\n new_x = chromosome[0] + (random.random()-0.5) * sigma\n new_y = chromosome[1] + (random.random()-0.5) * sigma\n return new_x, new_y\n\n\nclass MyLogger():\n def __init__(self):\n self.i = 0\n\n def log(self, pop):\n self.i += 1\n best = max([i.fitness for i in pop.evaluate()])\n print(f\"the best score i={self.i} => {best}\")\n\n\nif __name__ == \"__main__\":\n logger = MyLogger()\n random.seed(42)\n\n pop = Population(chromosomes=[random_start() for _ in range(200)],\n eval_function=func_to_optimise,\n maximize=True, concurrent_workers=2)\n\n evo1 = (Evolution()\n .survive(fraction=0.1)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=0.2)\n .evaluate()\n .callback(logger.log))\n\n evo2 = (Evolution()\n .survive(n=10)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=0.1)\n .evaluate()\n .callback(logger.log))\n\n evo3 = (Evolution()\n .repeat(evo1, n=20)\n .repeat(evo2, n=20))\n\n pop = pop.evolve(evo3, n=3)\n" }, { "path": "examples/simple_logging.py", "content": "\"\"\"\nThis example demonstrates how logging works in evolutions.\n\"\"\"\n\nimport random\nfrom tempfile import NamedTemporaryFile\nfrom evol import Population, Evolution\nfrom evol.logger import BaseLogger\n\nrandom.seed(42)\n\n\ndef random_start():\n \"\"\"\n This function generates a random (x,y) coordinate in the searchspace\n \"\"\"\n return (random.random() - 0.5) * 20, (random.random() - 0.5) * 20\n\n\ndef func_to_optimise(xy):\n \"\"\"\n This is the function we want to optimise (maximize)\n \"\"\"\n x, y = xy\n return -(1 - x) ** 2 - 2 * (2 - x ** 2) ** 2\n\n\ndef pick_random_parents(pop):\n \"\"\"\n This is how we are going to select parents from the population\n \"\"\"\n mom = random.choice(pop)\n dad = random.choice(pop)\n return mom, dad\n\n\ndef make_child(mom, dad):\n \"\"\"\n This is how two parents are going to make a child.\n Note that the output of a tuple, just like the output of `random_start`\n \"\"\"\n child_x = (mom[0] + dad[0]) / 2\n child_y = (mom[1] + dad[1]) / 2\n return child_x, child_y\n\n\ndef add_noise(chromosome, sigma):\n \"\"\"\n This is a function that will add some noise to the chromosome.\n \"\"\"\n new_x = chromosome[0] + (random.random() - 0.5) * sigma\n new_y = chromosome[1] + (random.random() - 0.5) * sigma\n return new_x, new_y\n\n\nwith NamedTemporaryFile() as tmpfile:\n logger = BaseLogger(target=tmpfile.name)\n pop = Population(chromosomes=[random_start() for _ in range(200)],\n eval_function=func_to_optimise,\n maximize=True, concurrent_workers=2)\n\n evo1 = (Evolution()\n .survive(fraction=0.1)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=0.2)\n .callback(logger.log))\n\n evo2 = (Evolution()\n .survive(n=10)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=0.1)\n .callback(logger.log))\n\n evo3 = (Evolution()\n .repeat(evo1, n=20)\n .repeat(evo2, n=20))\n\n pop = pop.evolve(evo3, n=3)\n" }, { "path": "examples/simple_nonlinear.py", "content": "import random\nfrom random import random as r\nfrom evol import Population, Evolution\n\nrandom.seed(42)\n\n\ndef random_start():\n \"\"\"\n This function generates a random (x,y) coordinate\n \"\"\"\n return (r() - 0.5) * 20, (r() - 0.5) * 20\n\n\ndef func_to_optimise(xy):\n \"\"\"\n This is the function we want to optimise (maximize)\n \"\"\"\n x, y = xy\n return -(1 - x) ** 2 - (2 - y ** 2) ** 2\n\n\ndef pick_random_parents(pop):\n \"\"\"\n This is how we are going to select parents from the population\n \"\"\"\n mom = random.choice(pop)\n dad = random.choice(pop)\n return mom, dad\n\n\ndef make_child(mom, dad):\n \"\"\"\n This function describes how two candidates combine into a\n new candidate. Note that the output is a tuple, just like\n the output of `random_start`. We leave it to the developer\n to ensure that chromosomes are of the same type.\n \"\"\"\n child_x = (mom[0] + dad[0]) / 2\n child_y = (mom[1] + dad[1]) / 2\n return child_x, child_y\n\n\ndef add_noise(chromosome, sigma):\n \"\"\"\n This is a function that will add some noise to the chromosome.\n \"\"\"\n new_x = chromosome[0] + (r() - 0.5) * sigma\n new_y = chromosome[1] + (r() - 0.5) * sigma\n return new_x, new_y\n\n\n# We start by defining a population with candidates.\npop = Population(chromosomes=[random_start() for _ in range(200)],\n eval_function=func_to_optimise, maximize=True)\n\n# We do a single step here and out comes a new population\npop.survive(fraction=0.5)\n\n# We do two steps here and out comes a new population\n(pop\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random_parents, combiner=make_child))\n\n# We do a three steps here and out comes a new population\n(pop\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=1))\n\n# This is inelegant but it works.\nfor i in range(5):\n pop = (pop\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=1))\n\n# We define a sequence of steps to change these candidates\nevo1 = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=1))\n\n# We define another sequence of steps to change these candidates\nevo2 = (Evolution()\n .survive(n=1)\n .breed(parent_picker=pick_random_parents, combiner=make_child)\n .mutate(mutate_function=add_noise, sigma=0.2))\n\n# We are combining two evolutions into a third one.\n# You don't have to but this approach demonstrates\n# the flexibility of the library.\nevo3 = (Evolution()\n .repeat(evo1, n=50)\n .repeat(evo2, n=10)\n .evaluate())\n\n# In this step we are telling evol to apply the evolutions\n# to the population of candidates.\npop = pop.evolve(evo3, n=5)\nprint(f\"the best score found: {max([i.fitness for i in pop])}\")\n" }, { "path": "examples/travelling_salesman.py", "content": "#!/usr/bin/env python\nfrom argparse import ArgumentParser\nfrom math import sqrt\nfrom random import random, seed, shuffle\nfrom typing import List, Optional\n\nfrom evol import Evolution, Population\nfrom evol.helpers.combiners.permutation import cycle_crossover\nfrom evol.helpers.groups import group_stratified\nfrom evol.helpers.mutators.permutation import swap_elements\nfrom evol.helpers.pickers import pick_random\n\n\ndef run_travelling_salesman(population_size: int = 100,\n n_iterations: int = 10,\n random_seed: int = 0,\n n_destinations: int = 50,\n concurrent_workers: Optional[int] = None,\n n_groups: int = 4,\n silent: bool = False):\n seed(random_seed)\n # Generate some destinations\n destinations = [(random(), random()) for _ in range(n_destinations)]\n\n # Given a list of destination indexes, this is our cost function\n def evaluate(ordered_destinations: List[int]) -> float:\n total = 0\n for x, y in zip(ordered_destinations, ordered_destinations[1:]):\n coordinates_x = destinations[x]\n coordinates_y = destinations[y]\n total += sqrt((coordinates_x[0] - coordinates_y[1])**2 + (coordinates_x[1] - coordinates_y[1])**2)\n return total\n\n # This generates a random solution\n def generate_solution() -> List[int]:\n indexes = list(range(n_destinations))\n shuffle(indexes)\n return indexes\n\n def print_function(population: Population):\n if population.generation % 5000 == 0 and not silent:\n print(f'{population.generation}: {population.documented_best.fitness:1.2f} / '\n f'{population.current_best.fitness:1.2f}')\n\n pop = Population.generate(generate_solution, eval_function=evaluate, maximize=False,\n size=population_size * n_groups, concurrent_workers=concurrent_workers)\n\n island_evo = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random, combiner=cycle_crossover)\n .mutate(swap_elements, elitist=True))\n\n evo = (Evolution()\n .evaluate(lazy=True)\n .callback(print_function)\n .repeat(evolution=island_evo, n=100, grouping_function=group_stratified, n_groups=n_groups))\n\n result = pop.evolve(evolution=evo, n=n_iterations)\n\n if not silent:\n print(f'Shortest route: {result.documented_best.chromosome}')\n print(f'Route length: {result.documented_best.fitness}')\n\n\ndef parse_arguments():\n parser = ArgumentParser(description='Run the travelling salesman example.')\n parser.add_argument('--population-size', type=int, default=100,\n help='the number of candidates to start the algorithm with')\n parser.add_argument('--n-iterations', type=int, default=10,\n help='the number of iterations to run')\n parser.add_argument('--random-seed', type=int, default=42,\n help='the random seed to set')\n parser.add_argument('--n-destinations', type=int, default=50,\n help='Number of destinations in the route.')\n parser.add_argument('--n-groups', type=int, default=4,\n help='Number of groups to group by.')\n parser.add_argument('--concurrent-workers', type=int, default=None,\n help='Concurrent workers to use to evaluate the population.')\n return parser.parse_args()\n\n\nif __name__ == \"__main__\":\n args = parse_arguments()\n run_travelling_salesman(**args.__dict__)\n" }, { "path": "examples/very_basic_tsp.py", "content": "\"\"\"\nThere are a few things to notice with this example.\n\n1. from the command line you can re-run and see a different matplotlib plot\n2. `n_crossover` is set via the `.breed()` method\n3. the functions you need for this application can be tested with unittests\n\n\"\"\"\n\nimport random\nimport math\nimport argparse\n\nfrom evol import Population, Evolution\n\n\ndef run_evolutionary(num_towns=42, population_size=100, num_iter=200, seed=42): # noqa: C901\n \"\"\"\n Runs a simple evolutionary algorithm against a simple TSP problem.\n The goal is to explain the `evol` library, this is not an algorithm\n that should perform well.\n \"\"\"\n\n # First we generate random towns as candidates with a seed\n random.seed(seed)\n coordinates = [(random.random(), random.random()) for i in range(num_towns)]\n\n # Next we define a few functions that we will need in order to create an algorithm.\n # Think of these functions as if they are lego blocks.\n def init_func(num_towns):\n \"\"\"\n This function generates an individual\n \"\"\"\n order = list(range(num_towns))\n random.shuffle(order)\n return order\n\n def dist(t1, t2):\n \"\"\"\n Calculates the distance between two towns.\n \"\"\"\n return math.sqrt((t1[0] - t2[0]) ** 2 + (t1[1] - t2[1]) ** 2)\n\n def eval_func(order):\n \"\"\"\n Evaluates a candidate chromosome, which is a list that represents town orders.\n \"\"\"\n return sum([dist(coordinates[order[i]], coordinates[order[i - 1]]) for i, t in enumerate(order)])\n\n def pick_random(parents):\n \"\"\"\n This function selects two parents\n \"\"\"\n return random.choice(parents), random.choice(parents)\n\n def partition(lst, n_crossover):\n division = len(lst) / n_crossover\n return [lst[round(division * i):round(division * (i + 1))] for i in range(n_crossover)]\n\n def crossover_ox(mom_order, dad_order, n_crossover):\n idx_split = partition(range(len(mom_order)), n_crossover=n_crossover)\n dad_idx = sum([list(d) for i, d in enumerate(idx_split) if i % 2 == 0], [])\n path = [-1 for _ in range(len(mom_order))]\n for idx in dad_idx:\n path[idx] = dad_order[idx]\n cities_visited = {p for p in path if p != -1}\n for i, d in enumerate(path):\n if d == -1:\n city = [p for p in mom_order if p not in cities_visited][0]\n path[i] = city\n cities_visited.add(city)\n return path\n\n def random_flip(chromosome):\n result = chromosome[:]\n idx1, idx2 = random.choices(list(range(len(chromosome))), k=2)\n result[idx1], result[idx2] = result[idx2], result[idx1]\n return result\n\n pop = Population(chromosomes=[init_func(num_towns) for _ in range(population_size)],\n eval_function=eval_func, maximize=False, concurrent_workers=2).evaluate()\n\n evo = (Evolution()\n .survive(fraction=0.1)\n .breed(parent_picker=pick_random, combiner=crossover_ox, n_crossover=2)\n .mutate(random_flip)\n .evaluate())\n\n print(\"will start the evolutionary program\")\n scores = []\n iterations = []\n for i in range(num_iter):\n print(f\"iteration: {i} best score: {min([individual.fitness for individual in pop])}\")\n for indiviual in pop:\n scores.append(indiviual.fitness)\n iterations.append(i)\n pop = pop.evolve(evo)\n\n try:\n import matplotlib.pylab as plt\n plt.scatter(iterations, scores, s=1, alpha=0.3)\n plt.title(\"population fitness vs. iteration\")\n plt.show()\n except ImportError:\n print(\"If you install matplotlib you will get a pretty plot.\")\n\n\nif __name__ == \"__main__\":\n parser = argparse.ArgumentParser(description='Run an example evol algorithm against a simple TSP problem.')\n parser.add_argument('--num_towns', type=int, default=42,\n help='the number of towns to generate for the TSP problem')\n parser.add_argument('--population_size', type=int, default=100,\n help='the number of candidates to start the algorithm with')\n parser.add_argument('--num_iter', type=int, default=100,\n help='the number of evolutionary cycles to run')\n parser.add_argument('--seed', type=int, default=42,\n help='the random seed for all this')\n\n args = parser.parse_args()\n print(f\"i am aware of these arguments: {args}\")\n run_evolutionary(num_towns=args.num_towns, population_size=args.population_size,\n num_iter=args.num_iter, seed=args.seed)\n" }, { "path": "setup.cfg", "content": "[metadata]\ndescription-file=README.md\n\n[aliases]\ntest=pytest\n\n[flake8]\nmax-complexity=10\nmax-line-length=120\nexclude = */__init__.py, */*/__init__.py" }, { "path": "setup.py", "content": "import codecs\nfrom os import path\nfrom re import search, M\nfrom setuptools import setup, find_packages\n\n\ndef load_readme():\n this_directory = path.abspath(path.dirname(__file__))\n with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f:\n return f.read()\n\n\nhere = path.abspath(path.dirname(__file__))\n\n\ndef read(*parts):\n with codecs.open(path.join(here, *parts), 'r') as fp:\n return fp.read()\n\n\ndef find_version(*file_paths):\n version_file = read(*file_paths)\n version_match = search(r\"^__version__ = ['\\\"]([^'\\\"]*)['\\\"]\", version_file, M)\n if version_match:\n return version_match.group(1)\n raise RuntimeError(\"Unable to find version string.\")\n\n\nsetup(\n name='evol',\n version=find_version('evol', '__init__.py'),\n description='A Grammar for Evolutionary Algorithms and Heuristics',\n long_description=load_readme(),\n long_description_content_type='text/markdown',\n license='MIT License',\n author=['Vincent D. Warmerdam', 'Rogier van der Geer'],\n author_email='vincentwarmerdam@gmail.com',\n url='https://github.com/godatadriven/evol',\n packages=find_packages(),\n keywords=['genetic', 'algorithms', 'heuristics'],\n python_requires='>=3.6',\n tests_require=[\n \"pytest>=3.3.1\", \"attrs==19.1.0\", \"flake8>=3.7.9\"\n ],\n extras_require={\n \"dev\": [\"pytest>=3.3.1\", \"attrs==19.1.0\", \"flake8>=3.7.9\"],\n \"docs\": [\"sphinx_rtd_theme\", \"Sphinx>=2.0.0\"],\n },\n setup_requires=[\n \"pytest-runner\"\n ],\n install_requires=[\n \"multiprocess>=0.70.6.1\"\n ],\n classifiers=['Intended Audience :: Developers',\n 'Intended Audience :: Science/Research',\n 'Programming Language :: Python :: 3.6',\n 'Development Status :: 3 - Alpha',\n 'License :: OSI Approved :: MIT License',\n 'Topic :: Scientific/Engineering',\n 'Topic :: Scientific/Engineering :: Artificial Intelligence']\n)\n" }, { "path": "test_local.sh", "content": "flake8 evol\nflake8 tests\nif [ -x \"$(command -v py.test-3)\" ]; then\n py.test-3\nelse\n python -m pytest\nfi" }, { "path": "tests/conftest.py", "content": "from random import seed, shuffle\n\nfrom pytest import fixture\n\nfrom evol import Individual, Population, ContestPopulation, Evolution\nfrom evol.helpers.pickers import pick_random\n\n\n@fixture(scope='module')\ndef simple_chromosomes():\n return list(range(-50, 50))\n\n\n@fixture(scope='module')\ndef shuffled_chromosomes():\n chromosomes = list(range(0, 100)) + list(range(0, 100)) + list(range(0, 100)) + list(range(0, 100))\n seed(0)\n shuffle(chromosomes)\n return chromosomes\n\n\n@fixture(scope='function')\ndef simple_individuals(simple_chromosomes):\n result = [Individual(chromosome=chromosome) for chromosome in simple_chromosomes]\n for individual in result:\n individual.fitness = 0\n return result\n\n\n@fixture(scope='module')\ndef simple_evaluation_function():\n def eval_func(x):\n return -x ** 2\n return eval_func\n\n\n@fixture(scope='function')\ndef evaluated_individuals(simple_chromosomes, simple_evaluation_function):\n result = [Individual(chromosome=chromosome) for chromosome in simple_chromosomes]\n for individual in result:\n individual.fitness = individual.chromosome\n return result\n\n\n@fixture(scope='module')\ndef simple_contest_evaluation_function():\n def eval_func(x, y, z):\n return [1, -1, 0] if x > y else [-1, 1, 0]\n return eval_func\n\n\n@fixture(scope='module')\ndef simple_evolution():\n return (\n Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random, n_parents=2, combiner=lambda x, y: x + y)\n .mutate(lambda x: x + 1, probability=0.1)\n )\n\n\n@fixture(scope='function')\ndef simple_population(simple_chromosomes, simple_evaluation_function):\n return Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n\n\n@fixture(scope='function')\ndef simple_contestpopulation(simple_chromosomes, simple_contest_evaluation_function):\n return ContestPopulation(chromosomes=simple_chromosomes, eval_function=simple_contest_evaluation_function,\n contests_per_round=35, individuals_per_contest=3)\n\n\n@fixture(scope='function', params=range(2))\ndef any_population(request, simple_population, simple_contestpopulation):\n if request.param == 0:\n return simple_population\n elif request.param == 1:\n return simple_contestpopulation\n else:\n raise ValueError(\"invalid internal test config\")\n" }, { "path": "tests/helpers/combiners/test_permutation_combiners.py", "content": "from random import seed\n\nfrom evol.helpers.combiners.permutation import order_one_crossover, cycle_crossover\n\n\ndef test_order_one_crossover_int():\n seed(53) # Fix result of select_partition\n x, y = (8, 4, 7, 3, 6, 2, 5, 1, 9, 0), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)\n a = order_one_crossover(x, y)\n assert a == (0, 4, 7, 3, 6, 2, 5, 1, 8, 9)\n\n\ndef test_order_one_crossover_str():\n seed(53) # Fix result of select_partition\n x, y = ('I', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'J', 'A'), ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')\n a = order_one_crossover(x, y)\n assert a == ('A', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'I', 'J')\n\n\ndef test_cycle_crossover_int():\n x, y = (8, 4, 7, 3, 6, 2, 5, 1, 9, 0), (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)\n a, b = cycle_crossover(x, y)\n assert a == (8, 1, 2, 3, 4, 5, 6, 7, 9, 0) and b == (0, 4, 7, 3, 6, 2, 5, 1, 8, 9)\n\n\ndef test_cycle_crossover_str():\n x, y = ('I', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'J', 'A'), ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')\n a, b = cycle_crossover(x, y)\n assert a == ('I', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'J', 'A')\n assert b == ('A', 'E', 'H', 'D', 'G', 'C', 'F', 'B', 'I', 'J')\n" }, { "path": "tests/helpers/mutators/test_permutation_mutators.py", "content": "from random import seed\n\nfrom evol.helpers.mutators.permutation import inversion, swap_elements\n\n\ndef test_inversion_int():\n seed(53) # Fix result of select_partition\n x = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)\n a = inversion(x)\n assert a == (0, 1, 2, 7, 6, 5, 4, 3, 8, 9)\n\n\ndef test_inversion_str():\n seed(53) # Fix result of select_partition\n x = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')\n a = inversion(x)\n assert a == ('A', 'B', 'C', 'H', 'G', 'F', 'E', 'D', 'I', 'J')\n\n\ndef test_swap_elements_int():\n seed(0)\n x = (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)\n a = swap_elements(x)\n assert a == (0, 1, 2, 3, 4, 5, 9, 7, 8, 6)\n\n\ndef test_swap_elements_str():\n seed(0)\n x = ('A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J')\n a = swap_elements(x)\n assert a == ('A', 'B', 'C', 'D', 'E', 'F', 'J', 'H', 'I', 'G')\n" }, { "path": "tests/helpers/test_groups.py", "content": "from random import seed\n\nfrom pytest import mark, raises\n\nfrom evol import Population\nfrom evol.helpers.groups import group_duplicate, group_stratified, group_random\n\n\n@mark.parametrize('n_groups', [1, 2, 3, 4])\ndef test_group_duplicate(simple_individuals, n_groups):\n indexes = group_duplicate(simple_individuals, n_groups=n_groups)\n assert all(len(index) == len(set(index)) for index in indexes)\n assert all(len(index) == len(simple_individuals) for index in indexes)\n assert len(indexes) == n_groups\n\n\ndef test_group_random(simple_individuals):\n seed(42)\n indexes = group_random(simple_individuals, n_groups=4)\n assert sum(len(index) for index in indexes) == len(simple_individuals)\n seed(41)\n alt_indexes = group_random(simple_individuals, n_groups=4)\n assert indexes != alt_indexes\n\n\nclass TestGroupStratified:\n\n def test_unique(self, evaluated_individuals):\n indexes = group_stratified(evaluated_individuals, n_groups=2)\n assert set(index for island in indexes for index in island) == set(range(len(evaluated_individuals)))\n\n @mark.parametrize('n_groups', (2, 4))\n def test_is_stratified(self, shuffled_chromosomes, n_groups):\n population = Population(shuffled_chromosomes, eval_function=lambda x: x).evaluate()\n islands = population.group(group_stratified, n_groups=n_groups)\n # All islands should have the same total fitness\n assert len(set(sum(map(lambda i: i.fitness, island.individuals)) for island in islands)) == 1\n\n @mark.parametrize('n_groups', (3, 5))\n def test_is_nearly_stratified(self, shuffled_chromosomes, n_groups):\n population = Population(shuffled_chromosomes, eval_function=lambda x: x).evaluate()\n islands = population.group(group_stratified, n_groups=n_groups)\n # All islands should have roughly the same total fitness\n sum_fitnesses = [sum(map(lambda i: i.fitness, island.individuals)) for island in islands]\n assert max(sum_fitnesses) - min(sum_fitnesses) < n_groups * len(islands[0])\n\n def test_must_be_evaluated(self, simple_population):\n with raises(RuntimeError):\n simple_population.group(group_stratified)\n" }, { "path": "tests/helpers/test_helpers_utils.py", "content": "from evol.helpers.utils import rotating_window, sliding_window\n\n\ndef test_sliding_window():\n assert list(sliding_window([1, 2, 3, 4])) == [(1, 2), (2, 3), (3, 4)]\n\n\ndef test_rotating_window():\n assert list(rotating_window([1, 2, 3, 4])) == [(4, 1), (1, 2), (2, 3), (3, 4)]\n" }, { "path": "tests/problems/test_functions.py", "content": "from evol.problems.functions import Rosenbrock, Sphere, Rastrigin\n\n\ndef test_rosenbrock_optimality():\n problem = Rosenbrock(size=2)\n assert problem.eval_function((1, 1)) == 0.0\n problem = Rosenbrock(size=5)\n assert problem.eval_function((1, 1, 1, 1, 1)) == 0.0\n\n\ndef test_sphere_optimality():\n problem = Sphere(size=2)\n assert problem.eval_function((0, 0)) == 0.0\n problem = Sphere(size=5)\n assert problem.eval_function((0, 0, 0, 0, 0)) == 0.0\n\n\ndef test_rastrigin_optimality():\n problem = Rastrigin(size=2)\n assert problem.eval_function((0, 0)) == 0.0\n problem = Rastrigin(size=5)\n assert problem.eval_function((0, 0, 0, 0, 0)) == 0.0\n" }, { "path": "tests/problems/test_santa.py", "content": "import math\n\nimport pytest\n\nfrom evol.problems.routing import MagicSanta\n\n\n@pytest.fixture\ndef base_problem():\n return MagicSanta(city_coordinates=[(0, 1), (1, 0), (1, 1)],\n home_coordinate=(0, 0),\n gift_weight=[0, 0, 0])\n\n\n@pytest.fixture\ndef adv_problem():\n return MagicSanta(city_coordinates=[(0, 1), (1, 1), (0, 1)],\n home_coordinate=(0, 0),\n gift_weight=[5, 1, 1],\n sleigh_weight=2)\n\n\ndef test_error_raised_wrong_cities(base_problem):\n # we want an error if we see too many cities\n with pytest.raises(ValueError) as execinfo1:\n base_problem.eval_function([[0, 1, 2, 3]])\n assert \"Extra: {3}\" in str(execinfo1.value)\n # we want an error if we see too few cities\n with pytest.raises(ValueError) as execinfo2:\n base_problem.eval_function([[0, 2]])\n assert \"Missing: {1}\" in str(execinfo2.value)\n # we want an error if we see multiple occurences of cities\n with pytest.raises(ValueError) as execinfo3:\n base_problem.eval_function([[0, 2], [0, 1]])\n assert \"Multiple occurrences found for cities: {0}\" in str(execinfo3.value)\n\n\ndef test_base_score_method(base_problem):\n assert base_problem.distance((0, 0), (0, 2)) == 2\n expected = 1 + math.sqrt(2) + 1 + math.sqrt(2)\n assert base_problem.eval_function([[0, 1, 2]]) == pytest.approx(expected)\n assert base_problem.eval_function([[2, 1, 0]]) == pytest.approx(expected)\n base_problem.sleigh_weight = 2\n assert base_problem.eval_function([[2, 1, 0]]) == pytest.approx(2*expected)\n\n\ndef test_sleight_gift_weights(adv_problem):\n expected = (2+7) + (2+2) + (2+1) + (2+0)\n assert adv_problem.eval_function([[0, 1, 2]]) == pytest.approx(expected)\n\n\ndef test_multiple_routes(adv_problem):\n expected = (2 + 6) + (2 + 1) + math.sqrt(2)*(2 + 0) + (2 + 1) + (2 + 0)\n assert adv_problem.eval_function([[0, 1], [2]]) == pytest.approx(expected)\n" }, { "path": "tests/problems/test_tsp.py", "content": "import math\nimport pytest\nfrom evol.problems.routing import TSPProblem\nfrom evol.problems.routing.coordinates import united_states_capitols\n\n\ndef test_distance_func():\n problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])\n assert problem.distance_matrix[0][0] == 0\n assert problem.distance_matrix[1][0] == 1\n assert problem.distance_matrix[0][1] == 1\n assert problem.distance_matrix[1][1] == 0\n assert problem.distance_matrix[0][2] == 1\n assert problem.distance_matrix[0][3] == pytest.approx(math.sqrt(2))\n\n\ndef test_score_method():\n problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])\n expected = 1 + math.sqrt(2) + 1 + math.sqrt(2)\n assert problem.eval_function([0, 1, 2, 3]) == pytest.approx(expected)\n\n\ndef test_score_method_can_error():\n problem = TSPProblem.from_coordinates([(0, 0), (0, 1), (1, 0), (1, 1)])\n\n with pytest.raises(ValueError) as execinfo1:\n problem.eval_function([0, 1, 2, 3, 4])\n assert \"Solution is longer than number of towns\" in str(execinfo1.value)\n\n with pytest.raises(ValueError) as execinfo2:\n problem.eval_function([0, 1, 2])\n assert \"3\" in str(execinfo2.value)\n assert \"missing\" in str(execinfo2.value)\n\n with pytest.raises(ValueError) as execinfo3:\n problem.eval_function([0, 1, 2, 2])\n assert \"3\" in str(execinfo3.value)\n assert \"missing\" in str(execinfo3.value)\n\n\ndef test_can_initialize_our_datasets():\n problem = TSPProblem.from_coordinates(united_states_capitols)\n for i in range(len(united_states_capitols)):\n assert problem.distance_matrix[i][i] == 0\n" }, { "path": "tests/test_callback.py", "content": "from evol import Population, Evolution\nfrom evol.exceptions import StopEvolution\n\n\nclass PopCounter:\n def __init__(self):\n self.count = 0\n self.sum = 0\n\n def add(self, pop):\n for i in pop:\n self.count += 1\n self.sum += i.chromosome\n\n\nclass TestPopulationSimple:\n def test_simple_counter_works(self, simple_chromosomes, simple_evaluation_function):\n counter = PopCounter()\n\n pop = Population(chromosomes=simple_chromosomes,\n eval_function=simple_evaluation_function)\n evo = (Evolution()\n .mutate(lambda x: x)\n .callback(lambda p: counter.add(p)))\n\n pop.evolve(evolution=evo, n=1)\n assert counter.count == len(simple_chromosomes)\n assert counter.sum == sum(simple_chromosomes)\n pop.evolve(evolution=evo, n=10)\n assert counter.count == len(simple_chromosomes) * 11\n assert counter.sum == sum(simple_chromosomes) * 11\n\n def test_simple_counter_works_every(self, simple_chromosomes, simple_evaluation_function):\n counter = PopCounter()\n\n pop = Population(chromosomes=simple_chromosomes,\n eval_function=simple_evaluation_function)\n evo = (Evolution()\n .mutate(lambda x: x)\n .callback(lambda p: counter.add(p), every=2))\n\n pop.evolve(evolution=evo, n=10)\n assert counter.count == len(simple_chromosomes) * 5\n assert counter.sum == sum(simple_chromosomes) * 5\n\n def test_is_evaluated(self, simple_population):\n def assert_is_evaluated(pop: Population):\n assert pop.current_best is not None\n\n simple_population.evaluate(lazy=True)\n simple_population.callback(assert_is_evaluated)\n\n def test_stop(self, simple_population, simple_evolution):\n\n def func(pop):\n if pop.generation == 5:\n raise StopEvolution\n\n evo = simple_evolution.callback(func)\n assert simple_population.evolve(evo, n=10).generation == 5\n" }, { "path": "tests/test_conditions.py", "content": "from time import monotonic, sleep\n\nfrom pytest import raises\n\nfrom evol import Population\nfrom evol.conditions import Condition, MinimumProgress, TimeLimit\nfrom evol.exceptions import StopEvolution\n\n\nclass TestCondition:\n\n def test_context(self):\n with Condition(lambda pop: False):\n assert len(Condition.conditions) == 1\n with TimeLimit(60):\n assert len(Condition.conditions) == 2\n assert len(Condition.conditions) == 0\n\n def test_check(self, simple_population):\n with Condition(lambda pop: False):\n with raises(StopEvolution):\n Condition.check(simple_population)\n Condition.check(simple_population)\n\n def test_evolve(self, simple_population, simple_evolution):\n with Condition(lambda pop: pop.generation < 3):\n result = simple_population.evolve(simple_evolution, n=5)\n assert result.generation == 3\n\n def test_sequential(self, simple_population, simple_evolution):\n with Condition(lambda pop: pop.generation < 3):\n result = simple_population.evolve(simple_evolution, n=10)\n assert result.generation == 3\n with Condition(lambda pop: pop.generation < 6):\n result = result.evolve(simple_evolution, n=10)\n assert result.generation == 6\n\n\nclass TestMinimumProgress:\n\n def test_evolve(self, simple_evolution):\n # The initial population contains the optimal solution.\n pop = Population(list(range(100)), eval_function=lambda x: x**2, maximize=False)\n with MinimumProgress(window=10):\n pop = pop.evolve(simple_evolution, n=100)\n assert pop.generation == 10\n\n\nclass TestTimeLimit:\n\n def test_evolve(self, simple_population, simple_evolution):\n evo = simple_evolution.callback(lambda p: sleep(1))\n start_time = monotonic()\n with TimeLimit(seconds=2):\n pop = simple_population.evolve(evo, n=10)\n assert monotonic() - start_time > 1\n assert pop.generation == 2\n" }, { "path": "tests/test_evolution.py", "content": "from pytest import mark\n\nfrom evol import Evolution, Population\nfrom evol.helpers.groups import group_random, group_duplicate, group_stratified\nfrom evol.helpers.pickers import pick_random\n\n\nclass TestEvolution:\n\n def test_add_step(self):\n evo = Evolution()\n assert len(evo.chain) == 0\n evo_step = evo._add_step('step')\n assert len(evo.chain) == 0 # original unchanged\n assert evo_step.chain == ['step'] # copy with extra step\n\n def test_repr(self):\n assert repr(Evolution()) == 'Evolution()'\n assert repr(Evolution().evaluate()) == 'Evolution(\\n EvaluationStep())'\n r = 'Evolution(\\n RepeatStep() with evolution (10x):\\n Evolution(\\n' \\\n ' EvaluationStep()\\n SurviveStep()))'\n assert repr(Evolution().repeat(Evolution().survive(fraction=0.9), n=10)) == r\n\n\nclass TestPopulationEvolve:\n\n def test_repeat_step(self):\n pop = Population([0 for i in range(100)], lambda x: x)\n evo = Evolution().repeat(Evolution().survive(fraction=0.9), n=10)\n # Check whether an Evolution inside another Evolution is actually applied\n assert len(pop.evolve(evo, n=2)) < 50\n\n @mark.parametrize('n_groups', [2, 4, 5])\n @mark.parametrize('grouping_function', [group_stratified, group_duplicate, group_random])\n def test_repeat_step_grouped(self, n_groups, grouping_function):\n calls = []\n\n def callback(pop):\n calls.append(len(pop))\n\n sub_evo = (\n Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda x, y: x + y)\n .callback(callback_function=callback)\n )\n\n pop = Population([0 for _ in range(100)], lambda x: x)\n evo = (\n Evolution()\n .evaluate(lazy=True)\n .repeat(sub_evo, grouping_function=grouping_function, n_groups=n_groups)\n )\n assert len(pop.evolve(evo, n=2)) == 100\n assert len(calls) == 2 * n_groups\n" }, { "path": "tests/test_examples.py", "content": "from pytest import mark\nimport sys\n\nsys.path.append('.')\n\nfrom examples.number_of_parents import run_evolutionary # noqa: E402\nfrom examples.rock_paper_scissors import run_rock_paper_scissors # noqa: E402\nfrom examples.travelling_salesman import run_travelling_salesman # noqa: E402\n\n\nN_WORKERS = [1, 2, None]\n\n\n@mark.parametrize('concurrent_workers', N_WORKERS)\ndef test_number_of_parents(concurrent_workers):\n run_evolutionary(verbose=False, workers=concurrent_workers)\n\n\n@mark.parametrize('grouped', (False, True))\ndef test_rock_paper_scissors(grouped):\n history = run_rock_paper_scissors(silent=True, n_iterations=16, grouped=grouped)\n assert len(set(h['generation'] for h in history.history)) == 16\n\n\n@mark.skipif(sys.version_info < (3, 7), reason='PyTest cannot deal with the multiprocessing in 3.6.')\n@mark.parametrize('n_groups', (1, 4))\ndef test_travelling_salesman(n_groups):\n run_travelling_salesman(concurrent_workers=2, n_groups=n_groups, n_iterations=4, silent=True)\n" }, { "path": "tests/test_individual.py", "content": "from copy import copy\n\nfrom evol import Individual\n\n\nclass TestIndividual:\n\n def test_init(self):\n chromosome = (3, 4)\n ind = Individual(chromosome=chromosome)\n assert chromosome == ind.chromosome\n assert ind.fitness is None\n\n def test_copy(self):\n individual = Individual(chromosome=(1, 2))\n individual.evaluate(sum)\n copied_individual = copy(individual)\n assert individual.chromosome == copied_individual.chromosome\n assert individual.fitness == copied_individual.fitness\n assert individual.id == copied_individual.id\n copied_individual.mutate(lambda x: (2, 3))\n assert individual.chromosome == (1, 2)\n\n def test_evaluate(self):\n ind = Individual(chromosome=(1, 2))\n ind.evaluate(sum)\n\n def eval_func1(chromosome):\n raise RuntimeError\n\n ind.evaluate(eval_function=eval_func1, lazy=True)\n assert ind.fitness == 3\n\n def eval_func2(chromosome):\n return 5\n\n ind.evaluate(eval_function=eval_func2, lazy=False)\n assert ind.fitness == 5\n\n def test_mutate(self):\n ind = Individual(chromosome=(1, 2, 3))\n ind.evaluate(sum)\n\n def mutate_func(chromosome, value):\n return chromosome[0], value, chromosome[2]\n\n ind.mutate(mutate_func, value=5)\n assert (1, 5, 3) == ind.chromosome\n assert ind.fitness is None\n" }, { "path": "tests/test_logging.py", "content": "import random\n\nfrom evol import Population, Evolution\nfrom evol.helpers.pickers import pick_random\nfrom evol.logger import BaseLogger, SummaryLogger\n\n\nclass TestLoggerSimple:\n\n def test_baselogger_write_file_no_stdout(self, tmpdir, capsys, simple_chromosomes, simple_evaluation_function):\n log_file = tmpdir.join('log.txt')\n logger = BaseLogger(target=log_file, stdout=False)\n pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n # we should see that a file was created with an appropriate number of rows\n pop.evaluate()\n logger.log(pop)\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == len(simple_chromosomes)\n # we should see that a file was created with an appropriate number of rows\n logger.log(pop)\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == (2 * len(simple_chromosomes))\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n # there should be nothing printed\n assert len(read_stdout) == 0\n\n def test_baselogger_can_write_to_stdout(self, capsys, simple_chromosomes):\n logger = BaseLogger(target=None, stdout=True)\n\n pop = Population(chromosomes=simple_chromosomes,\n eval_function=lambda x: x)\n pop.evaluate()\n logger.log(pop)\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == len(pop)\n\n def test_baselogger_can_accept_kwargs(self, tmpdir, simple_chromosomes, simple_evaluation_function):\n log_file = tmpdir.join('log.txt')\n logger = BaseLogger(target=log_file, stdout=False)\n pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop.evaluate()\n # we should see that a file was created with an appropriate number of rows\n logger.log(pop, foo=\"bar\")\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == len(simple_chromosomes)\n assert all([\"bar\" in line for line in f.readlines()])\n # we should see that a file was created with an appropriate number of rows\n logger.log(pop, foo=\"meh\")\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == (2 * len(simple_chromosomes))\n assert all(['meh' in line for line in f.readlines()[-10:]])\n\n def test_baselogger_works_via_evolution_callback(self, tmpdir, capsys):\n log_file = tmpdir.join('log.txt')\n logger = BaseLogger(target=log_file, stdout=True)\n pop = Population(chromosomes=range(10), eval_function=lambda x: x)\n evo = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda mom, dad: (mom + dad) / 2 + (random.random() - 0.5),\n n_parents=2)\n .callback(logger.log, foo='bar'))\n pop.evolve(evolution=evo, n=2)\n # check characteristics of the file\n with open(log_file, \"r\") as f:\n read_file = [item.replace(\"\\n\", \"\") for item in f.readlines()]\n # size of the log should be appropriate\n assert len(read_file) == 2 * len(pop)\n # bar needs to be in every single line\n assert all(['bar' in row for row in read_file])\n # check characteristics of stoud\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == 2 * len(pop)\n assert all(['bar' in row for row in read_stdout])\n\n def test_summarylogger_write_file_mo_stdout(self, tmpdir, capsys):\n log_file = tmpdir.join('log.txt')\n logger = SummaryLogger(target=log_file, stdout=False)\n pop = Population(chromosomes=range(10), eval_function=lambda x: x)\n # we should see that a file was created with an appropriate number of rows\n pop.evaluate()\n logger.log(pop)\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == 1\n # we should see that a file was created with an appropriate number of rows\n logger.log(pop)\n with open(log_file, \"r\") as f:\n assert len(f.readlines()) == 2\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n # there should be nothing printed\n assert len(read_stdout) == 0\n\n def test_summarylogger_can_write_to_stdout(self, capsys, simple_chromosomes, simple_evaluation_function):\n logger = SummaryLogger(target=None, stdout=True)\n pop = Population(chromosomes=range(10),\n eval_function=lambda x: x)\n pop.evaluate()\n logger.log(pop)\n logger.log(pop)\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == 2\n\n def test_summary_logger_can_accept_kwargs(self, tmpdir, simple_chromosomes, simple_evaluation_function):\n log_file = tmpdir.join('log.txt')\n logger = SummaryLogger(target=log_file, stdout=False)\n pop = Population(chromosomes=simple_chromosomes,\n eval_function=simple_evaluation_function)\n pop.evaluate()\n # lets make a first simple log\n logger.log(pop, foo=\"bar\", buzz=\"meh\")\n with open(log_file, \"r\") as f:\n read_lines = f.readlines()\n assert len(read_lines) == 1\n first_line = read_lines[0]\n assert \"bar\" in first_line\n assert \"meh\" in first_line\n last_entry = first_line.split(\",\")\n assert len(last_entry) == 6\n # lets log another one\n logger.log(pop, buzz=\"moh\")\n with open(log_file, \"r\") as f:\n read_lines = f.readlines()\n assert len(read_lines) == 2\n first_line = read_lines[-1]\n assert \"moh\" in first_line\n last_entry = first_line.split(\",\")\n assert len(last_entry) == 5\n\n def test_summarylogger_works_via_evolution(self, tmpdir, capsys):\n log_file = tmpdir.join('log.txt')\n logger = SummaryLogger(target=log_file, stdout=True)\n pop = Population(chromosomes=list(range(10)), eval_function=lambda x: x)\n evo = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda mom, dad: (mom + dad) / 2 + (random.random() - 0.5),\n n_parents=2)\n .evaluate()\n .callback(logger.log, foo='bar'))\n pop.evolve(evolution=evo, n=5)\n # check characteristics of the file\n with open(log_file, \"r\") as f:\n read_file = [item.replace(\"\\n\", \"\") for item in f.readlines()]\n # size of the log should be appropriate\n assert len(read_file) == 5\n # bar needs to be in every single line\n assert all(['bar' in row for row in read_file])\n # check characteristics of stoud\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == 5\n assert all(['bar' in row for row in read_stdout])\n\n def test_two_populations_can_use_same_logger(self, tmpdir, capsys):\n log_file = tmpdir.join('log.txt')\n logger = SummaryLogger(target=log_file, stdout=True)\n pop1 = Population(chromosomes=list(range(10)), eval_function=lambda x: x)\n pop2 = Population(chromosomes=list(range(10)), eval_function=lambda x: x)\n evo = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda mom, dad: (mom + dad) + 1,\n n_parents=2)\n .evaluate()\n .callback(logger.log, foo=\"dino\"))\n pop1.evolve(evolution=evo, n=5)\n pop2.evolve(evolution=evo, n=5)\n # two evolutions have now been applied, lets check the output!\n with open(log_file, \"r\") as f:\n read_file = [item.replace(\"\\n\", \"\") for item in f.readlines()]\n # print(read_file)\n # size of the log should be appropriate\n assert len(read_file) == 10\n # dino needs to be in every single line\n assert all(['dino' in row for row in read_file])\n # check characteristics of stoud\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == 10\n assert all(['dino' in row for row in read_stdout])\n\n def test_every_mechanic_in_evolution_log(self, tmpdir, capsys):\n log_file = tmpdir.join('log.txt')\n logger = SummaryLogger(target=log_file, stdout=True)\n pop = Population(chromosomes=list(range(10)), eval_function=lambda x: x)\n evo = (Evolution()\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda mom, dad: (mom + dad) + 1,\n n_parents=2)\n .evaluate()\n .callback(logger.log, every=2))\n pop.evolve(evolution=evo, n=100)\n with open(log_file, \"r\") as f:\n read_file = [item.replace(\"\\n\", \"\") for item in f.readlines()]\n assert len(read_file) == 50\n # check characteristics of stoud\n read_stdout = [line for line in capsys.readouterr().out.split('\\n') if line != '']\n assert len(read_stdout) == 50\n" }, { "path": "tests/test_parallel_population.py", "content": "" }, { "path": "tests/test_population.py", "content": "from time import sleep, time\n\nimport os\nfrom copy import copy\nfrom pytest import raises, mark\nfrom random import random, choices, seed\n\nfrom evol import Population, ContestPopulation\nfrom evol.helpers.groups import group_duplicate, group_stratified\nfrom evol.helpers.pickers import pick_random\nfrom evol.population import Contest\n\n\nclass TestPopulationSimple:\n\n def test_filter_works(self, simple_chromosomes, simple_evaluation_function):\n pop = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n assert len(pop.filter(func=lambda i: random() > 0.5)) < 200\n\n def test_population_init(self, simple_chromosomes):\n pop = Population(simple_chromosomes, eval_function=lambda x: x)\n assert len(pop) == len(simple_chromosomes)\n assert pop.intended_size == len(pop)\n\n def test_population_generate(self, simple_evaluation_function):\n def init_func():\n return 1\n\n pop = Population.generate(init_function=init_func, eval_function=simple_evaluation_function, size=200)\n assert len(pop) == 200\n assert pop.intended_size == 200\n assert pop.individuals[0].chromosome == 1\n\n def test_is_evaluated(self, any_population):\n assert not any_population.is_evaluated\n assert any_population.evaluate().is_evaluated\n\n\nclass TestPopulationCopy:\n\n def test_population_copy(self, any_population):\n copied_population = copy(any_population)\n for key in any_population.__dict__.keys():\n if key not in ('id', 'individuals'):\n assert copied_population.__dict__[key] == any_population.__dict__[key]\n\n def test_population_is_evaluated(self, any_population):\n evaluated_population = any_population.evaluate()\n copied_population = copy(evaluated_population)\n assert evaluated_population.is_evaluated\n assert copied_population.is_evaluated\n\n\nclass TestPopulationEvaluate:\n\n cpus = os.cpu_count()\n latency = 0.005\n\n def test_individuals_are_not_initially_evaluated(self, any_population):\n assert all([i.fitness is None for i in any_population])\n\n def test_evaluate_lambda(self, simple_chromosomes):\n # without concurrency (note that I'm abusing a boolean operator to introduce some latency)\n pop = Population(simple_chromosomes, eval_function=lambda x: (sleep(self.latency) or x))\n t0 = time()\n pop.evaluate()\n t1 = time()\n single_proc_time = t1 - t0\n for individual in pop:\n assert individual.chromosome == individual.fitness\n # with concurrency\n pop = Population(simple_chromosomes, eval_function=lambda x: (sleep(self.latency) or x),\n concurrent_workers=self.cpus)\n t0 = time()\n pop.evaluate()\n t1 = time()\n multi_proc_time = t1 - t0\n for individual in pop:\n assert individual.chromosome == individual.fitness\n if self.cpus > 1:\n assert multi_proc_time < single_proc_time\n\n def test_evaluate_func(self, simple_chromosomes):\n def evaluation_function(x):\n sleep(self.latency)\n return x * x\n pop = Population(simple_chromosomes, eval_function=evaluation_function)\n t0 = time()\n pop.evaluate()\n t1 = time()\n single_proc_time = t1 - t0\n for individual in pop:\n assert evaluation_function(individual.chromosome) == individual.fitness\n # with concurrency\n pop = Population(simple_chromosomes, eval_function=evaluation_function, concurrent_workers=self.cpus)\n t0 = time()\n pop.evaluate()\n t1 = time()\n multi_proc_time = t1 - t0\n for individual in pop:\n assert evaluation_function(individual.chromosome) == individual.fitness\n if self.cpus > 1:\n assert multi_proc_time < single_proc_time\n\n def test_evaluate_lazy(self, any_population):\n pop = any_population\n pop.evaluate(lazy=True) # should evaluate\n\n def raise_function(_):\n raise Exception\n\n pop.eval_function = raise_function\n pop.evaluate(lazy=True) # should not evaluate\n with raises(Exception):\n pop.evaluate(lazy=False)\n\n\nclass TestPopulationSurvive:\n\n def test_survive_n_works(self, simple_chromosomes, simple_evaluation_function):\n pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop3 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n assert len(pop1) == len(simple_chromosomes)\n assert len(pop2.survive(n=50)) == 50\n assert len(pop3.survive(n=75, luck=True)) == 75\n\n def test_survive_p_works(self, simple_chromosomes, simple_evaluation_function):\n pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop3 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n assert len(pop1) == len(simple_chromosomes)\n assert len(pop2.survive(fraction=0.5)) == len(simple_chromosomes) * 0.5\n assert len(pop3.survive(fraction=0.1, luck=True)) == len(simple_chromosomes) * 0.1\n\n def test_survive_n_and_p_works(self, simple_evaluation_function):\n chromosomes = list(range(200))\n pop1 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)\n pop2 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)\n pop3 = Population(chromosomes=chromosomes, eval_function=simple_evaluation_function)\n assert len(pop1.survive(fraction=0.5, n=200)) == 100\n assert len(pop2.survive(fraction=0.9, n=10)) == 10\n assert len(pop3.survive(fraction=0.5, n=190, luck=True)) == 100\n\n def test_breed_increases_generation(self, any_population):\n assert any_population.breed(parent_picker=pick_random, combiner=lambda mom, dad: mom).generation == 1\n\n def test_survive_throws_correct_errors(self, any_population):\n \"\"\"If the resulting population is zero or larger than initial we need to see errors.\"\"\"\n with raises(RuntimeError):\n any_population.survive(n=0)\n with raises(ValueError):\n any_population.survive(n=250)\n with raises(ValueError):\n any_population.survive()\n\n\nclass TestPopulationBreed:\n\n def test_breed_amount_works(self, simple_chromosomes, simple_evaluation_function):\n pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop1.survive(n=50).breed(parent_picker=lambda population: choices(population, k=2),\n combiner=lambda mom, dad: (mom + dad) / 2)\n assert len(pop1) == len(simple_chromosomes)\n pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop2.survive(n=50).breed(parent_picker=lambda population: choices(population, k=2),\n combiner=lambda mom, dad: (mom + dad) / 2, population_size=400)\n assert len(pop2) == 400\n assert pop2.intended_size == 400\n assert pop1.generation == 1\n assert pop2.generation == 1\n\n def test_breed_works_with_kwargs(self, simple_chromosomes, simple_evaluation_function):\n pop1 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop1.survive(n=50).breed(parent_picker=pick_random,\n combiner=lambda mom, dad: (mom + dad) / 2,\n n_parents=2)\n assert len(pop1) == len(simple_chromosomes)\n pop2 = Population(chromosomes=simple_chromosomes, eval_function=simple_evaluation_function)\n pop2.survive(n=50).breed(parent_picker=pick_random,\n combiner=lambda *parents: sum(parents)/len(parents),\n population_size=400, n_parents=3)\n assert len(pop2) == 400\n assert pop2.intended_size == 400\n\n def test_breed_raises_with_multiple_values_for_kwarg(self, simple_population):\n\n (simple_population\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda x, y: x + y))\n\n with raises(TypeError):\n (simple_population\n .survive(fraction=0.5)\n .breed(parent_picker=pick_random,\n combiner=lambda x, y: x + y, y=2))\n\n\nclass TestPopulationMutate:\n\n def test_mutate_lambda(self):\n pop = Population([1]*100, eval_function=lambda x: x).mutate(lambda x: x+1)\n for chromosome in pop.chromosomes:\n assert chromosome == 2\n assert len(pop) == 100\n\n def test_mutate_inplace(self):\n pop = Population([1]*100, eval_function=lambda x: x)\n pop.mutate(lambda x: x+1)\n for chromosome in pop.chromosomes:\n assert chromosome == 2\n\n def test_mutate_func(self):\n def mutate_func(x):\n return -x\n population = Population([1]*100, eval_function=lambda x: x)\n population.mutate(mutate_func)\n for chromosome in population.chromosomes:\n assert chromosome == -1\n assert len(population) == 100\n\n def test_mutate_probability(self):\n seed(0)\n pop = Population([1]*100, eval_function=lambda x: x).mutate(lambda x: x+1, probability=0.5).evaluate()\n assert min(individual.chromosome for individual in pop.individuals) == 1\n assert max(individual.chromosome for individual in pop.individuals) == 2\n assert pop.current_best.fitness == 2\n assert pop.documented_best.fitness == 2\n assert len(pop) == 100\n\n def test_mutate_zero_probability(self):\n pop = Population([1]*100, eval_function=lambda x: x).mutate(lambda x: x+1, probability=0)\n for chromosome in pop.chromosomes:\n assert chromosome == 1\n\n def test_mutate_func_kwargs(self):\n def mutate_func(x, y=0):\n return x+y\n pop = Population([1]*100, eval_function=lambda x: x).mutate(mutate_func, y=16)\n for chromosome in pop.chromosomes:\n assert chromosome == 17\n\n def test_mutate_elitist(self):\n pop = Population([1, 1, 3], eval_function=lambda x: x).evaluate().mutate(lambda x: x + 1, elitist=True)\n for chromosome in pop.chromosomes:\n assert 1 < chromosome <= 3\n assert len(pop) == 3\n\n\nclass TestPopulationWeights:\n\n def test_weights(self, simple_chromosomes, simple_evaluation_function):\n for maximize in (False, True):\n pop = Population(chromosomes=simple_chromosomes,\n eval_function=simple_evaluation_function, maximize=maximize)\n with raises(RuntimeError):\n assert min(pop._individual_weights) >= 0\n pop.evaluate()\n assert max(pop._individual_weights) == 1\n assert min(pop._individual_weights) == 0\n if maximize:\n assert pop._individual_weights[0] == 0\n else:\n assert pop._individual_weights[0] == 1\n\n\nclass TestPopulationBest:\n\n def test_current_best(self, simple_chromosomes):\n for maximize, best in ((True, max(simple_chromosomes)), (False, min(simple_chromosomes))):\n pop = Population(chromosomes=simple_chromosomes, eval_function=float, maximize=maximize)\n assert pop.current_best is None\n pop.evaluate()\n assert pop.current_best.chromosome == best\n\n def test_current_worst(self, simple_chromosomes):\n for maximize, worst in ((False, max(simple_chromosomes)), (True, min(simple_chromosomes))):\n pop = Population(chromosomes=simple_chromosomes, eval_function=float, maximize=maximize)\n assert pop.current_worst is None\n pop.evaluate()\n assert pop.current_worst.chromosome == worst\n\n def test_mutate_resets(self):\n pop = Population(chromosomes=[1, 1, 1], eval_function=float, maximize=True)\n assert pop.current_best is None and pop.current_worst is None\n pop.evaluate()\n assert pop.current_best.fitness == 1 and pop.current_worst.fitness == 1\n pop.mutate(lambda x: x)\n assert pop.current_best is None and pop.current_worst is None\n\n def test_documented_best(self):\n pop = Population(chromosomes=[100, 100, 100], eval_function=lambda x: x*2, maximize=True)\n assert pop.documented_best is None\n pop.evaluate()\n assert pop.documented_best.fitness == pop.current_best.fitness\n pop.mutate(mutate_function=lambda x: x - 10, probability=1).evaluate()\n assert pop.documented_best.fitness - 20 == pop.current_best.fitness\n\n\nclass TestPopulationIslands:\n\n @mark.parametrize('n_groups', [1, 2, 3, 4])\n def test_groups(self, simple_population, n_groups):\n groups = simple_population.group(group_duplicate, n_groups=n_groups)\n assert len(groups) == n_groups\n assert type(groups) == list\n assert all(type(group) is Population for group in groups)\n\n def test_no_groups(self, simple_population):\n with raises(ValueError):\n simple_population.group(group_duplicate, n_groups=0)\n\n def test_empty_group(self, simple_population):\n def rogue_grouping_function(*args):\n return [[1, 2, 3], []]\n\n with raises(ValueError):\n simple_population.group(rogue_grouping_function)\n\n @mark.parametrize('result, error', [\n (['a', 'b', 'c'], TypeError),\n ([None, None], TypeError),\n ([10, 100, 1000], IndexError)\n ])\n def test_invalid_group(self, simple_population, result, error):\n def rogue_grouping_function(*args):\n return [result]\n\n with raises(error):\n simple_population.group(rogue_grouping_function)\n\n def test_not_evaluated(self, simple_population):\n with raises(RuntimeError):\n simple_population.group(group_stratified, n_groups=3)\n\n def test_combine(self, simple_population):\n groups = simple_population.evaluate().group(group_stratified, n_groups=3)\n combined = Population.combine(*groups)\n assert combined.intended_size == simple_population.intended_size\n\n def test_combine_nothing(self):\n with raises(ValueError):\n Population.combine()\n\n\nclass TestContest:\n\n def test_assign_score(self, simple_individuals):\n contest = Contest(simple_individuals)\n contest.assign_scores(range(len(simple_individuals)))\n for score, individual in zip(range(len(simple_individuals)), simple_individuals):\n assert individual.fitness == score\n\n @mark.parametrize('individuals_per_contest,contests_per_round', [(2, 1), (5, 1), (7, 1), (2, 5), (5, 4), (3, 3)])\n def test_generate_n_contests(self, simple_individuals, individuals_per_contest, contests_per_round):\n contests = Contest.generate(simple_individuals, contests_per_round=contests_per_round,\n individuals_per_contest=individuals_per_contest)\n for contest in contests:\n contest.assign_scores([1]*individuals_per_contest) # Now the fitness equals the number of contests played\n # All individuals competed in the same number of contests\n assert len({individual.fitness for individual in simple_individuals}) == 1\n # The number of contests is _at least_ contests_per_round\n assert all([individual.fitness >= contests_per_round for individual in simple_individuals])\n # The number of contests is smaller than contests_per_round + individuals_per_contest\n assert all([individual.fitness < contests_per_round + individuals_per_contest\n for individual in simple_individuals])\n\n\nclass TestContestPopulation:\n\n def test_init(self):\n cp = ContestPopulation([0, 1, 2], lambda x: x, contests_per_round=15, individuals_per_contest=15)\n assert cp.contests_per_round == 15\n assert cp.individuals_per_contest == 15\n\n\nclass TestContestPopulationBest:\n\n def test_no_documented(self):\n pop = ContestPopulation([0, 1, 2], lambda x, y: [0, 0], contests_per_round=100, individuals_per_contest=2)\n pop.evaluate()\n assert pop.documented_best is None\n # with concurrency\n pop = ContestPopulation([0, 1, 2], lambda x, y: [0, 0], contests_per_round=100, individuals_per_contest=2,\n concurrent_workers=3)\n pop.evaluate()\n assert pop.documented_best is None\n pop = ContestPopulation([0, 1, 2],\n lambda x, y: [x, y],\n contests_per_round=100, individuals_per_contest=2)\n pop.evaluate()\n assert pop.documented_best is None\n # with concurrency\n pop = ContestPopulation([0, 1, 2],\n lambda x, y: [x, y],\n contests_per_round=100, individuals_per_contest=2,\n concurrent_workers=3)\n pop.evaluate()\n assert pop.documented_best is None\n" }, { "path": "tests/test_serialization.py", "content": "from os import listdir\nfrom pytest import raises\n\nfrom evol import Population, Evolution\nfrom evol.serialization import SimpleSerializer\n\n\nclass TestPickleCheckpoint:\n method = 'pickle'\n extension = '.pkl'\n exception = AttributeError\n\n def test_checkpoint(self, tmpdir, simple_population):\n directory = tmpdir.mkdir(\"ckpt\")\n simple_population.checkpoint(target=directory, method=self.method)\n assert len(listdir(directory)) == 1\n assert listdir(directory)[0].endswith(self.extension)\n\n def test_unserializable_chromosome(self, tmpdir):\n directory = tmpdir.mkdir(\"ckpt\")\n\n class UnSerializableChromosome:\n def __init__(self, x):\n self.x = x\n\n pop = Population([UnSerializableChromosome(i) for i in range(10)], lambda x: x.x)\n with raises(self.exception):\n pop.checkpoint(target=directory, method=self.method)\n\n def test_load(self, tmpdir, simple_population):\n directory = tmpdir.mkdir(\"ckpt\")\n simple_population.checkpoint(target=directory, method=self.method)\n pop = Population.load(directory, lambda x: x['x'])\n assert len(simple_population) == len(pop)\n assert all(x.__dict__ == y.__dict__ for x, y in zip(simple_population, pop))\n\n def test_load_invalid_target(self, tmpdir):\n directory = tmpdir.mkdir('ckpt')\n with raises(FileNotFoundError):\n Population.load(directory.join('no_file' + self.extension), lambda x: x)\n with raises(FileNotFoundError):\n Population.load(directory, lambda x: x)\n txt_file = directory.join('file.txt')\n txt_file.write('Something')\n with raises(ValueError):\n Population.load(txt_file.strpath, lambda x: x)\n\n def test_checkpoint_invalid_target(self, tmpdir, simple_population):\n directory = tmpdir.mkdir(\"ckpt\")\n with raises(ValueError):\n simple_population.checkpoint(target=None, method=self.method)\n txt_file = directory.join('file.txt')\n txt_file.write('Something')\n with raises(FileNotFoundError):\n simple_population.checkpoint(target=txt_file, method=self.method)\n # FileExistsError is difficult to test due to timing\n\n def test_override_default_path(self, tmpdir, simple_chromosomes, simple_evaluation_function):\n # With checkpoint_target init\n directory1 = tmpdir.mkdir(\"ckpt1\")\n pop1 = Population(simple_chromosomes, simple_evaluation_function, checkpoint_target=directory1)\n pop1.checkpoint(method=self.method)\n assert len(listdir(directory1)) == 1\n # With serializer init\n directory2 = tmpdir.mkdir(\"ckpt2\")\n pop2 = Population(simple_chromosomes, simple_evaluation_function,\n serializer=SimpleSerializer(target=directory2))\n pop2.checkpoint(method=self.method)\n assert len(listdir(directory2)) == 1\n # With override\n directory3 = tmpdir.mkdir(\"ckpt3\")\n pop1.checkpoint(target=directory3, method=self.method)\n pop2.checkpoint(target=directory3, method=self.method)\n assert len(listdir(directory3)) == 2\n\n def test_evolution(self, tmpdir, simple_population):\n directory = tmpdir.mkdir(\"ckpt\")\n evo = Evolution().mutate(lambda x: x+1).checkpoint(target=directory, method=self.method, every=1)\n simple_population.evolve(evolution=evo, n=100)\n assert len(listdir(directory)) == 100\n\n def test_every(self, tmpdir, simple_population):\n directory = tmpdir.mkdir('ckpt')\n evo = Evolution().mutate(lambda x: x+1).checkpoint(target=directory, method=self.method, every=10)\n simple_population.evolve(evolution=evo, n=9)\n assert len(listdir(directory)) == 0\n simple_population.evolve(evolution=evo, n=11)\n assert len(listdir(directory)) == 2\n\n\nclass TestJsonCheckpoint(TestPickleCheckpoint):\n method = 'json'\n extension = '.json'\n exception = TypeError\n" }, { "path": "tests/test_utils.py", "content": "from pytest import mark\n\nfrom evol import Population, Individual\nfrom evol.helpers.pickers import pick_random\nfrom evol.utils import offspring_generator, select_arguments\n\n\nclass TestOffspringGenerator:\n\n def test_simple_combiner(self, simple_population: Population):\n def combiner(x, y):\n return 1\n\n result = offspring_generator(parents=simple_population.individuals,\n parent_picker=pick_random, combiner=combiner)\n assert isinstance(next(result), Individual)\n assert next(result).chromosome == 1\n\n @mark.parametrize('n_parents', [1, 2, 3, 4])\n def test_args(self, n_parents: int, simple_population: Population):\n def combiner(*parents, n_parents):\n assert len(parents) == n_parents\n return 1\n\n result = offspring_generator(parents=simple_population.individuals, n_parents=n_parents,\n parent_picker=pick_random, combiner=combiner)\n assert isinstance(next(result), Individual)\n assert next(result).chromosome == 1\n\n def test_simple_picker(self, simple_population: Population):\n def combiner(x):\n return 1\n\n def picker(parents):\n return parents[0]\n\n result = offspring_generator(parents=simple_population.individuals, parent_picker=picker, combiner=combiner)\n assert isinstance(next(result), Individual)\n assert next(result).chromosome == 1\n\n def test_multiple_offspring(self, simple_population: Population):\n def combiner(x, y):\n yield 1\n yield 2\n\n result = offspring_generator(parents=simple_population.individuals,\n parent_picker=pick_random, combiner=combiner)\n for _ in range(10):\n assert next(result).chromosome == 1\n assert next(result).chromosome == 2\n\n\nclass TestSelectArguments:\n\n @mark.parametrize('args,kwargs,result', [((1, ), {}, 1), ((1, 2), {'x': 1}, 3), ((4, 5), {'z': 8}, 9)])\n def test_no_kwargs(self, args, kwargs, result):\n @select_arguments\n def fct(*args):\n return sum(args)\n assert fct(*args, **kwargs) == result\n\n @mark.parametrize('args,kwargs,result', [((1, ), {}, 1), ((1, 2), {'x': 1}, 3), ((4, 5), {'z': 8}, 17)])\n def test_with_kwargs(self, args, kwargs, result):\n @select_arguments\n def fct(*args, z=0):\n return sum(args)+z\n assert fct(*args, **kwargs) == result\n\n @mark.parametrize('args,kwargs,result', [((1,), {'b': 3}, 4), ((1, 2), {'x': 1}, 4), ((4, 5), {'z': 8}, 17)])\n def test_all_kwargs(self, args, kwargs, result):\n @select_arguments\n def fct(a, b=0, **kwargs):\n return a + b + sum(kwargs.values())\n assert fct(*args, **kwargs) == result\n" } ]