Full Code of benedekrozemberczki/shapley for AI

Repository: benedekrozemberczki/shapley
Branch: master
Commit: 153c987a3d46
Files: 31
Total size: 42.3 KB

Directory structure:
gitextract_5e844ro4/

├── .coveragerc
├── .github/
│   └── workflows/
│       └── main.yml
├── .gitignore
├── LICENSE
├── README.md
├── docs/
│   ├── Makefile
│   ├── index.html
│   ├── requirements_1.txt
│   └── source/
│       ├── _figures/
│       │   └── build.sh
│       ├── _static/
│       │   └── css/
│       │       └── custom.css
│       ├── conf.py
│       ├── index.rst
│       ├── modules/
│       │   └── root.rst
│       └── notes/
│           ├── installation.rst
│           ├── introduction.rst
│           └── resources.rst
├── examples/
│   ├── exact_calculation_example.py
│   ├── exact_marginal_contributions_example.py
│   ├── multilinear_extension_example.py
│   └── permutation_sampler_example.py
├── readthedocs.yml
├── setup.py
├── shapley/
│   ├── __init__.py
│   ├── solution_concept.py
│   ├── solvers/
│   │   ├── __init__.py
│   │   ├── exact_enumeration.py
│   │   ├── expected_marginal_contributions.py
│   │   ├── multilinear_extension.py
│   │   └── permutation_sampler.py
│   └── version.py
└── tests/
    └── unit/
        └── test_shapley.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .coveragerc
================================================
[report]
exclude_lines =
    def create_integrand_vectors
    def create_standard_deviation_vector


================================================
FILE: .github/workflows/main.yml
================================================

name: CI

on:

  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]
  workflow_dispatch:

jobs:
    build:
      runs-on: ubuntu-latest
      steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v1
        with:
          python-version: 3.8.17
      - uses: s-weigand/setup-conda@v1
        with:
          activate-conda: true
          python-version: 3.8
      - run: conda --version
      - run: which python
      - name: Run installation.
        run: |
         conda install -y scipy
         pip install codecov
         pip install pytest
         python setup.py install
      - name: Install main package
        run: |
          pip install -e .[test]
      - name: Run test-suite
        run: |
          python -m pytest
      - name: Generate coverage report
        if: success()
        run: |
          pip install coverage
          coverage run -m pytest
      - name: Upload coverage report to codecov
        uses: codecov/codecov-action@v1
        if: success()
        with:
          file: coverage.xml


================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/



================================================
FILE: LICENSE
================================================
MIT License

Copyright (c) 2020 Benedek Rozemberczki

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
[pypi-image]: https://badge.fury.io/py/shapley.svg
[pypi-url]: https://pypi.python.org/pypi/shapley
[size-image]: https://img.shields.io/github/repo-size/benedekrozemberczki/shapley.svg
[size-url]: https://github.com/benedekrozemberczki/shapley/archive/master.zip
[build-image]: https://github.com/benedekrozemberczki/shapley/workflows/CI/badge.svg
[build-url]: https://github.com/benedekrozemberczki/shapley/actions?query=workflow%3ACI
[docs-image]: https://readthedocs.org/projects/shapley/badge/?version=latest
[docs-url]: https://shapley.readthedocs.io/en/latest/?badge=latest
[coverage-image]: https://codecov.io/gh/benedekrozemberczki/shapley/branch/master/graph/badge.svg
[coverage-url]: https://codecov.io/github/benedekrozemberczki/shapley?branch=master
[arxiv-image]: https://img.shields.io/badge/ArXiv-2101.02153-orange.svg
[arxiv-url]: https://arxiv.org/abs/2101.02153

<p align="center">
  <img width="90%" src="https://github.com/benedekrozemberczki/shapley/raw/master/shapley.jpg?sanitize=true" />
</p>

[![PyPI Version][pypi-image]][pypi-url]
[![Docs Status][docs-image]][docs-url]
[![Repo size][size-image]][size-url]
[![Code Coverage][coverage-image]][coverage-url]
[![Build Status][build-image]][build-url]
[![Arxiv][arxiv-image]][arxiv-url]

**[Documentation](https://shapley.readthedocs.io)** | **[External Resources](https://shapley.readthedocs.io/en/latest/notes/resources.html)**
| **[Research Paper](https://arxiv.org/abs/2101.02153)**

*Shapley* is a Python library for evaluating binary classifiers in a machine learning ensemble.

The library consists of various methods to compute (approximate) the Shapley value of players (models) in weighted voting games (ensemble games) - a class of transferable utility cooperative games. We covered the exact enumeration based computation and various widely know approximation methods from economics and computer science research papers. There are also functionalities to identify the heterogeneity of the player pool based on the [Shapley entropy](https://arxiv.org/abs/2101.02153). In addition, the framework comes with a [detailed documentation](https://shapley.readthedocs.io/en/latest/), an intuitive [tutorial](https://shapley.readthedocs.io/en/latest/notes/introduction.html), 100% test coverage, and illustrative toy [examples](https://github.com/benedekrozemberczki/shapley/tree/master/examples).

-------------------------------------------------------

**Citing**

If you find *Shapley* useful in your research please consider adding the following citation:

```bibtex
@inproceedings{rozemberczki2021shapley,
      title = {{The Shapley Value of Classifiers in Ensemble Games}}, 
      author = {Benedek Rozemberczki and Rik Sarkar},
      year = {2021},
      booktitle={Proceedings of the 30th ACM International Conference on Information and Knowledge Management},
      pages = {1558–1567},
}
```

--------------------------------------------------------------

**A simple example**

Shapley makes solving voting games quite easy - see the accompanying [tutorial](https://shapley.readthedocs.io/en/latest/notes/introduction.html#applications). For example, this is all it takes to solve a weighted voting game with defined on the fly with permutation sampling:

```python
import numpy as np
from shapley import PermutationSampler

W = np.random.uniform(0, 1, (1, 7))
W = W/W.sum()
q = 0.5

solver = PermutationSampler()
solver.solve_game(W, q)
shapley_values = solver.get_solution()
```
----------------------------------------------------------------------------------

**Methods Included**

In detail, the following methods can be used.


* **[Expected Marginal Contribution Approximation](https://shapley.readthedocs.io/en/latest/modules/root.html#shapley.solvers.expected_marginal_contributions.ExpectedMarginalContributions)** from Fatima *et al.*: [A Linear Approximation Method for the Shapley Value](https://www.sciencedirect.com/science/article/pii/S0004370208000696)

* **[Multilinear Extension](https://shapley.readthedocs.io/en/latest/modules/root.html#shapley.solvers.multilinear_extension.MultilinearExtension)** from Owen: [Multilinear Extensions of Games](https://www.jstor.org/stable/2661445?seq=1#metadata_info_tab_contents)

* **[Monte Carlo Permutation Sampling](https://shapley.readthedocs.io/en/latest/modules/root.html#shapley.solvers.permutation_sampler.PermutationSampler)** from Maleki *et al.*: [Bounding the Estimation Error of Sampling-based Shapley Value Approximation](https://arxiv.org/abs/1306.4265)

* **[Exact Enumeration](https://shapley.readthedocs.io/en/latest/modules/root.html#shapley.solvers.exact_enumeration.ExactEnumeration)** from Shapley: [A Value for N-Person Games](https://www.rand.org/pubs/papers/P0295.html)

--------------------------------------------------------------------------------


Head over to our [documentation](https://shapley.readthedocs.io) to find out more about installation, creation of datasets and a full list of implemented methods and available datasets.
For a quick start, check out the [examples](https://github.com/benedekrozemberczki/shapley/tree/master/examples) in the `examples/` directory.

If you notice anything unexpected, please open an [issue](https://benedekrozemberczki/shapley/issues). If you are missing a specific method, feel free to open a [feature request](https://github.com/benedekrozemberczki/shapley/issues).

--------------------------------------------------------------------------------

**Installation**

```
$ pip install shapley
```

**Running tests**

```
$ python setup.py test
```
----------------------------------------------------------------------------------

**Running examples**

```
$ cd examples
$ python permutation_sampler_example.py
```

----------------------------------------------------------------------------------

**License**

- [MIT License](https://github.com/benedekrozemberczki/shapley/blob/master/LICENSE)


================================================
FILE: docs/Makefile
================================================
SPHINXBUILD   = sphinx-build
SPHINXPROJ    = shapley
SOURCEDIR     = source
BUILDDIR      = build

.PHONY: help Makefile

%: Makefile
	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)"


================================================
FILE: docs/index.html
================================================
<!DOCTYPE html>

<html>
<head>
  <title>Redirect</title>
  <meta http-equiv="refresh" content="0; url=https://karateclub.readthedocs.io" />
</head>
</html>


================================================
FILE: docs/requirements_1.txt
================================================
numpy
sphinx==4.0.2
sphinx_rtd_theme==0.5.2
scipy
six
nbsphinx
nbsphinx_link


================================================
FILE: docs/source/_figures/build.sh
================================================
#!/bin/sh

for filename in *.tex; do
  basename=$(basename $filename .tex)
  pdflatex "$basename.tex"
  pdf2svg "$basename.pdf" "$basename.svg"
done


================================================
FILE: docs/source/_static/css/custom.css
================================================
.wy-side-nav-search .wy-dropdown > a img.logo, .wy-side-nav-search > a img.logo {
  height: 150px;
}

.wy-side-nav-search {
  background: rgb(243,244,247);
}

.wy-side-nav-search > div.version {
  color: black;
}

.wy-nav-content-wrap {
  background: inherit;
}

.wy-side-nav-search input[type="text"] {
  border: none;
  box-shadow: none;
  background: white;
  border-radius: 0;
  font-size: 100%;
}

.wy-menu-vertical li.current a,
.wy-menu-vertical li.toctree-l1.current > a {
  border: none;
}

.ethical-rtd > div.ethical-sidebar,
.ethical-rtd > div.ethical-footer {
  display: none !important;
}

h1 {
  text-transform: uppercase;
  font-family: inherit;
  font-weight: 200;
}

h2,
.rst-content .toctree-wrapper p.caption {
  font-family: inherit;
  font-weight: 200;
}

.rst-content a:visited {
  color: #3091d1;
}

/* Begin code */
.rst-content pre.literal-block,
.rst-content div[class^="highlight"] {
  border: none;
}

.rst-content pre.literal-block,
.rst-content div[class^="highlight"] pre,
.rst-content .linenodiv pre {
  font-size: 80%;
}

.highlight {
  background: #f6f8fa;
  border-radius: 6px;
}

.highlight .kn,
.highlight .k {
  color: #d73a49;
}

.highlight .nn {
  color: inherit;
  font-weight: inherit;
}

.highlight .nc {
  color: #e36209;
  font-weight: inherit;
}

.highlight .fm,
.highlight .nd,
.highlight .nf,
.highlight .nb {
  color: #6f42c1;
}

.highlight .bp,
.highlight .n {
  color: inherit;
}

.highlight .kc,
.highlight .s1,
.highlight .s2,
.highlight .mi,
.highlight .mf,
.highlight .bp,
.highlight .bn,
.highlight .ow {
  color: #005cc5;
  font-weight: inherit;
}

.highlight .c1 {
  color: #6a737d;
}

.rst-content code.xref {
  padding: .2em .4em;
  background: rgba(27,31,35,.05);
  border-radius: 6px;
  border: none;
}
/* End code */

.rst-content dl:not(.docutils) dt,
.rst-content dl:not(.docutils) dl dt {
  background: rgb(243,244,247);
}

.rst-content dl:not(.docutils) dt.field-odd {
  text-transform: uppercase;
  background: inherit;
  border: none;
  padding: 6px 0;
}

.rst-content dl:not(.docutils) .property:first-child .pre {
  text-transform: uppercase;
  font-style: normal;
}

em.sig-param span.n:first-child, em.sig-param span.n:nth-child(2) {
  color: black;
  font-style: normal;
}

em.sig-param span.n:nth-child(3),
em.sig-param span.n:nth-child(3) a {
  color: inherit;
  font-weight: normal;
  font-style: normal;
}

em.sig-param span.default_value {
  font-family: SFMono-Regular,Menlo,Monaco,Consolas,"Liberation Mono","Courier New",Courier,monospace;
  font-style: normal;
  font-size: 90%;
}

.sig-paren {
  padding: 0 4px;
}

.wy-table-responsive table td,
.wy-table-responsive table th {
  white-space: normal;
}

.wy-table-bordered-all,
.rst-content table.docutils {
  border: none;
}

.wy-table-bordered-all td,
.rst-content table.docutils td {
  border: none;
}

.wy-table-odd td,
.wy-table-striped tr:nth-child(2n-1) td,
.rst-content table.docutils:not(.field-list) tr:nth-child(2n-1) td {
  background: rgb(243,244,247);
}

.wy-table td,
.rst-content table.docutils td,
.rst-content table.field-list td,
.wy-table th,
.rst-content table.docutils th,
.rst-content table.field-list th {
  padding: 14px;
}

table.colwidths-given tr td p,
table.colwidths-given tr th p {
  text-align: center;
}

table.colwidths-given tr td:first-child p,
table.colwidths-given tr th:first-child p {
  text-align: left;
}


================================================
FILE: docs/source/conf.py
================================================
import datetime
import sphinx_rtd_theme
import doctest
import sys
import os

sys.path.insert(0, os.path.abspath('../../'))

extensions = [
    'sphinx.ext.autodoc',
    'sphinx.ext.doctest',
    'sphinx.ext.intersphinx',
    'sphinx.ext.mathjax',
    'sphinx.ext.napoleon',
    'sphinx.ext.viewcode',
    'sphinx.ext.githubpages',
]

autodoc_default_options = {
    'member-order': 'bysource',
    'special-members': '__init__',
    'undoc-members': True,
}


autodoc_mock_imports = ['numba']

source_suffix = '.rst'
master_doc = 'index'

author = 'Benedek Rozemberczki'
project = 'shapley'
copyright = '{}, {}'.format(datetime.datetime.now().year, author)


html_theme = 'sphinx_rtd_theme'
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

doctest_default_flags = doctest.NORMALIZE_WHITESPACE
intersphinx_mapping = {'python': ('https://docs.python.org/', None)}

html_theme_options = {
    'collapse_navigation': False,
    'display_version': True,
    'logo_only': True,
}

html_logo = '_static/img/shapley_logo_text.jpg'
html_static_path = ['_static']
html_context = {'css_files': ['_static/css/custom.css']}

add_module_names = False


def setup(app):
    def skip(app, what, name, obj, skip, options):
        members = [
            '__init__',
            '__repr__',
            '__weakref__',
            '__dict__',
            '__module__',
        ]
        return True if name in members else skip

    app.connect('autodoc-skip-member', skip)


================================================
FILE: docs/source/index.rst
================================================
:github_url: https://github.com/benedekrozemberczki/shapley

Shapley Documentation
===============================

*Shapley* is a Python library for evaluating binary classifiers in a machine learning ensemble. The library consists of various methods to compute (approximate) the Shapley value of players (models) in weighted voting games (ensemble games) - a class of transferable utility cooperative games. We covered the exact enumeration based computation and various widely know approximation methods from economics and computer science research papers. There are also functionalities to identify the heterogeneity of the player pool based on the `Shapley entropy <https://arxiv.org/abs/2101.02153>`_. In addition, the framework comes with a `detailed documentation <https://shapley.readthedocs.io/en/latest/>`_, an intuitive `tutorial <https://shapley.readthedocs.io/en/latest/notes/introduction.html>`_, 100% test coverage and illustrative toy `examples <https://github.com/benedekrozemberczki/shapley/tree/master/examples>`_.

.. code-block:: latex

    >@inproceedings{rozemberczki2021shapley,
                    title = {{The Shapley Value of Classifiers in Ensemble Games}}, 
                    author = {Benedek Rozemberczki and Rik Sarkar},
                    year = {2021},
                    booktitle={Proceedings of the 30th ACM International Conference on Information and Knowledge Management},
                    pages = {1558–1567},
    }


.. toctree::
   :glob:
   :maxdepth: 2
   :caption: Notes

   notes/installation
   notes/introduction
   notes/examples
   notes/resources

.. toctree::
   :glob:
   :maxdepth: 2
   :caption: Package Reference

   modules/root


================================================
FILE: docs/source/modules/root.rst
================================================
Shapley 
==================

.. contents:: Contents
    :local:

Exact Solution
--------------------

.. automodule:: shapley.solvers.exact_enumeration
    :members:
    :inherited-members:

Approximate Solutions 
--------------------

.. automodule:: shapley.solvers.permutation_sampler
    :members:
    :inherited-members:

.. automodule:: shapley.solvers.multilinear_extension
    :members:
    :inherited-members:

.. automodule:: shapley.solvers.expected_marginal_contributions
    :members:
    :inherited-members:
    :exclude-members: create_integrand_vectors, create_standard_deviation_vector


================================================
FILE: docs/source/notes/installation.rst
================================================
Installation
============

Install the package by using pip:

    .. code-block:: none

        $ pip install shapley

Upgrade your outdated Shapley version by using:

    .. code-block:: none

        $ pip install shapley --upgrade


To check your current package version just simply run:

    .. code-block:: none

        $ pip freeze | grep shapley


================================================
FILE: docs/source/notes/introduction.rst
================================================
Introduction by example
=======================

*Shapley* is a Python library for evaluating binary classifiers in a machine learning ensemble. The library consists of various methods to compute (approximate) the Shapley value of players (models) in weighted voting games (ensemble games) - a class of transferable utility cooperative games. We covered the exact enumeration based computation and various widely know approximation methods from economics and computer science research papers. There are also functionalities to identify the heterogeneity of the player pool based on the `Shapley entropy <https://arxiv.org/abs/2101.02153>`_. In addition, the framework comes with a `detailed documentation <https://shapley.readthedocs.io/en/latest/>`_, an intuitive `tutorial <https://shapley.readthedocs.io/en/latest/notes/introduction.html>`_, 100% test coverage and illustrative toy `examples <https://github.com/benedekrozemberczki/shapley/tree/master/examples>`_.


--------------------------------------------------------------------------------

**Citing**

If you find *Shapley* useful in your research, please consider citing the following paper:

.. code-block:: latex

    >@inproceedings{rozemberczki2021shapley,
                    title = {{The Shapley Value of Classifiers in Ensemble Games}}, 
                    author = {Benedek Rozemberczki and Rik Sarkar},
                    year = {2021},
                    booktitle={Proceedings of the 30th ACM International Conference on Information and Knowledge Management},
                    pages = {1558–1567},
    }


Overview
=======================
--------------------------------------------------------------------------------

We provide a brief overview pf the fundamental concepts and features of Shapley through the following simple examples:

.. contents::
    :local:

Standardized dataset ingestion
------------------------------

Shapley assumes that the weights in the game(s) are provided by a ``Numpy float array``. This array is 2 dimensional, the first dimension corresponds to the games and the second one corresponds to the players. The quota for the games is set universally by a ``float`` value. We assume that all of the games can be won by a certain coalition. 

Standardized output generation
------------------------------
The Shapley values are returned as a ``Numpy float array`` which has 2 dimensions - rows represent the games and columns are players. A given value in the ``Numpy array`` is the Shapley value of a player in a game. 

API driven design
-----------------

Shapley uses an API driven design which involves a  ``SolutionConcept`` meta class. All of the Shapley value calculation techniques inherit from this class the methods used to verify the input and output. Methods used to estimate the average Shapley values and the Shapley entropy are also inherited from this meta class. Shapley value calculation ``solvers`` are created by parametrizing the appropriate constructor. Just like in the example below where we define a ``Monte Carlo Permutation Sampler``.

.. code-block:: python

    from shapley import PermutationSampler

    solver = PermutationSampler()

We define a quota value and a weight matrix so we can solve a single game with permutation sampling. It is worth noting that this game is extremely large - it has 70 players.

.. code-block:: python
    
    import numpy as np

    W = np.random.uniform(0, 1, (1, 70))
    W = W/W.sum()
    q = 0.5

We solve the game by using the ``solve_game`` method which uses the weights and the quota as input. The approximate Shapley values are returned by the ``get_solution`` method.

.. code-block:: python
    
    solver.solve_game(W, q)
    shapley_values = solver.get_solution()



================================================
FILE: docs/source/notes/resources.rst
================================================
External resources
==================

Shapley Value Approximations
----------------------------

* Shaheen Fatimaa, Michael Wooldridge, Nicholas Jennings: **A Linear Approximation Method for the Shapley Value** `Paper <https://www.sciencedirect.com/science/article/pii/S0004370208000696>`_

* Guillermo Owen: **Multilinear Extensions of Games** `Paper <https://www.jstor.org/stable/2661445?seq=1#metadata_info_tab_contents>`_

* Sasan Maleki, Long Tran-Thanh, Greg Hines, Talal Rahwan, Alex Rogers: **Bounding the Estimation Error of Sampling-based Shapley Value Approximation** `Paper <https://arxiv.org/abs/1306.4265>`_

* Lloyd Shapley: **A Value for N-Person Games** `Paper <https://www.rand.org/pubs/papers/P0295.html>`_


================================================
FILE: examples/exact_calculation_example.py
================================================
"""Exact Enumeration Example"""

import numpy as np
from shapley import ExactEnumeration

W = np.random.uniform(0, 1, (1, 7))
W = W/W.sum()
q = 0.5

solver = ExactEnumeration()
solver.solve_game(W, q)
shapley_values = solver.get_solution()

print(shapley_values)


================================================
FILE: examples/exact_marginal_contributions_example.py
================================================
"""Expected Marginal Contributions Example"""

import numpy as np
from shapley import ExpectedMarginalContributions

W = np.random.uniform(0, 1, (1, 20))
W = W/W.sum()
q = 0.35

solver = ExpectedMarginalContributions()
solver.solve_game(W, q)
shapley_values = solver.get_solution()

print(shapley_values)


================================================
FILE: examples/multilinear_extension_example.py
================================================
"""Multilinear Extension Example"""

import numpy as np
from shapley import MultilinearExtension

W = np.random.uniform(0, 1, (20, 20))
W = W/W.sum(1)
q = 0.35

solver = MultilinearExtension()
solver.solve_game(W, q)
shapley_values = solver.get_solution()

print(shapley_values)


================================================
FILE: examples/permutation_sampler_example.py
================================================
"""Permutation Sampler Example"""

import numpy as np
from shapley import PermutationSampler

W = np.random.uniform(0, 1, (1, 7))
W = W/W.sum()
q = 0.5

solver = PermutationSampler()
solver.solve_game(W, q)
shapley_values = solver.get_solution()

print(shapley_values)


================================================
FILE: readthedocs.yml
================================================
version: 2

build:
   image: latest

python:
   version: 3.7
   system_packages: true
   install:
      - requirements: docs/requirements_1.txt
      - method: setuptools
        path: .

formats: []


================================================
FILE: setup.py
================================================
import os
from setuptools import find_packages, setup

on_rtd = os.environ.get("READTHEDOCS") == "True"

install_requires = ["numpy", "scipy", "six"]

if not on_rtd:
    install_requires.append("numba")

setup_requires = ["pytest-runner"]


tests_require = ["pytest", "pytest-cov", "mock", "unittest"]



keywords = [
    "shapley",
    "random-forest",
    "data-science",
    "shap",
    "fairness",
    "ensemble",
    "expert-system",
    "explanability",
    "voting-classifier",
    "classifier",
    "machine-learning",
    "deep-learning",
    "deeplearning",
    "game-theory",
    "cooperative-game"
]


setup(
    name="shapley",
    packages=find_packages(),
    version="1.0.3",
    license="MIT",
    description="A general purpose library to quantify the value of classifiers in an ensemble.",
    author="Benedek Rozemberczki",
    author_email="benedek.rozemberczki@gmail.com",
    url="https://github.com/benedekrozemberczki/shapley",
    download_url="https://github.com/benedekrozemberczki/shapley/archive/v_10003.tar.gz",
    keywords=keywords,
    install_requires=install_requires,
    setup_requires=setup_requires,
    tests_require=tests_require,
    classifiers=[
        "Development Status :: 3 - Alpha",
        "Intended Audience :: Developers",
        "Topic :: Software Development :: Build Tools",
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3.7",
    ],
)


================================================
FILE: shapley/__init__.py
================================================
from shapley.solvers import *
from shapley.solution_concept import SolutionConcept
from shapley.version import __version__  # noqa:F401,F403

__all__ = [
    "shapley",
]


================================================
FILE: shapley/solution_concept.py
================================================
"""Solution Concept base class."""


import numpy as np
from abc import ABCMeta, abstractmethod


class SolutionConcept(metaclass=ABCMeta):
    """Solution Concept base class with constructor and public methods."""

    @abstractmethod
    def get_solution(self) -> np.ndarray:
        r"""Returning the solution.

        Return Types:
            Phi (Numpy array): Approximate Shapley matrix of players in the game(s) with size :math:`n \times m`.
        """
        pass

    @abstractmethod
    def solve_game(self, W: np.ndarray, q: float):
        r"""Solving the weigted voting game(s).

        Args:
            W (Numpy array): An :math:`n \times m` matrix of voting weights for the :math:`n` games with :math:`m` players.
            q (float): Quota in the games.
        """
        pass

    @abstractmethod
    def setup(self, W: np.ndarray):
        """Creating an empty Shapley value matrix and a player pool."""
        pass

    def _check_quota(self, q: float):
        """Checking for negative quota."""
        assert 0.0 <= q

    def _verify_result_shape(self, W: np.ndarray, Phi: np.ndarray):
        """Checking the shape of the Shapley value matrix."""
        assert W.shape == Phi.shape

    def _verify_distribution(self, Phi: np.ndarray):
        """Verify distribution hypothesis."""
        assert (np.sum(Phi) - Phi.shape[0]) < 0.0001

    def _run_sanity_check(self, W: np.ndarray, Phi: np.ndarray):
        """Checking the basic assumptions about the Shapley values."""
        self._verify_result_shape(W, Phi)
        self._verify_distribution(Phi)

    def _set_average_shapley(self):
        """Calculating the average Shapley value scores."""
        self._Phi_tilde = np.mean(self._Phi, axis=0)

    def _set_shapley_entropy(self):
        """Calculating the Shapley entropy score."""
        self._shapley_entropy = -np.sum(self._Phi_tilde * np.log(self._Phi_tilde))

    def get_average_shapley(self) -> np.ndarray:
        """Getting the average Shapley value scores."""
        return self._Phi_tilde

    def get_shapley_entropy(self) -> float:
        """Getting the Shapley entropy score."""
        return self._shapley_entropy


================================================
FILE: shapley/solvers/__init__.py
================================================
from .exact_enumeration import ExactEnumeration
from .permutation_sampler import PermutationSampler
from .multilinear_extension import MultilinearExtension
from .expected_marginal_contributions import ExpectedMarginalContributions


================================================
FILE: shapley/solvers/exact_enumeration.py
================================================
"""Exact Enumeration Based Shapley Value."""

import itertools
import numpy as np
from shapley.solution_concept import SolutionConcept


class ExactEnumeration(SolutionConcept):
    r"""Exact enumeration of all permutations and finding the pivotal voters. It
    is designed with a generator of the permutations. For details see this paper:
    `"A Value for N-Person Games." <https://www.rand.org/pubs/papers/P0295.html>`_
    """

    def setup(self, W: np.ndarray):
        """Creating an empty Shapley value matrix and a player pool."""
        self._Phi = np.zeros(W.shape)
        self._indices = [i for i in range(W.shape[1])]
        self.permutations = 0

    def _run_permutations(self, W: np.ndarray, q: float):
        """Creating Monte Carlo permutations and finding the marginal voter."""
        for perm in itertools.permutations(self._indices):
            self.permutations = self.permutations + 1
            indices = list(perm)
            W_perm = W[:, indices]
            cum_sum = np.cumsum(W_perm, axis=1)
            pivotal = np.array(indices)[np.argmax(cum_sum > q, axis=1)]
            self._Phi[np.arange(W.shape[0]), pivotal] += 1.0
        self._Phi = self._Phi / self.permutations

    def solve_game(self, W: np.ndarray, q: float):
        r"""Solving the weigted voting game(s).

        Args:
            W (Numpy array): An :math:`n \times m` matrix of voting weights for the :math:`n` games with :math:`m` players.
            q (float): Quota in the games.
        """
        self._check_quota(q)
        self.setup(W)
        self._run_permutations(W, q)
        self._run_sanity_check(W, self._Phi)
        self._set_average_shapley()
        self._set_shapley_entropy()

    def get_solution(self) -> np.ndarray:
        r"""Returning the solution.

        Return Types:
            Phi (Numpy array): Approximate Shapley matrix of players in the game(s) with size :math:`n \times m`.
        """
        return self._Phi


================================================
FILE: shapley/solvers/expected_marginal_contributions.py
================================================
"""Expected Marginal Contributions Based Shapley Value Approximation."""

import numpy as np
from numba import jit
from scipy.stats import norm
from shapley.solution_concept import SolutionConcept


@jit
def create_integrand_vectors(player_count, q, w):
    """
    Creating the vector of standard deviations and integrand limits.
    :param player_count (int): Number of players in the game.
    :param q (float): Quota to win the game.
    :param w (float): Weight of the player in the game.

    :return a_s: Vector of lower integrand limits.
    :return b_s: Vector of upper integrand limits.
    """
    a_s = (q - w) / np.linspace(1, player_count - 1, player_count - 1)
    b_s = (q - 10**-20) / np.linspace(1, player_count - 1, player_count - 1)
    return a_s, b_s


@jit
def create_standard_deviation_vector(var, player_count):
    """
    Creating the vector of standard deviations and integrand limits.
    :param var (float): Variance of the weights in the game.
    :param player_count (int): Number of players in the game.

    :return sigma_s: Vector of standard deviations for the game.
    """
    sigma_s = np.power(var / np.linspace(1, player_count - 1, player_count - 1), 0.5)
    return sigma_s


class ExpectedMarginalContributions(SolutionConcept):
    r"""The expected marginal contributions approximation of the Shapley value in a weighted
    voting game using the technique proposed by Fatima. For details see this paper:
    `"A Linear Approximation Method for the Shapley Value."
     <https://www.sciencedirect.com/science/article/pii/S0004370208000696>`_
    """

    def __init__(self, epsilon: float = 10**-8):
        self.epsilon = epsilon

    def setup(self, W: np.ndarray):
        """Creating an empty Shapley value matrix."""
        self._Phi = np.zeros(W.shape)

    def _approximate(self, W: np.ndarray, q: float):
        """Using the naive multilinear approximation method."""
        for data_point in range(W.shape[0]):
            mu = float(np.mean(W[data_point, :]))
            var = float(np.var(W[data_point, :]))
            sigma_s = create_standard_deviation_vector(var, W.shape[1])
            for player_index in range(W.shape[1]):
                w = W[data_point, player_index]
                a_s, b_s = create_integrand_vectors(W.shape[1], q, w)
                shap_in_game = norm.cdf(b_s, loc=mu, scale=sigma_s) - norm.cdf(
                    a_s, loc=mu, scale=sigma_s
                )
                self._Phi[data_point, player_index] += np.sum(shap_in_game)

        self._Phi = self._Phi / np.sum(self._Phi, axis=1).reshape(-1, 1)

    def solve_game(self, W: np.ndarray, q: float):
        r"""Solving the weigted voting game(s).

        Args:
            W (Numpy array): An :math:`n \times m` matrix of voting weights for the :math:`n` games with :math:`m` players.
            q (float): Quota in the games.
        """
        self._check_quota(q)
        self.setup(W)
        self._approximate(W, q)
        self._run_sanity_check(W, self._Phi)
        self._set_average_shapley()
        self._set_shapley_entropy()

    def get_solution(self) -> np.ndarray:
        r"""Returning the solution.

        Return Types:
            Phi (Numpy array): Approximate Shapley matrix of players in the game(s) with size :math:`n \times m`.
        """
        return self._Phi


================================================
FILE: shapley/solvers/multilinear_extension.py
================================================
"""Multilinear Extension Based Shapley Value Approximation."""

import random
import numpy as np
from scipy.stats import norm
from shapley.solution_concept import SolutionConcept


class MultilinearExtension(SolutionConcept):
    r"""The multilinear extension approximation of the Shapley value in a weighted
    voting game using the technique proposed by Owen. For details see this paper:
    `"Multilinear Extensions of Games."  <https://www.jstor.org/stable/2661445#metadata_info_tab_contents>`_
    """

    def setup(self, W: np.ndarray):
        """Creating an empty Shapley value matrix."""
        self._Phi = np.zeros(W.shape)

    def _approximate(self, W: np.ndarray, q: float):
        """Using the naive multilinear approximation method."""
        mu = np.tile(np.sum(W, axis=1).reshape(-1, 1), W.shape[1]) - W
        std = np.tile(
            np.sum(np.square(W), axis=1).reshape(-1, 1), W.shape[1]
        ) - np.square(W)
        upper = (np.zeros(W.shape) + q - mu) / std
        lower = (np.zeros(W.shape) + q - W - mu) / std
        self._Phi = norm.cdf(upper, 0, 1) - norm.cdf(lower, 0, 1)
        self._Phi = self._Phi / np.sum(self._Phi, axis=1).reshape(-1, 1)

    def solve_game(self, W: np.ndarray, q: float):
        r"""Solving the weigted voting game(s).

        Args:
            W (Numpy array): An :math:`n \times m` matrix of voting weights for the :math:`n` games with :math:`m` players.
            q (float): Quota in the games.
        """
        self._check_quota(q)
        self.setup(W)
        self._approximate(W, q)
        self._run_sanity_check(W, self._Phi)
        self._set_average_shapley()
        self._set_shapley_entropy()

    def get_solution(self) -> np.ndarray:
        r"""Returning the solution.

        Return Types:
            Phi (Numpy array): Approximate Shapley matrix of players in the game(s) with size :math:`n \times m`.
        """
        return self._Phi


================================================
FILE: shapley/solvers/permutation_sampler.py
================================================
"""Permutation Sampling Based Shapley Value Approximation."""

import random
import numpy as np
from shapley.solution_concept import SolutionConcept


class PermutationSampler(SolutionConcept):
    r"""Permutation sampler to solve a block of weighted voting games. The solver
    samples random permutations of the players uniformly. In each permutation we
    identify the pivotal voter. We use the empirical probability of becoming the
    pivotal player as the Shapley value estimate. For details see this paper:
    `"Bounding the Estimation Error of Sampling-based Shapley Value Approximation."
    <https://arxiv.org/abs/1306.4265>`_

    Args:
        permutations (int): Number of permutations. The default is 1000.
    """

    def __init__(self, permutations: int = 1000):
        self.permutations = permutations

    def setup(self, W: np.ndarray):
        """Creating an empty Shapley value matrix and a player pool."""
        self._Phi = np.zeros(W.shape)
        self._indices = [i for i in range(W.shape[1])]

    def _run_permutations(self, W: np.ndarray, q: float):
        """Creating Monte Carlo permutations and finding the marginal voter."""
        for _ in range(self.permutations):
            random.shuffle(self._indices)
            W_perm = W[:, self._indices]
            cum_sum = np.cumsum(W_perm, axis=1)
            pivotal = np.array(self._indices)[np.argmax(cum_sum > q, axis=1)]
            self._Phi[np.arange(W.shape[0]), pivotal] += 1.0
        self._Phi = self._Phi / self.permutations

    def solve_game(self, W: np.ndarray, q: float):
        r"""Solving the weigted voting game(s).

        Args:
            W (Numpy array): An :math:`n \times m` matrix of voting weights for the :math:`n` games with :math:`m` players.
            q (float): Quota in the games.
        """
        self._check_quota(q)
        self.setup(W)
        self._run_permutations(W, q)
        self._run_sanity_check(W, self._Phi)
        self._set_average_shapley()
        self._set_shapley_entropy()

    def get_solution(self) -> np.ndarray:
        r"""Returning the solution.

        Return Types:
            Phi (Numpy array): Approximate Shapley matrix of players in the game(s) with size :math:`n \times m`.
        """
        return self._Phi


================================================
FILE: shapley/version.py
================================================
"""Contains the version of Shapley."""

__version__ = "1.0.3"


================================================
FILE: tests/unit/test_shapley.py
================================================
"""Generic shape and entropy tests."""

import pytest
import unittest

import math
import numpy as np
from shapley import ExactEnumeration
from shapley import PermutationSampler
from shapley import MultilinearExtension
from shapley import ExpectedMarginalContributions


class TestShapley(unittest.TestCase):

    def test_permutation_sampling(self):
        """
        Testing the Permutation Sampler class.
        """
        solver = PermutationSampler()

        W = np.random.uniform(0, 1, (100, 100))
        solver.solve_game(W, q=50)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = PermutationSampler(permutations=10000)

        W = np.random.uniform(0, 1, (100, 50))
        solver.solve_game(W, q=20)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = PermutationSampler(permutations=10000)

        W = np.random.uniform(0, 1, (10, 13))
        solver.solve_game(W, q=3)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0


    def test_multilinear_extension(self):
        """
        Testing the Multilinear Extension class.
        """

        solver = MultilinearExtension()

        W = np.random.uniform(0, 1, (100, 97))
        solver.solve_game(W, q=0.25)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = MultilinearExtension()

        W = np.random.uniform(0, 1, (100, 48))
        solver.solve_game(W, q=0.25)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = MultilinearExtension()

        W = np.random.uniform(0, 1, (10, 13))
        solver.solve_game(W, q=0.13)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

    def test_exact_enumeration(self):
        """
        Testing the Exact Enumeration class.
        """

        solver = ExactEnumeration()

        W = np.random.uniform(0, 1, (100, 7))
        solver.solve_game(W, q=2.5)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > -0.001

        solver = ExactEnumeration()

        W = np.random.uniform(0, 1, (100, 6))
        solver.solve_game(W, q=2)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > -0.001

        solver = ExactEnumeration()

        W = np.random.uniform(0, 1, (10, 5))
        solver.solve_game(W, q=3)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > -0.001


    def test_expected_marginal_contributions(self):
        """
        Testing the Expected Marginal Contributions class.
        """

        solver = ExpectedMarginalContributions()

        W = np.random.uniform(0, 1, (100, 97))
        solver.solve_game(W, q=15.5)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = ExpectedMarginalContributions()

        W = np.random.uniform(0, 1, (100, 48))
        solver.solve_game(W, q=12.5)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0

        solver = ExpectedMarginalContributions(epsilon=10 ** -5)

        W = np.random.uniform(0, 1, (10, 13))
        solver.solve_game(W, q=3.0)
        Phi = solver.get_solution()
        Phi_tilde = solver.get_average_shapley()
        entropy = solver.get_shapley_entropy()

        assert Phi.shape == W.shape
        assert Phi_tilde.shape == (W.shape[1],)
        assert -math.log(1.0 / W.shape[1]) - entropy > 0