[ { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug Report\nabout: Please use this issue template if you are filing a bug report.\n---\n\n# Brief Description\n\n\n\n# System Information\n\n\n\n- Operating system: macOS/Linux/Windows \n- OS details (optional): \n- Python version (required): \n\n# Minimally Reproducible Code\n\n\n\n# Error Messages\n\n\n\n```\n\n```\n" }, { "path": ".github/ISSUE_TEMPLATE/documentation_fix.md", "content": "---\nname: Propose a Documentation Fix\nabout: Use this issue tracker template if you'd like to propose a fix to the documentation.\n---\n\n# Brief Description of Fix\n\n\n\nCurrently, the docs...\n\nI would like to propose a change, such that now the docs...\n\n# Relevant Context\n\n\n\n- [Link to documentation page](hhttps://pandas-log.readthedocs.io)\n" }, { "path": ".github/ISSUE_TEMPLATE/new_examples.md", "content": "---\nname: Add/Modify Notebooks\nabout: Use this specific template if you'd like to contribute a notebook to the examples gallery or modify an existing one.\n---\n\n# Brief Description\n\n\n\nI'd like to write a notebook that...\n\n(optional but encouraged) This notebook would likely cover the following pandas-log functionality:\n\n- \n- \n- \n\n\n\n# Dataset\n\n\n" }, { "path": ".github/ISSUE_TEMPLATE/new_proposed_feature.md", "content": "---\nname: Propose New Feature\nabout: If you'd like to propose a new feature, please use this template.\n---\n\n# Brief Description\n\n\n\nI would like to propose...\n" }, { "path": ".github/pull_request_template.md", "content": "\n\n# PR Description\n\nPlease describe the changes proposed in the pull request: \n\n- \n- \n- \n\n\n\n\n\n**This PR resolves #(put issue number here, and remove parentheses).**\n\n\n\n# PR Checklist\n\n\n\nPlease ensure that you have done the following:\n\n1. [ ] PR in from a fork off your branch. Do not PR from ``:master, but rather from ``:.\n\n2. [ ] If you're not on the contributors list, add yourself to `AUTHORS.rst`.\n\n\n## Quick Check\n\nTo do a very quick check that everything is correct, follow these steps below:\n\n- [ ] Run the command `make format` from pandsa-log's top-level directory. This will automatically run:\n - black formatting\n - fix imports with isort\n" }, { "path": ".gitignore", "content": ".idea\nexplore_pandas_log.*\npandas_log/__pycache__\n.ipynb_checkpoints\n.cache\n.eggs\n*.egg-info\n" }, { "path": "AUTHORS.rst", "content": "=======\nCredits\n=======\n\nDevelopment Lead\n----------------\n\n* Eyal Trabelsi \n\nContributors\n------------\n\n* Charles Davis \n" }, { "path": "CONTRIBUTING.rst", "content": "\n============\nContributing\n============\n\nContributions are welcome, and they are greatly appreciated!\nEvery little bit helps, and credit will always be given.\n\nThe following sections detail a variety of ways to contribute,\nas well as how to get started.\n\nTypes of Contributions\n=======================\n\nWrite Documentation\n--------------------\n``pandas-log`` could always use more documentation,\nwhether as part of the official ``pandas-log`` docs, in docstrings, or the examples gallery.\n\nDuring sprints, we require newcomers to the project to\nfirst contribute a documentation fix before contributing a code fix.\nDoing so has numerous benefits:\n\n1. You become familiar with the project by first reading through the docs.\n2. Your documentation contribution will be a pain point that you have full context on.\n3. Your contribution will be impactful because documentation is the project's front-facing interface.\n4. Your first contribution will be simpler, because you won't have to wrestle with build systems.\n5. You can choose between getting set up locally first (recommended), or instead directly making edits on the GitHub web UI (also not a problem).\n6. Every newcomer is equal in our eyes, and it's the most egalitarian way to get started (regardless of experience).\n\nRemote contributors outside of sprints and prior contributors\nwho are joining us at the sprints need not adhere to this rule,\nas a good prior assumption is that you are a motivated user of the library already.\nIf you have made a prior pull request to the library,\nwe would like to encourage you to mentor newcomers in lieu of coding contributions.\n\nDocumentation can come in many forms. For example, you might want to contribute:\n\n- Fixes for a typographical, grammatical, or spelling error.\n- Changes for a docstring that was unclear.\n- Clarifications for installation/setup instructions that are unclear.\n- Corrections to a sentence/phrase/word choice that didn't make sense.\n- New example/tutorial notebooks using the library.\n- Edits to existing tutorial notebooks with better code style.\n\nIn particular, contributing new tutorial notebooks and\nimproving the clarity of existing ones are great ways to\nget familiar with the library and find pain points that\nyou can propose as fixes or enhancements to the library.\n\nReport Bugs\n------------\nReport bugs at https://github.com/eyaltrabelsi/pandas-log/issues.\n\nIf you are reporting a bug, please include:\n\n* Your operating system name and version.\n* Any details about your local setup that might be helpful in troubleshooting.\n* Detailed steps to reproduce the bug.\n\nFix Bugs\n---------\nLook through the GitHub issues for bugs.\nAnything tagged with ``bug`` and ``available to hack on`` is open to\nwhoever wants to implement it.\n\nDo be sure to claim the issue for yourself by indicating,\n\"I would like to work on this issue.\"\nIf you would like to discuss it further before going forward,\nyou are more than welcome to discuss on the GitHub issue tracker.\n\n\nSubmit Feedback\n-----------------\nThe best way to send feedback is to file an issue at https://github.com/eyaltrabelsi/pandas-log/issues.\n\nIf you are proposing a feature:\n\n* Explain in detail how it would work.\n* Keep the scope as narrow as possible, to make it easier to implement.\n* Remember that this is a volunteer-driven project, and that contributions\n are welcome :)\n\n\n\n\n\nGet Started!\n====================\n\nReady to contribute? Here's how to set up ``pandas_log`` for local development.\n\n1. Fork the `pandas_log` repo on GitHub: https://github.com/eyaltrabelsi/pandas-log.\n\n2. Clone your fork locally::\n\n $ git clone git@github.com:your_name_here/pandas_log.git\n\n3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::\n\n $ mkvirtualenv pandas_log\n $ cd pandas_log/\n $ python setup.py develop\n\n4. Create a branch for local development::\n\n $ git checkout -b name-of-your-bugfix-or-feature\n\n Now you can make your changes locally.\n\n5. When you're done making changes, check that your changes pass flake8 and the\n tests, including testing other Python versions with tox::\n\n $ flake8 pandas_log tests\n $ python setup.py test or pytest\n $ tox\n\n To get flake8 and tox, just pip install them into your virtualenv.\n\n6. Commit your changes and push your branch to GitHub::\n\n $ git add .\n $ git commit -m \"Your detailed description of your changes.\"\n $ git push origin name-of-your-bugfix-or-feature\n\n7. Submit a pull request through the GitHub website.\n\n\nPull Request Guidelines\n----------------------\nBefore you submit a pull request, check that it meets these guidelines:\n\n1. The pull request should include tests.\n2. If the pull request adds functionality, the docs should be updated. Put\n your new functionality into a function with a docstring, and add the\n feature to the list in README.rst.\n\n\n\nDeploying\n---------\nA reminder for the maintainers on how to deploy.\nMake sure all your changes are committed (including an entry in HISTORY.rst).\nThen run::\n\n$ bump2version patch # possible: major / minor / patch\n$ git push\n$ git push --tags\n\nTravis will then deploy to PyPI if tests pass.\n" }, { "path": "HISTORY.rst", "content": "=======\nHistory\n=======\n\n0.0.0 (2019-09-18)\n------------------\n\n* First release on PyPI.\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2019, Eyal Trabelsi\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n\n" }, { "path": "MANIFEST.in", "content": "include CONTRIBUTING.rst\ninclude LICENSE\ninclude README.rst\n\nrecursive-include tests *\nrecursive-exclude * __pycache__\nrecursive-exclude * *.py[co]\n\nrecursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif\n" }, { "path": "Makefile", "content": "cat.PHONY: clean clean-test clean-pyc clean-build docs help\n.DEFAULT_GOAL := help\n\ndefine BROWSER_PYSCRIPT\nimport os, webbrowser, sys\n\ntry:\n\tfrom urllib import pathname2url\nexcept:\n\tfrom urllib.request import pathname2url\n\nwebbrowser.open(\"file://\" + pathname2url(os.path.abspath(sys.argv[1])))\nendef\nexport BROWSER_PYSCRIPT\n\ndefine PRINT_HELP_PYSCRIPT\nimport re, sys\n\nfor line in sys.stdin:\nmake\n\tif match:\n\t\ttarget, help = match.groups()\n\t\tprint(\"%-20s %s\" % (target, help))\nendef\nexport PRINT_HELP_PYSCRIPT\n\nBROWSER := python -c \"$$BROWSER_PYSCRIPT\"\n\nhelp:\n\t@python -c \"$$PRINT_HELP_PYSCRIPT\" < $(MAKEFILE_LIST)\n\nclean:\n\tfind . -name '*.pyc' -exec rm -f {} +\n\tfind . -name '*.pyo' -exec rm -f {} +\n\tfind . -name '*~' -exec rm -f {} +\n\tfind . -name '__pycache__' -exec rm -fr {} +\n\trm -fr .tox/\n\trm -f .coverage\n\trm -fr htmlcov/\n\trm -fr .pytest_cache\n\trm -fr build/\n\trm -fr dist/\n\trm -fr .eggs/\n\tfind . -name '*.egg-info' -exec rm -fr {} +\n\tfind . -name '*.egg' -exec rm -f {} +\n\nformat:\n\tisort -rc pandas_log -y -up -tc\n\tblack -l 79 pandas_log\n\ntest: ## run tests on every Python version with tox\n\ttox\n\ndocs: ## generate Sphinx HTML documentation, including API docs\n\trm -f docs/pandas_log.rst\n\trm -f docs/modules.rst\n\tsphinx-apidoc -o docs/ pandas_log\n\t$(MAKE) -C docs clean\n\t$(MAKE) -C docs html\n\t$(BROWSER) docs/_build/html/index.html\n\nrelease: dist ## package and upload a release\n\ttwine upload dist/*\n\ndist: clean ## builds source and wheel package\n\tpython setup.py sdist\n\tpython setup.py bdist_wheel\n\tls -l dist\n" }, { "path": "README.rst", "content": "==========\npandas-log\n==========\n\n\n.. image:: https://img.shields.io/pypi/v/pandas_log.svg\n :target: https://pypi.python.org/pypi/pandas_log\n\n.. image:: https://img.shields.io/travis/eyaltrabelsi/pandas-log.svg\n :target: https://travis-ci.org/eyaltrabelsi/pandas-log\n\n.. image:: https://readthedocs.org/projects/pandas-log/badge/?version=latest\n :target: https://pandas-log.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. image:: https://pyup.io/repos/github/eyaltrabelsi/pandas-log/shield.svg\n :target: https://pyup.io/repos/github/eyaltrabelsi/pandas-log/\n :alt: Updates\n\nThe goal of pandas-log is to provide feedback about basic pandas operations. It provides simple wrapper functions for the most common functions, such as ``.query``, ``.apply``, ``.merge``, ``.group_by`` and more.\n\nWhy pandas-log?\n---------------\n``Pandas-log`` is a Python implementation of the R package ``tidylog``, and provides a feedback about basic pandas operations.\n\nThe pandas has been invaluable for the data science ecosystem and usually consists of a series of steps that involve transforming raw data into an understandable/usable format.\nThese series of steps need to be run in a certain sequence and if the result is unexpected it's hard to understand what happened. ``Pandas-log`` log metadata on each operation which will allow to pinpoint the issues.\n\n\n\nLets look at an example, first we need to load ``pandas-log`` after ``pandas`` and create a dataframe:\n\n.. code-block:: python\n\n import pandas\n import pandas_log\n\n with pandas_log.enable():\n df = pd.DataFrame({\"name\": ['Alfred', 'Batman', 'Catwoman'],\n \"toy\": [np.nan, 'Batmobile', 'Bullwhip'],\n \"born\": [pd.NaT, pd.Timestamp(\"1940-04-25\"), pd.NaT]})\n\n\n``pandas-log`` will give you feedback, for instance when filtering a data frame or adding a new variable:\n\n.. code-block:: python\n\n df.assign(toy=lambda x: x.toy.map(str.lower))\n .query(\"name != 'Batman'\")\n\n``pandas-log`` can be especially helpful in longer pipes:\n\n.. code-block:: python\n\n df.assign(toy=lambda x: x.toy.map(str.lower))\n .query(\"name != 'Batman'\")\n .dropna()\\\n .assign(lower_name=lambda x: x.name.map(str.lower))\n .reset_index()\n\nFor medium article `go here\n`_\n\nFor a full walkthrough `go here\n`_\n\n\nInstallation\n------------\n``pandas-log`` is currently installable from PyPI:\n\n.. code-block:: bash\n\n pip install pandas-log\n\n\nContributing\n------------\nFollow `contribution docs\n`_ for a full description of the process of contributing to ``pandas-log``.\n" }, { "path": "TODO.rst", "content": "- Add statistics from tidylog from it.\n- add formats of warnings.\n- todo check if copy is first\n- add logs for series as well." }, { "path": "docs/Makefile", "content": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD = python -msphinx\nSPHINXPROJ = pandas_log\nSOURCEDIR = .\nBUILDDIR = _build\n\n# Put it first so that \"make\" without argument is like \"make help\".\nhelp:\n\t@$(SPHINXBUILD) -M help \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n\n.PHONY: help Makefile\n\n# Catch-all target: route all unknown targets to Sphinx using the new\n# \"make mode\" option. $(O) is meant as a shortcut for $(SPHINXOPTS).\n%: Makefile\n\t@$(SPHINXBUILD) -M $@ \"$(SOURCEDIR)\" \"$(BUILDDIR)\" $(SPHINXOPTS) $(O)\n" }, { "path": "docs/conf.py", "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n#\n# pandas_log documentation build configuration file, created by\n# sphinx-quickstart on Fri Jun 9 13:47:02 2017.\n#\n# This file is execfile()d with the current directory set to its\n# containing dir.\n#\n# Note that not all possible configuration values are present in this\n# autogenerated file.\n#\n# All configuration values have a default; values that are commented out\n# serve to show the default.\n\n# If extensions (or modules to document with autodoc) are in another\n# directory, add these directories to sys.path here. If the directory is\n# relative to the documentation root, use os.path.abspath to make it\n# absolute, like shown here.\n#\nimport os\nimport sys\nsys.path.insert(0, os.path.abspath('..'))\n\n# import pandas_log\n\n# -- General configuration ---------------------------------------------\n\n# If your documentation needs a minimal Sphinx version, state it here.\n#\n# needs_sphinx = '1.0'\n\n# Add any Sphinx extension module names here, as strings. They can be\n# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.\nextensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode']\n\n# Add any paths that contain templates here, relative to this directory.\ntemplates_path = ['_templates']\n\n# The suffix(es) of source filenames.\n# You can specify multiple suffix as a list of string:\n#\n# source_suffix = ['.rst', '.md']\nsource_suffix = '.rst'\n\n# The master toctree document.\nmaster_doc = 'index'\n\n# General information about the project.\nproject = u'pandas-log'\ncopyright = u\"2019, Eyal Trabelsi\"\nauthor = u\"Eyal Trabelsi\"\n\n# The version info for the project you're documenting, acts as replacement\n# for |version| and |release|, also used in various other places throughout\n# the built documents.\n#\n# The short X.Y version.\nversion = '0.1.1'\n# The full version, including alpha/beta/rc tags.\nrelease = '0.1.1'\n\n# The language for content autogenerated by Sphinx. Refer to documentation\n# for a list of supported languages.\n#\n# This is also used if you do content translation via gettext catalogs.\n# Usually you set \"language\" from the command line for these cases.\nlanguage = None\n\n# List of patterns, relative to source directory, that match files and\n# directories to ignore when looking for source files.\n# This patterns also effect to html_static_path and html_extra_path\nexclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']\n\n# The name of the Pygments (syntax highlighting) style to use.\npygments_style = 'sphinx'\n\n# If true, `todo` and `todoList` produce output, else they produce nothing.\ntodo_include_todos = False\n\n\n# -- Options for HTML output -------------------------------------------\n\n# The theme to use for HTML and HTML Help pages. See the documentation for\n# a list of builtin themes.\n#\nhtml_theme = 'alabaster'\n\n# Theme options are theme-specific and customize the look and feel of a\n# theme further. For a list of options available for each theme, see the\n# documentation.\n#\n# html_theme_options = {}\n\n# Add any paths that contain custom static files (such as style sheets) here,\n# relative to this directory. They are copied after the builtin static files,\n# so a file named \"default.css\" will overwrite the builtin \"default.css\".\nhtml_static_path = ['_static']\n\n\n# -- Options for HTMLHelp output ---------------------------------------\n\n# Output file base name for HTML help builder.\nhtmlhelp_basename = 'pandas_logdoc'\n\n\n# -- Options for LaTeX output ------------------------------------------\n\nlatex_elements = {\n # The paper size ('letterpaper' or 'a4paper').\n #\n # 'papersize': 'letterpaper',\n\n # The font size ('10pt', '11pt' or '12pt').\n #\n # 'pointsize': '10pt',\n\n # Additional stuff for the LaTeX preamble.\n #\n # 'preamble': '',\n\n # Latex figure (float) alignment\n #\n # 'figure_align': 'htbp',\n}\n\n# Grouping the document tree into LaTeX files. List of tuples\n# (source start file, target name, title, author, documentclass\n# [howto, manual, or own class]).\nlatex_documents = [\n (master_doc, 'pandas_log.tex',\n u'pandas-log Documentation',\n u'Eyal Trabelsi', 'manual'),\n]\n\n\n# -- Options for manual page output ------------------------------------\n\n# One entry per manual page. List of tuples\n# (source start file, name, description, authors, manual section).\nman_pages = [\n (master_doc, 'pandas_log',\n u'pandas-log Documentation',\n [author], 1)\n]\n\n\n# -- Options for Texinfo output ----------------------------------------\n\n# Grouping the document tree into Texinfo files. List of tuples\n# (source start file, target name, title, author,\n# dir menu entry, description, category)\ntexinfo_documents = [\n (master_doc, 'pandas_log',\n u'pandas-log Documentation',\n author,\n 'pandas_log',\n 'One line description of project.',\n 'Miscellaneous'),\n]\n\n\n\n" }, { "path": "docs/contributing.rst", "content": ".. include:: ../CONTRIBUTING.rst\n" }, { "path": "docs/index.rst", "content": "Welcome to pandas-log's documentation!\n======================================\n\n.. toctree::\n :maxdepth: 2\n :caption: Contents:\n\n readme\n installation\n usage\n modules\n contributing\n\nIndices and tables\n==================\n* :ref:`genindex`\n* :ref:`modindex`\n* :ref:`search`\n" }, { "path": "docs/installation.rst", "content": ".. highlight:: shell\n\n============\nInstallation\n============\n\n\nStable release\n--------------\n\nTo install pandas-log, run this command in your terminal:\n\n.. code-block:: console\n\n $ pip install pandas-log\n\nThis is the preferred method to install pandas-log, as it will always install the most recent stable release.\n\nIf you don't have `pip`_ installed, this `Python installation guide`_ can guide\nyou through the process.\n\n.. _pip: https://pip.pypa.io\n.. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/\n\n\nFrom sources\n------------\n\nThe sources for pandas-log can be downloaded from the `Github repo`_.\n\nYou can either clone the public repository:\n\n.. code-block:: console\n\n $ git clone git://github.com/eyaltrabelsi/pandas-log\n\nOr download the `tarball`_:\n\n.. code-block:: console\n\n $ curl -OJL https://github.com/eyaltrabelsi/pandas-log/tarball/master\n\nOnce you have a copy of the source, you can install it with:\n\n.. code-block:: console\n\n $ python setup.py install\n\n\n.. _Github repo: https://github.com/eyaltrabelsi/pandas-log\n.. _tarball: https://github.com/eyaltrabelsi/pandas-log/tarball/master\n" }, { "path": "docs/modules.rst", "content": "pandas_log\n==========\n\n.. toctree::\n :maxdepth: 4\n\n pandas_log\n" }, { "path": "docs/pandas_log.rst", "content": "pandas\\_log package\n===================\n\nSubmodules\n----------\n\npandas\\_log.aop\\_utils module\n-----------------------------\n\n.. automodule:: pandas_log.aop_utils\n :members:\n :undoc-members:\n :show-inheritance:\n\npandas\\_log.dataframe\\_logger module\n------------------------------------\n\n.. automodule:: pandas_log.dataframe_logger\n :members:\n :undoc-members:\n :show-inheritance:\n\npandas\\_log.pandas\\_log module\n------------------------------\n\n.. automodule:: pandas_log.pandas_log\n :members:\n :undoc-members:\n :show-inheritance:\n\npandas\\_log.patched\\_logs\\_functions module\n-------------------------------------------\n\n.. automodule:: pandas_log.patched_logs_functions\n :members:\n :undoc-members:\n :show-inheritance:\n\npandas\\_log.settings module\n---------------------------\n\n.. automodule:: pandas_log.settings\n :members:\n :undoc-members:\n :show-inheritance:\n\npandas\\_log.timer module\n------------------------\n\n.. automodule:: pandas_log.timer\n :members:\n :undoc-members:\n :show-inheritance:\n\n\nModule contents\n---------------\n\n.. automodule:: pandas_log\n :members:\n :undoc-members:\n :show-inheritance:\n" }, { "path": "docs/readme.rst", "content": "===============\nWhy pandas-log?\n===============\n\n``Pandas-log`` is a Python implementation of the R package ``tidylog``, and provides a feedback about basic pandas operations.\n\nThe pandas has been invaluable for the data science ecosystem and usually consists of a series of steps that involve transforming raw data into an understandable/usable format.\nThese series of steps need to be run in a certain sequence and if the result is unexpected it's hard to understand what happened.\n``Pandas-log`` log metadata on each operation which will allow to pinpoint the issues like:\n\n - Wrong predicate expressions.\n - Copying of our DataFrames.\n - Wrong joins/merge.\n - Performance Issues.\n - More\n\nFor medium article `go here\n`_\n\nFor a full walkthrough `go here\n`_\n" }, { "path": "docs/usage.rst", "content": "\nUsage\n=====\n\nFor a cleaner use-case I would `go here\n`_\n\nFirst we need to load some libraries including pandas-log\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: ipython3\n\n import pandas as pd\n import numpy as np\n import pandas_log\n\nLet’s take a look at our dataset:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: ipython3\n\n df = pd.read_csv(\"pokemon.csv\")\n df.head(10)\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgenerationlegendary
01BulbasaurGrassPoison3184549496565451False
12IvysaurGrassPoison4056062638080601False
23VenusaurGrassPoison525808283100100801False
33VenusaurMega VenusaurGrassPoison62580100123122120801False
44CharmanderFireNaN3093952436050651False
55CharmeleonFireNaN4055864588065801False
66CharizardFireFlying534788478109851001False
76CharizardMega Charizard XFireDragon63478130111130851001False
86CharizardMega Charizard YFireFlying63478104781591151001False
97SquirtleWaterNaN3144448655064431False
\n
\n\n\n\nLets say we want to find out:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWho is the weakest non-legendary fire pokemon?\n----------------------------------------------\n\n\n\nThe strategy will probably be something like:\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n1. Filter out legendary pokemons using ``.query()`` .\n2. Keep only fire pokemons using ``.query()`` .\n3. Drop Legendary column using ``.drop()`` .\n4. Keep the weakest pokemon among them using ``.nsmallest()`` .\n5. Reset index using ``.reset_index()`` .\n\n.. code:: ipython3\n\n res = (df.copy()\n .query(\"legendary==0\")\n .query(\"type_1=='fire' or type_2=='fire'\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n .reset_index(drop=True)\n )\n res\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
\n
\n\n\n\nOH NOO!!! Our code does not work !! We got no records\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n\nIf only there was a way to track those issue\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nFortunetly thats what **pandas-log** is for! either as a global function\nor context manager. This the example with pandas_log’s\n``context_manager``.\n\n.. code:: ipython3\n\n with pandas_log.enable():\n res = (df.query(\"legendary==0\")\n .query(\"type_1=='fire' or type_2=='fire'\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n )\n res\n\n\n.. parsed-literal::\n\n\n 1) \u001b[1mquery\u001b[0m(expr=\"legendary==0\", inplace=False):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 65 rows (8.125%), 735 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 199.4 kB.\n \t* Output Dataframe size is 188.5 kB.\n\n 2) \u001b[1mquery\u001b[0m(expr=\"type_1=='fire' or type_2=='fire'\", inplace=False):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 735 rows (100.0%), 0 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 188.5 kB.\n \t* Output Dataframe size is 0 Bytes.\n\n 3) \u001b[1mdrop\u001b[0m(labels=\"legendary\", axis=0, index=None, columns=None, level=None, inplace=False, errors='raise'):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed the following columns (legendary) now only have the following columns (attack, sp_def, speed, hp, total, type_2, #, name, type_1, generation, defense, sp_atk).\n \t* No change in number of rows of input df.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 0 Bytes.\n \t* Output Dataframe size is 0 Bytes.\n\n 4) \u001b[1mnsmallest\u001b[0m(n=1, columns=\"total\", keep='first'):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Picked 1 smallest rows by columns (total).\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 0 Bytes.\n \t* Output Dataframe size is 0 Bytes.\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
\n
\n\n\n\nWe can see clearly that in the second step (``.query()``) we filter all the rows!! and indeed we should of writen Fire as oppose to fire\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: ipython3\n\n res = (df.copy()\n .query(\"type_1=='Fire' or type_2=='Fire'\")\n .query(\"legendary==0\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n .reset_index(drop=True)\n )\n res\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\n
\n\n\n\nWhoala we got Slugma !!!!!!!!\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n\n\nSome more advance usage\n-----------------------\n\nOne can use verbose variable which allows lower level logs functionalities like whether the dataframe was copied as part of pipeline.\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThis can explain comparision issues.\n\n.. code:: ipython3\n\n with pandas_log.enable(verbose=True):\n res = (df.query(\"legendary==0\")\n .query(\"type_1=='Fire' or type_2=='Fire'\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n .reset_index(drop=True)\n )\n res\n\n\n.. parsed-literal::\n\n\n 1) \u001b[1mquery\u001b[0m(expr=\"legendary==0\", inplace=False):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 65 rows (8.125%), 735 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 199.4 kB.\n \t* Output Dataframe size is 188.5 kB.\n\n 2) \u001b[1mquery\u001b[0m(expr=\"type_1=='Fire' or type_2=='Fire'\", inplace=False):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 679 rows (92.38095238095238%), 56 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 188.5 kB.\n \t* Output Dataframe size is 14.4 kB.\n\n 3) \u001b[1mdrop\u001b[0m(labels=\"legendary\", axis=0, index=None, columns=None, level=None, inplace=False, errors='raise'):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed the following columns (legendary) now only have the following columns (attack, sp_def, speed, hp, total, type_2, #, name, type_1, generation, defense, sp_atk).\n \t* No change in number of rows of input df.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.4 kB.\n \t* Output Dataframe size is 14.3 kB.\n\n X) \u001b[1m__getitem__\u001b[0m(key=\"total\"):\n \tMetadata:\n\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.3 kB.\n \t* Output Dataframe size is 896 Bytes.\n\n X) \u001b[1mcopy\u001b[0m(deep=True):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Using default strategy (some metric might not be relevant).\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.3 kB.\n \t* Output Dataframe size is 14.3 kB.\n\n X) \u001b[1mreset_index\u001b[0m(level=None, drop=False, inplace=False, col_level=0, col_fill=''):\n \tMetadata:\n\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.3 kB.\n \t* Output Dataframe size is 14.0 kB.\n\n X) \u001b[1m__getitem__\u001b[0m(key=\"total\"):\n \tMetadata:\n\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.0 kB.\n \t* Output Dataframe size is 576 Bytes.\n\n 4) \u001b[1mnsmallest\u001b[0m(n=1, columns=\"total\", keep='first'):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Picked 1 smallest rows by columns (total).\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.3 kB.\n \t* Output Dataframe size is 236 Bytes.\n\n X) \u001b[1mcopy\u001b[0m(deep=True):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Using default strategy (some metric might not be relevant).\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 236 Bytes.\n \t* Output Dataframe size is 236 Bytes.\n\n X) \u001b[1mreset_index\u001b[0m(level=None, drop=False, inplace=False, col_level=0, col_fill=''):\n \tMetadata:\n\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 236 Bytes.\n \t* Output Dataframe size is 356 Bytes.\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\n
\n\n\n\nas we can see after both the drop and nsmallest functions the dataframe\nwas being copied\n\nOne can use silent variable which allows to suppress stdout\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: ipython3\n\n with pandas_log.enable(silent=True):\n res = (df.copy()\n .query(\"legendary==0\")\n .query(\"type_1=='Fire' or type_2=='Fire'\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n .reset_index(drop=True)\n )\n res\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\n
\n\n\n\nOne can use full_signature variable which allows to suppress the signature\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code:: ipython3\n\n with pandas_log.enable(full_signature=False):\n res = (df.query(\"legendary==0\")\n .query(\"type_1=='Fire' or type_2=='Fire'\")\n .drop(\"legendary\", axis=1)\n .nsmallest(1,\"total\")\n .reset_index(drop=True)\n )\n res\n\n\n.. parsed-literal::\n\n\n 1) \u001b[1mquery\u001b[0m(expr=\"legendary==0\", inplace=False):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 65 rows (8.125%), 735 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 199.4 kB.\n \t* Output Dataframe size is 188.5 kB.\n\n 2) \u001b[1mquery\u001b[0m(expr=\"type_1=='Fire' or type_2=='Fire'\"):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed 679 rows (92.38095238095238%), 56 rows remaining.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 188.5 kB.\n \t* Output Dataframe size is 14.4 kB.\n\n 3) \u001b[1mdrop\u001b[0m(labels=\"legendary\"):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Removed the following columns (legendary) now only have the following columns (attack, sp_def, speed, hp, total, type_2, #, name, type_1, generation, defense, sp_atk).\n \t* No change in number of rows of input df.\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.4 kB.\n \t* Output Dataframe size is 14.3 kB.\n\n 4) \u001b[1mnsmallest\u001b[0m(n=1, columns=\"total\"):\n \t\u001b[4mMetadata\u001b[0m:\n \t* Picked 1 smallest rows by columns (total).\n \t\u001b[4mExecution Stats\u001b[0m:\n \t* Execution time: Step Took a moment seconds..\n \t* Input Dataframe size is 14.3 kB.\n \t* Output Dataframe size is 236 Bytes.\n\n\n\n\n.. raw:: html\n\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\n
\n\n\n" }, { "path": "examples/README.rst", "content": "\nExamples\n========\n\nThis folder contains jupyter notebooks demonstrating different ways to\nimplement pandas-log in your workflow.\n\nGuidelines\n~~~~~~~~~~\n\nWhen contributing example notebooks please include a short explanation of\nwhere the data came from and what it contains" }, { "path": "examples/__init__.py", "content": "" }, { "path": "examples/pandas_log_intro.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Pandas-Log Usage Walkthrough\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Why pandas-log?\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"Pandas-log is a Python implementation of the R package tidylog, and provides a feedback about basic pandas operations.\\n\",\n \"\\n\",\n \"The pandas has been invaluable for the data science ecosystem and usually consists of a series of steps that involve transforming raw data into an understandable/usable format. These series of steps need to be run in a certain sequence and if the result is unexpected it's hard to understand what happened. Pandas-log log metadata on each operation which will allow to pinpoint the issues.\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Pandas-log Demo\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### First we need to load some libraries including pandas-log\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pandas as pd\\n\",\n \"import numpy as np\\n\",\n \"import pandas_log \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Let's take a look at our dataset:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgenerationlegendary
01BulbasaurGrassPoison3184549496565451False
12IvysaurGrassPoison4056062638080601False
23VenusaurGrassPoison525808283100100801False
33VenusaurMega VenusaurGrassPoison62580100123122120801False
44CharmanderFireNaN3093952436050651False
55CharmeleonFireNaN4055864588065801False
66CharizardFireFlying534788478109851001False
76CharizardMega Charizard XFireDragon63478130111130851001False
86CharizardMega Charizard YFireFlying63478104781591151001False
97SquirtleWaterNaN3144448655064431False
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" # name type_1 type_2 total hp attack defense \\\\\\n\",\n \"0 1 Bulbasaur Grass Poison 318 45 49 49 \\n\",\n \"1 2 Ivysaur Grass Poison 405 60 62 63 \\n\",\n \"2 3 Venusaur Grass Poison 525 80 82 83 \\n\",\n \"3 3 VenusaurMega Venusaur Grass Poison 625 80 100 123 \\n\",\n \"4 4 Charmander Fire NaN 309 39 52 43 \\n\",\n \"5 5 Charmeleon Fire NaN 405 58 64 58 \\n\",\n \"6 6 Charizard Fire Flying 534 78 84 78 \\n\",\n \"7 6 CharizardMega Charizard X Fire Dragon 634 78 130 111 \\n\",\n \"8 6 CharizardMega Charizard Y Fire Flying 634 78 104 78 \\n\",\n \"9 7 Squirtle Water NaN 314 44 48 65 \\n\",\n \"\\n\",\n \" sp_atk sp_def speed generation legendary \\n\",\n \"0 65 65 45 1 False \\n\",\n \"1 80 80 60 1 False \\n\",\n \"2 100 100 80 1 False \\n\",\n \"3 122 120 80 1 False \\n\",\n \"4 60 50 65 1 False \\n\",\n \"5 80 65 80 1 False \\n\",\n \"6 109 85 100 1 False \\n\",\n \"7 130 85 100 1 False \\n\",\n \"8 159 115 100 1 False \\n\",\n \"9 50 64 43 1 False \"\n ]\n },\n \"execution_count\": 2,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"df = pd.read_csv(\\\"pokemon.csv\\\")\\n\",\n \"df.head(10)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### Lets say we want to find out:\\n\",\n \"## Who is the weakest non-legendary fire pokemon?\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### The strategy will probably be something like:\\n\",\n \"1. Filter out legendary pokemons using `.query()` .\\n\",\n \"1. Keep only fire pokemons using `.query()` .\\n\",\n \"1. Drop Legendary column using `.drop()` .\\n\",\n \"1. Keep the weakest pokemon among them using `.nsmallest()` .\\n\",\n \"1. Reset index using `.reset_index()` .\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \"Empty DataFrame\\n\",\n \"Columns: [#, name, type_1, type_2, total, hp, attack, defense, sp_atk, sp_def, speed, generation]\\n\",\n \"Index: []\"\n ]\n },\n \"execution_count\": 3,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"res = (df.copy()\\n\",\n \" .query(\\\"legendary==0\\\")\\n\",\n \" .query(\\\"type_1=='fire' or type_2=='fire'\\\")\\n\",\n \" .drop(\\\"legendary\\\", axis=1)\\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" .reset_index(drop=True)\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### OH NOO!!! Our code does not work !! We got no records\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### If only there was a way to track those issue\\n\",\n \"\\n\",\n \"Fortunetly thats what **pandas-log** is for! either as a global function or context manager.\\n\",\n \"This the example with pandas_log's `context_manager`.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"1) \\u001b[1mquery\\u001b[0m(expr=\\\"legendary==0\\\", inplace=False):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 65 rows (8.125%), 735 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.002567 seconds.\\n\",\n \"\\t* Input Dataframe size is 199.4 kB.\\n\",\n \"\\t* Output Dataframe size is 188.5 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"2) \\u001b[1mquery\\u001b[0m(expr=\\\"type_1=='fire' or type_2=='fire'\\\", inplace=False):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 735 rows (100.0%), 0 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.003322 seconds.\\n\",\n \"\\t* Input Dataframe size is 188.5 kB.\\n\",\n \"\\t* Output Dataframe size is 0 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"3) \\u001b[1mdrop\\u001b[0m(labels=\\\"legendary\\\", axis=0, index=None, columns=None, level=None, inplace=False, errors='raise'):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* No change in number of rows of input df.\\n\",\n \"\\t* Removed the following columns (legendary) now only have the following columns (type_2, total, hp, defense, sp_def, speed, generation, sp_atk, type_1, attack, name, #).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.00073 seconds.\\n\",\n \"\\t* Input Dataframe size is 0 Bytes.\\n\",\n \"\\t* Output Dataframe size is 0 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"4) \\u001b[1mnsmallest\\u001b[0m(n=1, columns=\\\"total\\\", keep='first'):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Picked 1 smallest rows by columns (total).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.010866 seconds.\\n\",\n \"\\t* Input Dataframe size is 0 Bytes.\\n\",\n \"\\t* Output Dataframe size is 0 Bytes.\\n\",\n \"\\t\\u001b[4mTips\\u001b[0m:\\n\",\n \"\\t* Number of rows didn't change, if you are working on the entire dataset you can remove this operation.\\n\"\n ]\n },\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \"Empty DataFrame\\n\",\n \"Columns: [#, name, type_1, type_2, total, hp, attack, defense, sp_atk, sp_def, speed, generation]\\n\",\n \"Index: []\"\n ]\n },\n \"execution_count\": 4,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"with pandas_log.enable():\\n\",\n \" res = (df.query(\\\"legendary==0\\\")\\n\",\n \" .query(\\\"type_1=='fire' or type_2=='fire'\\\")\\n\",\n \" .drop(\\\"legendary\\\", axis=1)\\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### We can see clearly that in the second step (`.query()`) we filter all the rows!! and indeed we should of writen Fire as oppose to fire\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" # name type_1 type_2 total hp attack defense sp_atk sp_def \\\\\\n\",\n \"0 218 Slugma Fire NaN 250 40 40 40 70 40 \\n\",\n \"\\n\",\n \" speed generation \\n\",\n \"0 20 2 \"\n ]\n },\n \"execution_count\": 5,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"res = (df.copy()\\n\",\n \" .query(\\\"type_1=='Fire' or type_2=='Fire'\\\")\\n\",\n \" .query(\\\"legendary==0\\\") \\n\",\n \" .drop(\\\"legendary\\\", axis=1) \\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" .reset_index(drop=True)\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Whoala we got Slugma !!!!!!!!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Some more advance usage\\n\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### One can use verbose variable which allows lower level logs functionalities like whether the dataframe was copied as part of pipeline.\\n\",\n \"This can explain comparision issues.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"1) \\u001b[1mquery\\u001b[0m(expr=\\\"legendary==0\\\", inplace=False):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 65 rows (8.125%), 735 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.003288 seconds.\\n\",\n \"\\t* Input Dataframe size is 199.4 kB.\\n\",\n \"\\t* Output Dataframe size is 188.5 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"2) \\u001b[1mquery\\u001b[0m(expr=\\\"type_1=='Fire' or type_2=='Fire'\\\", inplace=False):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 679 rows (92.38095238095238%), 56 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.003339 seconds.\\n\",\n \"\\t* Input Dataframe size is 188.5 kB.\\n\",\n \"\\t* Output Dataframe size is 14.4 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"3) \\u001b[1mdrop\\u001b[0m(labels=\\\"legendary\\\", axis=0, index=None, columns=None, level=None, inplace=False, errors='raise'):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* No change in number of rows of input df.\\n\",\n \"\\t* Removed the following columns (legendary) now only have the following columns (hp, sp_def, generation, name, #, attack, type_2, sp_atk, type_1, defense, total, speed).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.000604 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.4 kB.\\n\",\n \"\\t* Output Dataframe size is 14.3 kB.\\n\",\n \"\\t\\u001b[4mTips\\u001b[0m:\\n\",\n \"\\t* Number of rows didn't change, if you are working on the entire dataset you can remove this operation.\\n\",\n \"\\n\",\n \"X) \\u001b[1m__getitem__\\u001b[0m(key=\\\"total\\\"):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* After transformation we received Series\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 3.3e-05 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.3 kB.\\n\",\n \"\\t* Output Dataframe size is 896 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"X) \\u001b[1mcopy\\u001b[0m(deep=True):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Using default strategy (some metric might not be relevant).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.000318 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.3 kB.\\n\",\n \"\\t* Output Dataframe size is 14.3 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"X) \\u001b[1mreset_index\\u001b[0m(level=None, drop=False, inplace=False, col_level=0, col_fill=''):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Using default strategy (some metric might not be relevant).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.00373 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.3 kB.\\n\",\n \"\\t* Output Dataframe size is 14.0 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"X) \\u001b[1m__getitem__\\u001b[0m(key=\\\"total\\\"):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* After transformation we received Series\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 1.1e-05 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.0 kB.\\n\",\n \"\\t* Output Dataframe size is 576 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"4) \\u001b[1mnsmallest\\u001b[0m(n=1, columns=\\\"total\\\", keep='first'):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Picked 1 smallest rows by columns (total).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.012324 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.3 kB.\\n\",\n \"\\t* Output Dataframe size is 236 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"X) \\u001b[1mcopy\\u001b[0m(deep=True):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Using default strategy (some metric might not be relevant).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.000137 seconds.\\n\",\n \"\\t* Input Dataframe size is 236 Bytes.\\n\",\n \"\\t* Output Dataframe size is 236 Bytes.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"X) \\u001b[1mreset_index\\u001b[0m(level=None, drop=False, inplace=False, col_level=0, col_fill=''):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Using default strategy (some metric might not be relevant).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.002781 seconds.\\n\",\n \"\\t* Input Dataframe size is 236 Bytes.\\n\",\n \"\\t* Output Dataframe size is 356 Bytes.\\n\",\n \"\\t\\n\"\n ]\n },\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" # name type_1 type_2 total hp attack defense sp_atk sp_def \\\\\\n\",\n \"0 218 Slugma Fire NaN 250 40 40 40 70 40 \\n\",\n \"\\n\",\n \" speed generation \\n\",\n \"0 20 2 \"\n ]\n },\n \"execution_count\": 6,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"with pandas_log.enable(verbose=True):\\n\",\n \" res = (df.query(\\\"legendary==0\\\")\\n\",\n \" .query(\\\"type_1=='Fire' or type_2=='Fire'\\\")\\n\",\n \" .drop(\\\"legendary\\\", axis=1)\\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" .reset_index(drop=True)\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"as we can see after both the drop and nsmallest functions the dataframe was being copied\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### One can use silent variable which allows to suppress stdout\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" # name type_1 type_2 total hp attack defense sp_atk sp_def \\\\\\n\",\n \"0 218 Slugma Fire NaN 250 40 40 40 70 40 \\n\",\n \"\\n\",\n \" speed generation \\n\",\n \"0 20 2 \"\n ]\n },\n \"execution_count\": 7,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"with pandas_log.enable(silent=True):\\n\",\n \" res = (df.copy()\\n\",\n \" .query(\\\"legendary==0\\\")\\n\",\n \" .query(\\\"type_1=='Fire' or type_2=='Fire'\\\")\\n\",\n \" .drop(\\\"legendary\\\", axis=1)\\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" .reset_index(drop=True)\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"#### One can use full_signature variable which allows to suppress the signature\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"1) \\u001b[1mquery\\u001b[0m(expr=\\\"legendary==0\\\", inplace=False):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 65 rows (8.125%), 735 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.001866 seconds.\\n\",\n \"\\t* Input Dataframe size is 199.4 kB.\\n\",\n \"\\t* Output Dataframe size is 188.5 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"2) \\u001b[1mquery\\u001b[0m(expr=\\\"type_1=='Fire' or type_2=='Fire'\\\"):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Removed 679 rows (92.38095238095238%), 56 rows remaining.\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.002914 seconds.\\n\",\n \"\\t* Input Dataframe size is 188.5 kB.\\n\",\n \"\\t* Output Dataframe size is 14.4 kB.\\n\",\n \"\\t\\n\",\n \"\\n\",\n \"3) \\u001b[1mdrop\\u001b[0m(labels=\\\"legendary\\\"):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* No change in number of rows of input df.\\n\",\n \"\\t* Removed the following columns (legendary) now only have the following columns (hp, sp_def, generation, name, #, attack, type_2, sp_atk, type_1, defense, total, speed).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.000561 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.4 kB.\\n\",\n \"\\t* Output Dataframe size is 14.3 kB.\\n\",\n \"\\t\\u001b[4mTips\\u001b[0m:\\n\",\n \"\\t* Number of rows didn't change, if you are working on the entire dataset you can remove this operation.\\n\",\n \"\\n\",\n \"4) \\u001b[1mnsmallest\\u001b[0m(n=1, columns=\\\"total\\\"):\\n\",\n \"\\t\\u001b[4mMetadata\\u001b[0m:\\n\",\n \"\\t* Picked 1 smallest rows by columns (total).\\n\",\n \"\\t\\u001b[4mExecution Stats\\u001b[0m:\\n\",\n \"\\t* Execution time: Step Took 0.008674 seconds.\\n\",\n \"\\t* Input Dataframe size is 14.3 kB.\\n\",\n \"\\t* Output Dataframe size is 236 Bytes.\\n\",\n \"\\t\\n\"\n ]\n },\n {\n \"data\": {\n \"text/html\": [\n \"
\\n\",\n \"\\n\",\n \"\\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \" \\n\",\n \"
#nametype_1type_2totalhpattackdefensesp_atksp_defspeedgeneration
0218SlugmaFireNaN2504040407040202
\\n\",\n \"
\"\n ],\n \"text/plain\": [\n \" # name type_1 type_2 total hp attack defense sp_atk sp_def \\\\\\n\",\n \"0 218 Slugma Fire NaN 250 40 40 40 70 40 \\n\",\n \"\\n\",\n \" speed generation \\n\",\n \"0 20 2 \"\n ]\n },\n \"execution_count\": 8,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"with pandas_log.enable(full_signature=False):\\n\",\n \" res = (df.query(\\\"legendary==0\\\")\\n\",\n \" .query(\\\"type_1=='Fire' or type_2=='Fire'\\\")\\n\",\n \" .drop(\\\"legendary\\\", axis=1)\\n\",\n \" .nsmallest(1,\\\"total\\\")\\n\",\n \" .reset_index(drop=True)\\n\",\n \" )\\n\",\n \"res \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.4\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 2\n}\n" }, { "path": "examples/pokemon.csv", "content": "#,name,type_1,type_2,total,hp,attack,defense,sp_atk,sp_def,speed,generation,legendary\n1,Bulbasaur,Grass,Poison,318,45,49,49,65,65,45,1,False\n2,Ivysaur,Grass,Poison,405,60,62,63,80,80,60,1,False\n3,Venusaur,Grass,Poison,525,80,82,83,100,100,80,1,False\n3,VenusaurMega Venusaur,Grass,Poison,625,80,100,123,122,120,80,1,False\n4,Charmander,Fire,,309,39,52,43,60,50,65,1,False\n5,Charmeleon,Fire,,405,58,64,58,80,65,80,1,False\n6,Charizard,Fire,Flying,534,78,84,78,109,85,100,1,False\n6,CharizardMega Charizard X,Fire,Dragon,634,78,130,111,130,85,100,1,False\n6,CharizardMega Charizard Y,Fire,Flying,634,78,104,78,159,115,100,1,False\n7,Squirtle,Water,,314,44,48,65,50,64,43,1,False\n8,Wartortle,Water,,405,59,63,80,65,80,58,1,False\n9,Blastoise,Water,,530,79,83,100,85,105,78,1,False\n9,BlastoiseMega Blastoise,Water,,630,79,103,120,135,115,78,1,False\n10,Caterpie,Bug,,195,45,30,35,20,20,45,1,False\n11,Metapod,Bug,,205,50,20,55,25,25,30,1,False\n12,Butterfree,Bug,Flying,395,60,45,50,90,80,70,1,False\n13,Weedle,Bug,Poison,195,40,35,30,20,20,50,1,False\n14,Kakuna,Bug,Poison,205,45,25,50,25,25,35,1,False\n15,Beedrill,Bug,Poison,395,65,90,40,45,80,75,1,False\n15,BeedrillMega Beedrill,Bug,Poison,495,65,150,40,15,80,145,1,False\n16,Pidgey,Normal,Flying,251,40,45,40,35,35,56,1,False\n17,Pidgeotto,Normal,Flying,349,63,60,55,50,50,71,1,False\n18,Pidgeot,Normal,Flying,479,83,80,75,70,70,101,1,False\n18,PidgeotMega Pidgeot,Normal,Flying,579,83,80,80,135,80,121,1,False\n19,Rattata,Normal,,253,30,56,35,25,35,72,1,False\n20,Raticate,Normal,,413,55,81,60,50,70,97,1,False\n21,Spearow,Normal,Flying,262,40,60,30,31,31,70,1,False\n22,Fearow,Normal,Flying,442,65,90,65,61,61,100,1,False\n23,Ekans,Poison,,288,35,60,44,40,54,55,1,False\n24,Arbok,Poison,,438,60,85,69,65,79,80,1,False\n25,Pikachu,Electric,,320,35,55,40,50,50,90,1,False\n26,Raichu,Electric,,485,60,90,55,90,80,110,1,False\n27,Sandshrew,Ground,,300,50,75,85,20,30,40,1,False\n28,Sandslash,Ground,,450,75,100,110,45,55,65,1,False\n29,Nidoran♀,Poison,,275,55,47,52,40,40,41,1,False\n30,Nidorina,Poison,,365,70,62,67,55,55,56,1,False\n31,Nidoqueen,Poison,Ground,505,90,92,87,75,85,76,1,False\n32,Nidoran♂,Poison,,273,46,57,40,40,40,50,1,False\n33,Nidorino,Poison,,365,61,72,57,55,55,65,1,False\n34,Nidoking,Poison,Ground,505,81,102,77,85,75,85,1,False\n35,Clefairy,Fairy,,323,70,45,48,60,65,35,1,False\n36,Clefable,Fairy,,483,95,70,73,95,90,60,1,False\n37,Vulpix,Fire,,299,38,41,40,50,65,65,1,False\n38,Ninetales,Fire,,505,73,76,75,81,100,100,1,False\n39,Jigglypuff,Normal,Fairy,270,115,45,20,45,25,20,1,False\n40,Wigglytuff,Normal,Fairy,435,140,70,45,85,50,45,1,False\n41,Zubat,Poison,Flying,245,40,45,35,30,40,55,1,False\n42,Golbat,Poison,Flying,455,75,80,70,65,75,90,1,False\n43,Oddish,Grass,Poison,320,45,50,55,75,65,30,1,False\n44,Gloom,Grass,Poison,395,60,65,70,85,75,40,1,False\n45,Vileplume,Grass,Poison,490,75,80,85,110,90,50,1,False\n46,Paras,Bug,Grass,285,35,70,55,45,55,25,1,False\n47,Parasect,Bug,Grass,405,60,95,80,60,80,30,1,False\n48,Venonat,Bug,Poison,305,60,55,50,40,55,45,1,False\n49,Venomoth,Bug,Poison,450,70,65,60,90,75,90,1,False\n50,Diglett,Ground,,265,10,55,25,35,45,95,1,False\n51,Dugtrio,Ground,,405,35,80,50,50,70,120,1,False\n52,Meowth,Normal,,290,40,45,35,40,40,90,1,False\n53,Persian,Normal,,440,65,70,60,65,65,115,1,False\n54,Psyduck,Water,,320,50,52,48,65,50,55,1,False\n55,Golduck,Water,,500,80,82,78,95,80,85,1,False\n56,Mankey,Fighting,,305,40,80,35,35,45,70,1,False\n57,Primeape,Fighting,,455,65,105,60,60,70,95,1,False\n58,Growlithe,Fire,,350,55,70,45,70,50,60,1,False\n59,Arcanine,Fire,,555,90,110,80,100,80,95,1,False\n60,Poliwag,Water,,300,40,50,40,40,40,90,1,False\n61,Poliwhirl,Water,,385,65,65,65,50,50,90,1,False\n62,Poliwrath,Water,Fighting,510,90,95,95,70,90,70,1,False\n63,Abra,Psychic,,310,25,20,15,105,55,90,1,False\n64,Kadabra,Psychic,,400,40,35,30,120,70,105,1,False\n65,Alakazam,Psychic,,500,55,50,45,135,95,120,1,False\n65,AlakazamMega Alakazam,Psychic,,590,55,50,65,175,95,150,1,False\n66,Machop,Fighting,,305,70,80,50,35,35,35,1,False\n67,Machoke,Fighting,,405,80,100,70,50,60,45,1,False\n68,Machamp,Fighting,,505,90,130,80,65,85,55,1,False\n69,Bellsprout,Grass,Poison,300,50,75,35,70,30,40,1,False\n70,Weepinbell,Grass,Poison,390,65,90,50,85,45,55,1,False\n71,Victreebel,Grass,Poison,490,80,105,65,100,70,70,1,False\n72,Tentacool,Water,Poison,335,40,40,35,50,100,70,1,False\n73,Tentacruel,Water,Poison,515,80,70,65,80,120,100,1,False\n74,Geodude,Rock,Ground,300,40,80,100,30,30,20,1,False\n75,Graveler,Rock,Ground,390,55,95,115,45,45,35,1,False\n76,Golem,Rock,Ground,495,80,120,130,55,65,45,1,False\n77,Ponyta,Fire,,410,50,85,55,65,65,90,1,False\n78,Rapidash,Fire,,500,65,100,70,80,80,105,1,False\n79,Slowpoke,Water,Psychic,315,90,65,65,40,40,15,1,False\n80,Slowbro,Water,Psychic,490,95,75,110,100,80,30,1,False\n80,SlowbroMega Slowbro,Water,Psychic,590,95,75,180,130,80,30,1,False\n81,Magnemite,Electric,Steel,325,25,35,70,95,55,45,1,False\n82,Magneton,Electric,Steel,465,50,60,95,120,70,70,1,False\n83,Farfetch'd,Normal,Flying,352,52,65,55,58,62,60,1,False\n84,Doduo,Normal,Flying,310,35,85,45,35,35,75,1,False\n85,Dodrio,Normal,Flying,460,60,110,70,60,60,100,1,False\n86,Seel,Water,,325,65,45,55,45,70,45,1,False\n87,Dewgong,Water,Ice,475,90,70,80,70,95,70,1,False\n88,Grimer,Poison,,325,80,80,50,40,50,25,1,False\n89,Muk,Poison,,500,105,105,75,65,100,50,1,False\n90,Shellder,Water,,305,30,65,100,45,25,40,1,False\n91,Cloyster,Water,Ice,525,50,95,180,85,45,70,1,False\n92,Gastly,Ghost,Poison,310,30,35,30,100,35,80,1,False\n93,Haunter,Ghost,Poison,405,45,50,45,115,55,95,1,False\n94,Gengar,Ghost,Poison,500,60,65,60,130,75,110,1,False\n94,GengarMega Gengar,Ghost,Poison,600,60,65,80,170,95,130,1,False\n95,Onix,Rock,Ground,385,35,45,160,30,45,70,1,False\n96,Drowzee,Psychic,,328,60,48,45,43,90,42,1,False\n97,Hypno,Psychic,,483,85,73,70,73,115,67,1,False\n98,Krabby,Water,,325,30,105,90,25,25,50,1,False\n99,Kingler,Water,,475,55,130,115,50,50,75,1,False\n100,Voltorb,Electric,,330,40,30,50,55,55,100,1,False\n101,Electrode,Electric,,480,60,50,70,80,80,140,1,False\n102,Exeggcute,Grass,Psychic,325,60,40,80,60,45,40,1,False\n103,Exeggutor,Grass,Psychic,520,95,95,85,125,65,55,1,False\n104,Cubone,Ground,,320,50,50,95,40,50,35,1,False\n105,Marowak,Ground,,425,60,80,110,50,80,45,1,False\n106,Hitmonlee,Fighting,,455,50,120,53,35,110,87,1,False\n107,Hitmonchan,Fighting,,455,50,105,79,35,110,76,1,False\n108,Lickitung,Normal,,385,90,55,75,60,75,30,1,False\n109,Koffing,Poison,,340,40,65,95,60,45,35,1,False\n110,Weezing,Poison,,490,65,90,120,85,70,60,1,False\n111,Rhyhorn,Ground,Rock,345,80,85,95,30,30,25,1,False\n112,Rhydon,Ground,Rock,485,105,130,120,45,45,40,1,False\n113,Chansey,Normal,,450,250,5,5,35,105,50,1,False\n114,Tangela,Grass,,435,65,55,115,100,40,60,1,False\n115,Kangaskhan,Normal,,490,105,95,80,40,80,90,1,False\n115,KangaskhanMega Kangaskhan,Normal,,590,105,125,100,60,100,100,1,False\n116,Horsea,Water,,295,30,40,70,70,25,60,1,False\n117,Seadra,Water,,440,55,65,95,95,45,85,1,False\n118,Goldeen,Water,,320,45,67,60,35,50,63,1,False\n119,Seaking,Water,,450,80,92,65,65,80,68,1,False\n120,Staryu,Water,,340,30,45,55,70,55,85,1,False\n121,Starmie,Water,Psychic,520,60,75,85,100,85,115,1,False\n122,Mr. Mime,Psychic,Fairy,460,40,45,65,100,120,90,1,False\n123,Scyther,Bug,Flying,500,70,110,80,55,80,105,1,False\n124,Jynx,Ice,Psychic,455,65,50,35,115,95,95,1,False\n125,Electabuzz,Electric,,490,65,83,57,95,85,105,1,False\n126,Magmar,Fire,,495,65,95,57,100,85,93,1,False\n127,Pinsir,Bug,,500,65,125,100,55,70,85,1,False\n127,PinsirMega Pinsir,Bug,Flying,600,65,155,120,65,90,105,1,False\n128,Tauros,Normal,,490,75,100,95,40,70,110,1,False\n129,Magikarp,Water,,200,20,10,55,15,20,80,1,False\n130,Gyarados,Water,Flying,540,95,125,79,60,100,81,1,False\n130,GyaradosMega Gyarados,Water,Dark,640,95,155,109,70,130,81,1,False\n131,Lapras,Water,Ice,535,130,85,80,85,95,60,1,False\n132,Ditto,Normal,,288,48,48,48,48,48,48,1,False\n133,Eevee,Normal,,325,55,55,50,45,65,55,1,False\n134,Vaporeon,Water,,525,130,65,60,110,95,65,1,False\n135,Jolteon,Electric,,525,65,65,60,110,95,130,1,False\n136,Flareon,Fire,,525,65,130,60,95,110,65,1,False\n137,Porygon,Normal,,395,65,60,70,85,75,40,1,False\n138,Omanyte,Rock,Water,355,35,40,100,90,55,35,1,False\n139,Omastar,Rock,Water,495,70,60,125,115,70,55,1,False\n140,Kabuto,Rock,Water,355,30,80,90,55,45,55,1,False\n141,Kabutops,Rock,Water,495,60,115,105,65,70,80,1,False\n142,Aerodactyl,Rock,Flying,515,80,105,65,60,75,130,1,False\n142,AerodactylMega Aerodactyl,Rock,Flying,615,80,135,85,70,95,150,1,False\n143,Snorlax,Normal,,540,160,110,65,65,110,30,1,False\n144,Articuno,Ice,Flying,580,90,85,100,95,125,85,1,True\n145,Zapdos,Electric,Flying,580,90,90,85,125,90,100,1,True\n146,Moltres,Fire,Flying,580,90,100,90,125,85,90,1,True\n147,Dratini,Dragon,,300,41,64,45,50,50,50,1,False\n148,Dragonair,Dragon,,420,61,84,65,70,70,70,1,False\n149,Dragonite,Dragon,Flying,600,91,134,95,100,100,80,1,False\n150,Mewtwo,Psychic,,680,106,110,90,154,90,130,1,True\n150,MewtwoMega Mewtwo X,Psychic,Fighting,780,106,190,100,154,100,130,1,True\n150,MewtwoMega Mewtwo Y,Psychic,,780,106,150,70,194,120,140,1,True\n151,Mew,Psychic,,600,100,100,100,100,100,100,1,False\n152,Chikorita,Grass,,318,45,49,65,49,65,45,2,False\n153,Bayleef,Grass,,405,60,62,80,63,80,60,2,False\n154,Meganium,Grass,,525,80,82,100,83,100,80,2,False\n155,Cyndaquil,Fire,,309,39,52,43,60,50,65,2,False\n156,Quilava,Fire,,405,58,64,58,80,65,80,2,False\n157,Typhlosion,Fire,,534,78,84,78,109,85,100,2,False\n158,Totodile,Water,,314,50,65,64,44,48,43,2,False\n159,Croconaw,Water,,405,65,80,80,59,63,58,2,False\n160,Feraligatr,Water,,530,85,105,100,79,83,78,2,False\n161,Sentret,Normal,,215,35,46,34,35,45,20,2,False\n162,Furret,Normal,,415,85,76,64,45,55,90,2,False\n163,Hoothoot,Normal,Flying,262,60,30,30,36,56,50,2,False\n164,Noctowl,Normal,Flying,442,100,50,50,76,96,70,2,False\n165,Ledyba,Bug,Flying,265,40,20,30,40,80,55,2,False\n166,Ledian,Bug,Flying,390,55,35,50,55,110,85,2,False\n167,Spinarak,Bug,Poison,250,40,60,40,40,40,30,2,False\n168,Ariados,Bug,Poison,390,70,90,70,60,60,40,2,False\n169,Crobat,Poison,Flying,535,85,90,80,70,80,130,2,False\n170,Chinchou,Water,Electric,330,75,38,38,56,56,67,2,False\n171,Lanturn,Water,Electric,460,125,58,58,76,76,67,2,False\n172,Pichu,Electric,,205,20,40,15,35,35,60,2,False\n173,Cleffa,Fairy,,218,50,25,28,45,55,15,2,False\n174,Igglybuff,Normal,Fairy,210,90,30,15,40,20,15,2,False\n175,Togepi,Fairy,,245,35,20,65,40,65,20,2,False\n176,Togetic,Fairy,Flying,405,55,40,85,80,105,40,2,False\n177,Natu,Psychic,Flying,320,40,50,45,70,45,70,2,False\n178,Xatu,Psychic,Flying,470,65,75,70,95,70,95,2,False\n179,Mareep,Electric,,280,55,40,40,65,45,35,2,False\n180,Flaaffy,Electric,,365,70,55,55,80,60,45,2,False\n181,Ampharos,Electric,,510,90,75,85,115,90,55,2,False\n181,AmpharosMega Ampharos,Electric,Dragon,610,90,95,105,165,110,45,2,False\n182,Bellossom,Grass,,490,75,80,95,90,100,50,2,False\n183,Marill,Water,Fairy,250,70,20,50,20,50,40,2,False\n184,Azumarill,Water,Fairy,420,100,50,80,60,80,50,2,False\n185,Sudowoodo,Rock,,410,70,100,115,30,65,30,2,False\n186,Politoed,Water,,500,90,75,75,90,100,70,2,False\n187,Hoppip,Grass,Flying,250,35,35,40,35,55,50,2,False\n188,Skiploom,Grass,Flying,340,55,45,50,45,65,80,2,False\n189,Jumpluff,Grass,Flying,460,75,55,70,55,95,110,2,False\n190,Aipom,Normal,,360,55,70,55,40,55,85,2,False\n191,Sunkern,Grass,,180,30,30,30,30,30,30,2,False\n192,Sunflora,Grass,,425,75,75,55,105,85,30,2,False\n193,Yanma,Bug,Flying,390,65,65,45,75,45,95,2,False\n194,Wooper,Water,Ground,210,55,45,45,25,25,15,2,False\n195,Quagsire,Water,Ground,430,95,85,85,65,65,35,2,False\n196,Espeon,Psychic,,525,65,65,60,130,95,110,2,False\n197,Umbreon,Dark,,525,95,65,110,60,130,65,2,False\n198,Murkrow,Dark,Flying,405,60,85,42,85,42,91,2,False\n199,Slowking,Water,Psychic,490,95,75,80,100,110,30,2,False\n200,Misdreavus,Ghost,,435,60,60,60,85,85,85,2,False\n201,Unown,Psychic,,336,48,72,48,72,48,48,2,False\n202,Wobbuffet,Psychic,,405,190,33,58,33,58,33,2,False\n203,Girafarig,Normal,Psychic,455,70,80,65,90,65,85,2,False\n204,Pineco,Bug,,290,50,65,90,35,35,15,2,False\n205,Forretress,Bug,Steel,465,75,90,140,60,60,40,2,False\n206,Dunsparce,Normal,,415,100,70,70,65,65,45,2,False\n207,Gligar,Ground,Flying,430,65,75,105,35,65,85,2,False\n208,Steelix,Steel,Ground,510,75,85,200,55,65,30,2,False\n208,SteelixMega Steelix,Steel,Ground,610,75,125,230,55,95,30,2,False\n209,Snubbull,Fairy,,300,60,80,50,40,40,30,2,False\n210,Granbull,Fairy,,450,90,120,75,60,60,45,2,False\n211,Qwilfish,Water,Poison,430,65,95,75,55,55,85,2,False\n212,Scizor,Bug,Steel,500,70,130,100,55,80,65,2,False\n212,ScizorMega Scizor,Bug,Steel,600,70,150,140,65,100,75,2,False\n213,Shuckle,Bug,Rock,505,20,10,230,10,230,5,2,False\n214,Heracross,Bug,Fighting,500,80,125,75,40,95,85,2,False\n214,HeracrossMega Heracross,Bug,Fighting,600,80,185,115,40,105,75,2,False\n215,Sneasel,Dark,Ice,430,55,95,55,35,75,115,2,False\n216,Teddiursa,Normal,,330,60,80,50,50,50,40,2,False\n217,Ursaring,Normal,,500,90,130,75,75,75,55,2,False\n218,Slugma,Fire,,250,40,40,40,70,40,20,2,False\n219,Magcargo,Fire,Rock,410,50,50,120,80,80,30,2,False\n220,Swinub,Ice,Ground,250,50,50,40,30,30,50,2,False\n221,Piloswine,Ice,Ground,450,100,100,80,60,60,50,2,False\n222,Corsola,Water,Rock,380,55,55,85,65,85,35,2,False\n223,Remoraid,Water,,300,35,65,35,65,35,65,2,False\n224,Octillery,Water,,480,75,105,75,105,75,45,2,False\n225,Delibird,Ice,Flying,330,45,55,45,65,45,75,2,False\n226,Mantine,Water,Flying,465,65,40,70,80,140,70,2,False\n227,Skarmory,Steel,Flying,465,65,80,140,40,70,70,2,False\n228,Houndour,Dark,Fire,330,45,60,30,80,50,65,2,False\n229,Houndoom,Dark,Fire,500,75,90,50,110,80,95,2,False\n229,HoundoomMega Houndoom,Dark,Fire,600,75,90,90,140,90,115,2,False\n230,Kingdra,Water,Dragon,540,75,95,95,95,95,85,2,False\n231,Phanpy,Ground,,330,90,60,60,40,40,40,2,False\n232,Donphan,Ground,,500,90,120,120,60,60,50,2,False\n233,Porygon2,Normal,,515,85,80,90,105,95,60,2,False\n234,Stantler,Normal,,465,73,95,62,85,65,85,2,False\n235,Smeargle,Normal,,250,55,20,35,20,45,75,2,False\n236,Tyrogue,Fighting,,210,35,35,35,35,35,35,2,False\n237,Hitmontop,Fighting,,455,50,95,95,35,110,70,2,False\n238,Smoochum,Ice,Psychic,305,45,30,15,85,65,65,2,False\n239,Elekid,Electric,,360,45,63,37,65,55,95,2,False\n240,Magby,Fire,,365,45,75,37,70,55,83,2,False\n241,Miltank,Normal,,490,95,80,105,40,70,100,2,False\n242,Blissey,Normal,,540,255,10,10,75,135,55,2,False\n243,Raikou,Electric,,580,90,85,75,115,100,115,2,True\n244,Entei,Fire,,580,115,115,85,90,75,100,2,True\n245,Suicune,Water,,580,100,75,115,90,115,85,2,True\n246,Larvitar,Rock,Ground,300,50,64,50,45,50,41,2,False\n247,Pupitar,Rock,Ground,410,70,84,70,65,70,51,2,False\n248,Tyranitar,Rock,Dark,600,100,134,110,95,100,61,2,False\n248,TyranitarMega Tyranitar,Rock,Dark,700,100,164,150,95,120,71,2,False\n249,Lugia,Psychic,Flying,680,106,90,130,90,154,110,2,True\n250,Ho-oh,Fire,Flying,680,106,130,90,110,154,90,2,True\n251,Celebi,Psychic,Grass,600,100,100,100,100,100,100,2,False\n252,Treecko,Grass,,310,40,45,35,65,55,70,3,False\n253,Grovyle,Grass,,405,50,65,45,85,65,95,3,False\n254,Sceptile,Grass,,530,70,85,65,105,85,120,3,False\n254,SceptileMega Sceptile,Grass,Dragon,630,70,110,75,145,85,145,3,False\n255,Torchic,Fire,,310,45,60,40,70,50,45,3,False\n256,Combusken,Fire,Fighting,405,60,85,60,85,60,55,3,False\n257,Blaziken,Fire,Fighting,530,80,120,70,110,70,80,3,False\n257,BlazikenMega Blaziken,Fire,Fighting,630,80,160,80,130,80,100,3,False\n258,Mudkip,Water,,310,50,70,50,50,50,40,3,False\n259,Marshtomp,Water,Ground,405,70,85,70,60,70,50,3,False\n260,Swampert,Water,Ground,535,100,110,90,85,90,60,3,False\n260,SwampertMega Swampert,Water,Ground,635,100,150,110,95,110,70,3,False\n261,Poochyena,Dark,,220,35,55,35,30,30,35,3,False\n262,Mightyena,Dark,,420,70,90,70,60,60,70,3,False\n263,Zigzagoon,Normal,,240,38,30,41,30,41,60,3,False\n264,Linoone,Normal,,420,78,70,61,50,61,100,3,False\n265,Wurmple,Bug,,195,45,45,35,20,30,20,3,False\n266,Silcoon,Bug,,205,50,35,55,25,25,15,3,False\n267,Beautifly,Bug,Flying,395,60,70,50,100,50,65,3,False\n268,Cascoon,Bug,,205,50,35,55,25,25,15,3,False\n269,Dustox,Bug,Poison,385,60,50,70,50,90,65,3,False\n270,Lotad,Water,Grass,220,40,30,30,40,50,30,3,False\n271,Lombre,Water,Grass,340,60,50,50,60,70,50,3,False\n272,Ludicolo,Water,Grass,480,80,70,70,90,100,70,3,False\n273,Seedot,Grass,,220,40,40,50,30,30,30,3,False\n274,Nuzleaf,Grass,Dark,340,70,70,40,60,40,60,3,False\n275,Shiftry,Grass,Dark,480,90,100,60,90,60,80,3,False\n276,Taillow,Normal,Flying,270,40,55,30,30,30,85,3,False\n277,Swellow,Normal,Flying,430,60,85,60,50,50,125,3,False\n278,Wingull,Water,Flying,270,40,30,30,55,30,85,3,False\n279,Pelipper,Water,Flying,430,60,50,100,85,70,65,3,False\n280,Ralts,Psychic,Fairy,198,28,25,25,45,35,40,3,False\n281,Kirlia,Psychic,Fairy,278,38,35,35,65,55,50,3,False\n282,Gardevoir,Psychic,Fairy,518,68,65,65,125,115,80,3,False\n282,GardevoirMega Gardevoir,Psychic,Fairy,618,68,85,65,165,135,100,3,False\n283,Surskit,Bug,Water,269,40,30,32,50,52,65,3,False\n284,Masquerain,Bug,Flying,414,70,60,62,80,82,60,3,False\n285,Shroomish,Grass,,295,60,40,60,40,60,35,3,False\n286,Breloom,Grass,Fighting,460,60,130,80,60,60,70,3,False\n287,Slakoth,Normal,,280,60,60,60,35,35,30,3,False\n288,Vigoroth,Normal,,440,80,80,80,55,55,90,3,False\n289,Slaking,Normal,,670,150,160,100,95,65,100,3,False\n290,Nincada,Bug,Ground,266,31,45,90,30,30,40,3,False\n291,Ninjask,Bug,Flying,456,61,90,45,50,50,160,3,False\n292,Shedinja,Bug,Ghost,236,1,90,45,30,30,40,3,False\n293,Whismur,Normal,,240,64,51,23,51,23,28,3,False\n294,Loudred,Normal,,360,84,71,43,71,43,48,3,False\n295,Exploud,Normal,,490,104,91,63,91,73,68,3,False\n296,Makuhita,Fighting,,237,72,60,30,20,30,25,3,False\n297,Hariyama,Fighting,,474,144,120,60,40,60,50,3,False\n298,Azurill,Normal,Fairy,190,50,20,40,20,40,20,3,False\n299,Nosepass,Rock,,375,30,45,135,45,90,30,3,False\n300,Skitty,Normal,,260,50,45,45,35,35,50,3,False\n301,Delcatty,Normal,,380,70,65,65,55,55,70,3,False\n302,Sableye,Dark,Ghost,380,50,75,75,65,65,50,3,False\n302,SableyeMega Sableye,Dark,Ghost,480,50,85,125,85,115,20,3,False\n303,Mawile,Steel,Fairy,380,50,85,85,55,55,50,3,False\n303,MawileMega Mawile,Steel,Fairy,480,50,105,125,55,95,50,3,False\n304,Aron,Steel,Rock,330,50,70,100,40,40,30,3,False\n305,Lairon,Steel,Rock,430,60,90,140,50,50,40,3,False\n306,Aggron,Steel,Rock,530,70,110,180,60,60,50,3,False\n306,AggronMega Aggron,Steel,,630,70,140,230,60,80,50,3,False\n307,Meditite,Fighting,Psychic,280,30,40,55,40,55,60,3,False\n308,Medicham,Fighting,Psychic,410,60,60,75,60,75,80,3,False\n308,MedichamMega Medicham,Fighting,Psychic,510,60,100,85,80,85,100,3,False\n309,Electrike,Electric,,295,40,45,40,65,40,65,3,False\n310,Manectric,Electric,,475,70,75,60,105,60,105,3,False\n310,ManectricMega Manectric,Electric,,575,70,75,80,135,80,135,3,False\n311,Plusle,Electric,,405,60,50,40,85,75,95,3,False\n312,Minun,Electric,,405,60,40,50,75,85,95,3,False\n313,Volbeat,Bug,,400,65,73,55,47,75,85,3,False\n314,Illumise,Bug,,400,65,47,55,73,75,85,3,False\n315,Roselia,Grass,Poison,400,50,60,45,100,80,65,3,False\n316,Gulpin,Poison,,302,70,43,53,43,53,40,3,False\n317,Swalot,Poison,,467,100,73,83,73,83,55,3,False\n318,Carvanha,Water,Dark,305,45,90,20,65,20,65,3,False\n319,Sharpedo,Water,Dark,460,70,120,40,95,40,95,3,False\n319,SharpedoMega Sharpedo,Water,Dark,560,70,140,70,110,65,105,3,False\n320,Wailmer,Water,,400,130,70,35,70,35,60,3,False\n321,Wailord,Water,,500,170,90,45,90,45,60,3,False\n322,Numel,Fire,Ground,305,60,60,40,65,45,35,3,False\n323,Camerupt,Fire,Ground,460,70,100,70,105,75,40,3,False\n323,CameruptMega Camerupt,Fire,Ground,560,70,120,100,145,105,20,3,False\n324,Torkoal,Fire,,470,70,85,140,85,70,20,3,False\n325,Spoink,Psychic,,330,60,25,35,70,80,60,3,False\n326,Grumpig,Psychic,,470,80,45,65,90,110,80,3,False\n327,Spinda,Normal,,360,60,60,60,60,60,60,3,False\n328,Trapinch,Ground,,290,45,100,45,45,45,10,3,False\n329,Vibrava,Ground,Dragon,340,50,70,50,50,50,70,3,False\n330,Flygon,Ground,Dragon,520,80,100,80,80,80,100,3,False\n331,Cacnea,Grass,,335,50,85,40,85,40,35,3,False\n332,Cacturne,Grass,Dark,475,70,115,60,115,60,55,3,False\n333,Swablu,Normal,Flying,310,45,40,60,40,75,50,3,False\n334,Altaria,Dragon,Flying,490,75,70,90,70,105,80,3,False\n334,AltariaMega Altaria,Dragon,Fairy,590,75,110,110,110,105,80,3,False\n335,Zangoose,Normal,,458,73,115,60,60,60,90,3,False\n336,Seviper,Poison,,458,73,100,60,100,60,65,3,False\n337,Lunatone,Rock,Psychic,440,70,55,65,95,85,70,3,False\n338,Solrock,Rock,Psychic,440,70,95,85,55,65,70,3,False\n339,Barboach,Water,Ground,288,50,48,43,46,41,60,3,False\n340,Whiscash,Water,Ground,468,110,78,73,76,71,60,3,False\n341,Corphish,Water,,308,43,80,65,50,35,35,3,False\n342,Crawdaunt,Water,Dark,468,63,120,85,90,55,55,3,False\n343,Baltoy,Ground,Psychic,300,40,40,55,40,70,55,3,False\n344,Claydol,Ground,Psychic,500,60,70,105,70,120,75,3,False\n345,Lileep,Rock,Grass,355,66,41,77,61,87,23,3,False\n346,Cradily,Rock,Grass,495,86,81,97,81,107,43,3,False\n347,Anorith,Rock,Bug,355,45,95,50,40,50,75,3,False\n348,Armaldo,Rock,Bug,495,75,125,100,70,80,45,3,False\n349,Feebas,Water,,200,20,15,20,10,55,80,3,False\n350,Milotic,Water,,540,95,60,79,100,125,81,3,False\n351,Castform,Normal,,420,70,70,70,70,70,70,3,False\n352,Kecleon,Normal,,440,60,90,70,60,120,40,3,False\n353,Shuppet,Ghost,,295,44,75,35,63,33,45,3,False\n354,Banette,Ghost,,455,64,115,65,83,63,65,3,False\n354,BanetteMega Banette,Ghost,,555,64,165,75,93,83,75,3,False\n355,Duskull,Ghost,,295,20,40,90,30,90,25,3,False\n356,Dusclops,Ghost,,455,40,70,130,60,130,25,3,False\n357,Tropius,Grass,Flying,460,99,68,83,72,87,51,3,False\n358,Chimecho,Psychic,,425,65,50,70,95,80,65,3,False\n359,Absol,Dark,,465,65,130,60,75,60,75,3,False\n359,AbsolMega Absol,Dark,,565,65,150,60,115,60,115,3,False\n360,Wynaut,Psychic,,260,95,23,48,23,48,23,3,False\n361,Snorunt,Ice,,300,50,50,50,50,50,50,3,False\n362,Glalie,Ice,,480,80,80,80,80,80,80,3,False\n362,GlalieMega Glalie,Ice,,580,80,120,80,120,80,100,3,False\n363,Spheal,Ice,Water,290,70,40,50,55,50,25,3,False\n364,Sealeo,Ice,Water,410,90,60,70,75,70,45,3,False\n365,Walrein,Ice,Water,530,110,80,90,95,90,65,3,False\n366,Clamperl,Water,,345,35,64,85,74,55,32,3,False\n367,Huntail,Water,,485,55,104,105,94,75,52,3,False\n368,Gorebyss,Water,,485,55,84,105,114,75,52,3,False\n369,Relicanth,Water,Rock,485,100,90,130,45,65,55,3,False\n370,Luvdisc,Water,,330,43,30,55,40,65,97,3,False\n371,Bagon,Dragon,,300,45,75,60,40,30,50,3,False\n372,Shelgon,Dragon,,420,65,95,100,60,50,50,3,False\n373,Salamence,Dragon,Flying,600,95,135,80,110,80,100,3,False\n373,SalamenceMega Salamence,Dragon,Flying,700,95,145,130,120,90,120,3,False\n374,Beldum,Steel,Psychic,300,40,55,80,35,60,30,3,False\n375,Metang,Steel,Psychic,420,60,75,100,55,80,50,3,False\n376,Metagross,Steel,Psychic,600,80,135,130,95,90,70,3,False\n376,MetagrossMega Metagross,Steel,Psychic,700,80,145,150,105,110,110,3,False\n377,Regirock,Rock,,580,80,100,200,50,100,50,3,True\n378,Regice,Ice,,580,80,50,100,100,200,50,3,True\n379,Registeel,Steel,,580,80,75,150,75,150,50,3,True\n380,Latias,Dragon,Psychic,600,80,80,90,110,130,110,3,True\n380,LatiasMega Latias,Dragon,Psychic,700,80,100,120,140,150,110,3,True\n381,Latios,Dragon,Psychic,600,80,90,80,130,110,110,3,True\n381,LatiosMega Latios,Dragon,Psychic,700,80,130,100,160,120,110,3,True\n382,Kyogre,Water,,670,100,100,90,150,140,90,3,True\n382,KyogrePrimal Kyogre,Water,,770,100,150,90,180,160,90,3,True\n383,Groudon,Ground,,670,100,150,140,100,90,90,3,True\n383,GroudonPrimal Groudon,Ground,Fire,770,100,180,160,150,90,90,3,True\n384,Rayquaza,Dragon,Flying,680,105,150,90,150,90,95,3,True\n384,RayquazaMega Rayquaza,Dragon,Flying,780,105,180,100,180,100,115,3,True\n385,Jirachi,Steel,Psychic,600,100,100,100,100,100,100,3,True\n386,DeoxysNormal Forme,Psychic,,600,50,150,50,150,50,150,3,True\n386,DeoxysAttack Forme,Psychic,,600,50,180,20,180,20,150,3,True\n386,DeoxysDefense Forme,Psychic,,600,50,70,160,70,160,90,3,True\n386,DeoxysSpeed Forme,Psychic,,600,50,95,90,95,90,180,3,True\n387,Turtwig,Grass,,318,55,68,64,45,55,31,4,False\n388,Grotle,Grass,,405,75,89,85,55,65,36,4,False\n389,Torterra,Grass,Ground,525,95,109,105,75,85,56,4,False\n390,Chimchar,Fire,,309,44,58,44,58,44,61,4,False\n391,Monferno,Fire,Fighting,405,64,78,52,78,52,81,4,False\n392,Infernape,Fire,Fighting,534,76,104,71,104,71,108,4,False\n393,Piplup,Water,,314,53,51,53,61,56,40,4,False\n394,Prinplup,Water,,405,64,66,68,81,76,50,4,False\n395,Empoleon,Water,Steel,530,84,86,88,111,101,60,4,False\n396,Starly,Normal,Flying,245,40,55,30,30,30,60,4,False\n397,Staravia,Normal,Flying,340,55,75,50,40,40,80,4,False\n398,Staraptor,Normal,Flying,485,85,120,70,50,60,100,4,False\n399,Bidoof,Normal,,250,59,45,40,35,40,31,4,False\n400,Bibarel,Normal,Water,410,79,85,60,55,60,71,4,False\n401,Kricketot,Bug,,194,37,25,41,25,41,25,4,False\n402,Kricketune,Bug,,384,77,85,51,55,51,65,4,False\n403,Shinx,Electric,,263,45,65,34,40,34,45,4,False\n404,Luxio,Electric,,363,60,85,49,60,49,60,4,False\n405,Luxray,Electric,,523,80,120,79,95,79,70,4,False\n406,Budew,Grass,Poison,280,40,30,35,50,70,55,4,False\n407,Roserade,Grass,Poison,515,60,70,65,125,105,90,4,False\n408,Cranidos,Rock,,350,67,125,40,30,30,58,4,False\n409,Rampardos,Rock,,495,97,165,60,65,50,58,4,False\n410,Shieldon,Rock,Steel,350,30,42,118,42,88,30,4,False\n411,Bastiodon,Rock,Steel,495,60,52,168,47,138,30,4,False\n412,Burmy,Bug,,224,40,29,45,29,45,36,4,False\n413,WormadamPlant Cloak,Bug,Grass,424,60,59,85,79,105,36,4,False\n413,WormadamSandy Cloak,Bug,Ground,424,60,79,105,59,85,36,4,False\n413,WormadamTrash Cloak,Bug,Steel,424,60,69,95,69,95,36,4,False\n414,Mothim,Bug,Flying,424,70,94,50,94,50,66,4,False\n415,Combee,Bug,Flying,244,30,30,42,30,42,70,4,False\n416,Vespiquen,Bug,Flying,474,70,80,102,80,102,40,4,False\n417,Pachirisu,Electric,,405,60,45,70,45,90,95,4,False\n418,Buizel,Water,,330,55,65,35,60,30,85,4,False\n419,Floatzel,Water,,495,85,105,55,85,50,115,4,False\n420,Cherubi,Grass,,275,45,35,45,62,53,35,4,False\n421,Cherrim,Grass,,450,70,60,70,87,78,85,4,False\n422,Shellos,Water,,325,76,48,48,57,62,34,4,False\n423,Gastrodon,Water,Ground,475,111,83,68,92,82,39,4,False\n424,Ambipom,Normal,,482,75,100,66,60,66,115,4,False\n425,Drifloon,Ghost,Flying,348,90,50,34,60,44,70,4,False\n426,Drifblim,Ghost,Flying,498,150,80,44,90,54,80,4,False\n427,Buneary,Normal,,350,55,66,44,44,56,85,4,False\n428,Lopunny,Normal,,480,65,76,84,54,96,105,4,False\n428,LopunnyMega Lopunny,Normal,Fighting,580,65,136,94,54,96,135,4,False\n429,Mismagius,Ghost,,495,60,60,60,105,105,105,4,False\n430,Honchkrow,Dark,Flying,505,100,125,52,105,52,71,4,False\n431,Glameow,Normal,,310,49,55,42,42,37,85,4,False\n432,Purugly,Normal,,452,71,82,64,64,59,112,4,False\n433,Chingling,Psychic,,285,45,30,50,65,50,45,4,False\n434,Stunky,Poison,Dark,329,63,63,47,41,41,74,4,False\n435,Skuntank,Poison,Dark,479,103,93,67,71,61,84,4,False\n436,Bronzor,Steel,Psychic,300,57,24,86,24,86,23,4,False\n437,Bronzong,Steel,Psychic,500,67,89,116,79,116,33,4,False\n438,Bonsly,Rock,,290,50,80,95,10,45,10,4,False\n439,Mime Jr.,Psychic,Fairy,310,20,25,45,70,90,60,4,False\n440,Happiny,Normal,,220,100,5,5,15,65,30,4,False\n441,Chatot,Normal,Flying,411,76,65,45,92,42,91,4,False\n442,Spiritomb,Ghost,Dark,485,50,92,108,92,108,35,4,False\n443,Gible,Dragon,Ground,300,58,70,45,40,45,42,4,False\n444,Gabite,Dragon,Ground,410,68,90,65,50,55,82,4,False\n445,Garchomp,Dragon,Ground,600,108,130,95,80,85,102,4,False\n445,GarchompMega Garchomp,Dragon,Ground,700,108,170,115,120,95,92,4,False\n446,Munchlax,Normal,,390,135,85,40,40,85,5,4,False\n447,Riolu,Fighting,,285,40,70,40,35,40,60,4,False\n448,Lucario,Fighting,Steel,525,70,110,70,115,70,90,4,False\n448,LucarioMega Lucario,Fighting,Steel,625,70,145,88,140,70,112,4,False\n449,Hippopotas,Ground,,330,68,72,78,38,42,32,4,False\n450,Hippowdon,Ground,,525,108,112,118,68,72,47,4,False\n451,Skorupi,Poison,Bug,330,40,50,90,30,55,65,4,False\n452,Drapion,Poison,Dark,500,70,90,110,60,75,95,4,False\n453,Croagunk,Poison,Fighting,300,48,61,40,61,40,50,4,False\n454,Toxicroak,Poison,Fighting,490,83,106,65,86,65,85,4,False\n455,Carnivine,Grass,,454,74,100,72,90,72,46,4,False\n456,Finneon,Water,,330,49,49,56,49,61,66,4,False\n457,Lumineon,Water,,460,69,69,76,69,86,91,4,False\n458,Mantyke,Water,Flying,345,45,20,50,60,120,50,4,False\n459,Snover,Grass,Ice,334,60,62,50,62,60,40,4,False\n460,Abomasnow,Grass,Ice,494,90,92,75,92,85,60,4,False\n460,AbomasnowMega Abomasnow,Grass,Ice,594,90,132,105,132,105,30,4,False\n461,Weavile,Dark,Ice,510,70,120,65,45,85,125,4,False\n462,Magnezone,Electric,Steel,535,70,70,115,130,90,60,4,False\n463,Lickilicky,Normal,,515,110,85,95,80,95,50,4,False\n464,Rhyperior,Ground,Rock,535,115,140,130,55,55,40,4,False\n465,Tangrowth,Grass,,535,100,100,125,110,50,50,4,False\n466,Electivire,Electric,,540,75,123,67,95,85,95,4,False\n467,Magmortar,Fire,,540,75,95,67,125,95,83,4,False\n468,Togekiss,Fairy,Flying,545,85,50,95,120,115,80,4,False\n469,Yanmega,Bug,Flying,515,86,76,86,116,56,95,4,False\n470,Leafeon,Grass,,525,65,110,130,60,65,95,4,False\n471,Glaceon,Ice,,525,65,60,110,130,95,65,4,False\n472,Gliscor,Ground,Flying,510,75,95,125,45,75,95,4,False\n473,Mamoswine,Ice,Ground,530,110,130,80,70,60,80,4,False\n474,Porygon-Z,Normal,,535,85,80,70,135,75,90,4,False\n475,Gallade,Psychic,Fighting,518,68,125,65,65,115,80,4,False\n475,GalladeMega Gallade,Psychic,Fighting,618,68,165,95,65,115,110,4,False\n476,Probopass,Rock,Steel,525,60,55,145,75,150,40,4,False\n477,Dusknoir,Ghost,,525,45,100,135,65,135,45,4,False\n478,Froslass,Ice,Ghost,480,70,80,70,80,70,110,4,False\n479,Rotom,Electric,Ghost,440,50,50,77,95,77,91,4,False\n479,RotomHeat Rotom,Electric,Fire,520,50,65,107,105,107,86,4,False\n479,RotomWash Rotom,Electric,Water,520,50,65,107,105,107,86,4,False\n479,RotomFrost Rotom,Electric,Ice,520,50,65,107,105,107,86,4,False\n479,RotomFan Rotom,Electric,Flying,520,50,65,107,105,107,86,4,False\n479,RotomMow Rotom,Electric,Grass,520,50,65,107,105,107,86,4,False\n480,Uxie,Psychic,,580,75,75,130,75,130,95,4,True\n481,Mesprit,Psychic,,580,80,105,105,105,105,80,4,True\n482,Azelf,Psychic,,580,75,125,70,125,70,115,4,True\n483,Dialga,Steel,Dragon,680,100,120,120,150,100,90,4,True\n484,Palkia,Water,Dragon,680,90,120,100,150,120,100,4,True\n485,Heatran,Fire,Steel,600,91,90,106,130,106,77,4,True\n486,Regigigas,Normal,,670,110,160,110,80,110,100,4,True\n487,GiratinaAltered Forme,Ghost,Dragon,680,150,100,120,100,120,90,4,True\n487,GiratinaOrigin Forme,Ghost,Dragon,680,150,120,100,120,100,90,4,True\n488,Cresselia,Psychic,,600,120,70,120,75,130,85,4,False\n489,Phione,Water,,480,80,80,80,80,80,80,4,False\n490,Manaphy,Water,,600,100,100,100,100,100,100,4,False\n491,Darkrai,Dark,,600,70,90,90,135,90,125,4,True\n492,ShayminLand Forme,Grass,,600,100,100,100,100,100,100,4,True\n492,ShayminSky Forme,Grass,Flying,600,100,103,75,120,75,127,4,True\n493,Arceus,Normal,,720,120,120,120,120,120,120,4,True\n494,Victini,Psychic,Fire,600,100,100,100,100,100,100,5,True\n495,Snivy,Grass,,308,45,45,55,45,55,63,5,False\n496,Servine,Grass,,413,60,60,75,60,75,83,5,False\n497,Serperior,Grass,,528,75,75,95,75,95,113,5,False\n498,Tepig,Fire,,308,65,63,45,45,45,45,5,False\n499,Pignite,Fire,Fighting,418,90,93,55,70,55,55,5,False\n500,Emboar,Fire,Fighting,528,110,123,65,100,65,65,5,False\n501,Oshawott,Water,,308,55,55,45,63,45,45,5,False\n502,Dewott,Water,,413,75,75,60,83,60,60,5,False\n503,Samurott,Water,,528,95,100,85,108,70,70,5,False\n504,Patrat,Normal,,255,45,55,39,35,39,42,5,False\n505,Watchog,Normal,,420,60,85,69,60,69,77,5,False\n506,Lillipup,Normal,,275,45,60,45,25,45,55,5,False\n507,Herdier,Normal,,370,65,80,65,35,65,60,5,False\n508,Stoutland,Normal,,500,85,110,90,45,90,80,5,False\n509,Purrloin,Dark,,281,41,50,37,50,37,66,5,False\n510,Liepard,Dark,,446,64,88,50,88,50,106,5,False\n511,Pansage,Grass,,316,50,53,48,53,48,64,5,False\n512,Simisage,Grass,,498,75,98,63,98,63,101,5,False\n513,Pansear,Fire,,316,50,53,48,53,48,64,5,False\n514,Simisear,Fire,,498,75,98,63,98,63,101,5,False\n515,Panpour,Water,,316,50,53,48,53,48,64,5,False\n516,Simipour,Water,,498,75,98,63,98,63,101,5,False\n517,Munna,Psychic,,292,76,25,45,67,55,24,5,False\n518,Musharna,Psychic,,487,116,55,85,107,95,29,5,False\n519,Pidove,Normal,Flying,264,50,55,50,36,30,43,5,False\n520,Tranquill,Normal,Flying,358,62,77,62,50,42,65,5,False\n521,Unfezant,Normal,Flying,488,80,115,80,65,55,93,5,False\n522,Blitzle,Electric,,295,45,60,32,50,32,76,5,False\n523,Zebstrika,Electric,,497,75,100,63,80,63,116,5,False\n524,Roggenrola,Rock,,280,55,75,85,25,25,15,5,False\n525,Boldore,Rock,,390,70,105,105,50,40,20,5,False\n526,Gigalith,Rock,,515,85,135,130,60,80,25,5,False\n527,Woobat,Psychic,Flying,313,55,45,43,55,43,72,5,False\n528,Swoobat,Psychic,Flying,425,67,57,55,77,55,114,5,False\n529,Drilbur,Ground,,328,60,85,40,30,45,68,5,False\n530,Excadrill,Ground,Steel,508,110,135,60,50,65,88,5,False\n531,Audino,Normal,,445,103,60,86,60,86,50,5,False\n531,AudinoMega Audino,Normal,Fairy,545,103,60,126,80,126,50,5,False\n532,Timburr,Fighting,,305,75,80,55,25,35,35,5,False\n533,Gurdurr,Fighting,,405,85,105,85,40,50,40,5,False\n534,Conkeldurr,Fighting,,505,105,140,95,55,65,45,5,False\n535,Tympole,Water,,294,50,50,40,50,40,64,5,False\n536,Palpitoad,Water,Ground,384,75,65,55,65,55,69,5,False\n537,Seismitoad,Water,Ground,509,105,95,75,85,75,74,5,False\n538,Throh,Fighting,,465,120,100,85,30,85,45,5,False\n539,Sawk,Fighting,,465,75,125,75,30,75,85,5,False\n540,Sewaddle,Bug,Grass,310,45,53,70,40,60,42,5,False\n541,Swadloon,Bug,Grass,380,55,63,90,50,80,42,5,False\n542,Leavanny,Bug,Grass,500,75,103,80,70,80,92,5,False\n543,Venipede,Bug,Poison,260,30,45,59,30,39,57,5,False\n544,Whirlipede,Bug,Poison,360,40,55,99,40,79,47,5,False\n545,Scolipede,Bug,Poison,485,60,100,89,55,69,112,5,False\n546,Cottonee,Grass,Fairy,280,40,27,60,37,50,66,5,False\n547,Whimsicott,Grass,Fairy,480,60,67,85,77,75,116,5,False\n548,Petilil,Grass,,280,45,35,50,70,50,30,5,False\n549,Lilligant,Grass,,480,70,60,75,110,75,90,5,False\n550,Basculin,Water,,460,70,92,65,80,55,98,5,False\n551,Sandile,Ground,Dark,292,50,72,35,35,35,65,5,False\n552,Krokorok,Ground,Dark,351,60,82,45,45,45,74,5,False\n553,Krookodile,Ground,Dark,519,95,117,80,65,70,92,5,False\n554,Darumaka,Fire,,315,70,90,45,15,45,50,5,False\n555,DarmanitanStandard Mode,Fire,,480,105,140,55,30,55,95,5,False\n555,DarmanitanZen Mode,Fire,Psychic,540,105,30,105,140,105,55,5,False\n556,Maractus,Grass,,461,75,86,67,106,67,60,5,False\n557,Dwebble,Bug,Rock,325,50,65,85,35,35,55,5,False\n558,Crustle,Bug,Rock,475,70,95,125,65,75,45,5,False\n559,Scraggy,Dark,Fighting,348,50,75,70,35,70,48,5,False\n560,Scrafty,Dark,Fighting,488,65,90,115,45,115,58,5,False\n561,Sigilyph,Psychic,Flying,490,72,58,80,103,80,97,5,False\n562,Yamask,Ghost,,303,38,30,85,55,65,30,5,False\n563,Cofagrigus,Ghost,,483,58,50,145,95,105,30,5,False\n564,Tirtouga,Water,Rock,355,54,78,103,53,45,22,5,False\n565,Carracosta,Water,Rock,495,74,108,133,83,65,32,5,False\n566,Archen,Rock,Flying,401,55,112,45,74,45,70,5,False\n567,Archeops,Rock,Flying,567,75,140,65,112,65,110,5,False\n568,Trubbish,Poison,,329,50,50,62,40,62,65,5,False\n569,Garbodor,Poison,,474,80,95,82,60,82,75,5,False\n570,Zorua,Dark,,330,40,65,40,80,40,65,5,False\n571,Zoroark,Dark,,510,60,105,60,120,60,105,5,False\n572,Minccino,Normal,,300,55,50,40,40,40,75,5,False\n573,Cinccino,Normal,,470,75,95,60,65,60,115,5,False\n574,Gothita,Psychic,,290,45,30,50,55,65,45,5,False\n575,Gothorita,Psychic,,390,60,45,70,75,85,55,5,False\n576,Gothitelle,Psychic,,490,70,55,95,95,110,65,5,False\n577,Solosis,Psychic,,290,45,30,40,105,50,20,5,False\n578,Duosion,Psychic,,370,65,40,50,125,60,30,5,False\n579,Reuniclus,Psychic,,490,110,65,75,125,85,30,5,False\n580,Ducklett,Water,Flying,305,62,44,50,44,50,55,5,False\n581,Swanna,Water,Flying,473,75,87,63,87,63,98,5,False\n582,Vanillite,Ice,,305,36,50,50,65,60,44,5,False\n583,Vanillish,Ice,,395,51,65,65,80,75,59,5,False\n584,Vanilluxe,Ice,,535,71,95,85,110,95,79,5,False\n585,Deerling,Normal,Grass,335,60,60,50,40,50,75,5,False\n586,Sawsbuck,Normal,Grass,475,80,100,70,60,70,95,5,False\n587,Emolga,Electric,Flying,428,55,75,60,75,60,103,5,False\n588,Karrablast,Bug,,315,50,75,45,40,45,60,5,False\n589,Escavalier,Bug,Steel,495,70,135,105,60,105,20,5,False\n590,Foongus,Grass,Poison,294,69,55,45,55,55,15,5,False\n591,Amoonguss,Grass,Poison,464,114,85,70,85,80,30,5,False\n592,Frillish,Water,Ghost,335,55,40,50,65,85,40,5,False\n593,Jellicent,Water,Ghost,480,100,60,70,85,105,60,5,False\n594,Alomomola,Water,,470,165,75,80,40,45,65,5,False\n595,Joltik,Bug,Electric,319,50,47,50,57,50,65,5,False\n596,Galvantula,Bug,Electric,472,70,77,60,97,60,108,5,False\n597,Ferroseed,Grass,Steel,305,44,50,91,24,86,10,5,False\n598,Ferrothorn,Grass,Steel,489,74,94,131,54,116,20,5,False\n599,Klink,Steel,,300,40,55,70,45,60,30,5,False\n600,Klang,Steel,,440,60,80,95,70,85,50,5,False\n601,Klinklang,Steel,,520,60,100,115,70,85,90,5,False\n602,Tynamo,Electric,,275,35,55,40,45,40,60,5,False\n603,Eelektrik,Electric,,405,65,85,70,75,70,40,5,False\n604,Eelektross,Electric,,515,85,115,80,105,80,50,5,False\n605,Elgyem,Psychic,,335,55,55,55,85,55,30,5,False\n606,Beheeyem,Psychic,,485,75,75,75,125,95,40,5,False\n607,Litwick,Ghost,Fire,275,50,30,55,65,55,20,5,False\n608,Lampent,Ghost,Fire,370,60,40,60,95,60,55,5,False\n609,Chandelure,Ghost,Fire,520,60,55,90,145,90,80,5,False\n610,Axew,Dragon,,320,46,87,60,30,40,57,5,False\n611,Fraxure,Dragon,,410,66,117,70,40,50,67,5,False\n612,Haxorus,Dragon,,540,76,147,90,60,70,97,5,False\n613,Cubchoo,Ice,,305,55,70,40,60,40,40,5,False\n614,Beartic,Ice,,485,95,110,80,70,80,50,5,False\n615,Cryogonal,Ice,,485,70,50,30,95,135,105,5,False\n616,Shelmet,Bug,,305,50,40,85,40,65,25,5,False\n617,Accelgor,Bug,,495,80,70,40,100,60,145,5,False\n618,Stunfisk,Ground,Electric,471,109,66,84,81,99,32,5,False\n619,Mienfoo,Fighting,,350,45,85,50,55,50,65,5,False\n620,Mienshao,Fighting,,510,65,125,60,95,60,105,5,False\n621,Druddigon,Dragon,,485,77,120,90,60,90,48,5,False\n622,Golett,Ground,Ghost,303,59,74,50,35,50,35,5,False\n623,Golurk,Ground,Ghost,483,89,124,80,55,80,55,5,False\n624,Pawniard,Dark,Steel,340,45,85,70,40,40,60,5,False\n625,Bisharp,Dark,Steel,490,65,125,100,60,70,70,5,False\n626,Bouffalant,Normal,,490,95,110,95,40,95,55,5,False\n627,Rufflet,Normal,Flying,350,70,83,50,37,50,60,5,False\n628,Braviary,Normal,Flying,510,100,123,75,57,75,80,5,False\n629,Vullaby,Dark,Flying,370,70,55,75,45,65,60,5,False\n630,Mandibuzz,Dark,Flying,510,110,65,105,55,95,80,5,False\n631,Heatmor,Fire,,484,85,97,66,105,66,65,5,False\n632,Durant,Bug,Steel,484,58,109,112,48,48,109,5,False\n633,Deino,Dark,Dragon,300,52,65,50,45,50,38,5,False\n634,Zweilous,Dark,Dragon,420,72,85,70,65,70,58,5,False\n635,Hydreigon,Dark,Dragon,600,92,105,90,125,90,98,5,False\n636,Larvesta,Bug,Fire,360,55,85,55,50,55,60,5,False\n637,Volcarona,Bug,Fire,550,85,60,65,135,105,100,5,False\n638,Cobalion,Steel,Fighting,580,91,90,129,90,72,108,5,True\n639,Terrakion,Rock,Fighting,580,91,129,90,72,90,108,5,True\n640,Virizion,Grass,Fighting,580,91,90,72,90,129,108,5,True\n641,TornadusIncarnate Forme,Flying,,580,79,115,70,125,80,111,5,True\n641,TornadusTherian Forme,Flying,,580,79,100,80,110,90,121,5,True\n642,ThundurusIncarnate Forme,Electric,Flying,580,79,115,70,125,80,111,5,True\n642,ThundurusTherian Forme,Electric,Flying,580,79,105,70,145,80,101,5,True\n643,Reshiram,Dragon,Fire,680,100,120,100,150,120,90,5,True\n644,Zekrom,Dragon,Electric,680,100,150,120,120,100,90,5,True\n645,LandorusIncarnate Forme,Ground,Flying,600,89,125,90,115,80,101,5,True\n645,LandorusTherian Forme,Ground,Flying,600,89,145,90,105,80,91,5,True\n646,Kyurem,Dragon,Ice,660,125,130,90,130,90,95,5,True\n646,KyuremBlack Kyurem,Dragon,Ice,700,125,170,100,120,90,95,5,True\n646,KyuremWhite Kyurem,Dragon,Ice,700,125,120,90,170,100,95,5,True\n647,KeldeoOrdinary Forme,Water,Fighting,580,91,72,90,129,90,108,5,False\n647,KeldeoResolute Forme,Water,Fighting,580,91,72,90,129,90,108,5,False\n648,MeloettaAria Forme,Normal,Psychic,600,100,77,77,128,128,90,5,False\n648,MeloettaPirouette Forme,Normal,Fighting,600,100,128,90,77,77,128,5,False\n649,Genesect,Bug,Steel,600,71,120,95,120,95,99,5,False\n650,Chespin,Grass,,313,56,61,65,48,45,38,6,False\n651,Quilladin,Grass,,405,61,78,95,56,58,57,6,False\n652,Chesnaught,Grass,Fighting,530,88,107,122,74,75,64,6,False\n653,Fennekin,Fire,,307,40,45,40,62,60,60,6,False\n654,Braixen,Fire,,409,59,59,58,90,70,73,6,False\n655,Delphox,Fire,Psychic,534,75,69,72,114,100,104,6,False\n656,Froakie,Water,,314,41,56,40,62,44,71,6,False\n657,Frogadier,Water,,405,54,63,52,83,56,97,6,False\n658,Greninja,Water,Dark,530,72,95,67,103,71,122,6,False\n659,Bunnelby,Normal,,237,38,36,38,32,36,57,6,False\n660,Diggersby,Normal,Ground,423,85,56,77,50,77,78,6,False\n661,Fletchling,Normal,Flying,278,45,50,43,40,38,62,6,False\n662,Fletchinder,Fire,Flying,382,62,73,55,56,52,84,6,False\n663,Talonflame,Fire,Flying,499,78,81,71,74,69,126,6,False\n664,Scatterbug,Bug,,200,38,35,40,27,25,35,6,False\n665,Spewpa,Bug,,213,45,22,60,27,30,29,6,False\n666,Vivillon,Bug,Flying,411,80,52,50,90,50,89,6,False\n667,Litleo,Fire,Normal,369,62,50,58,73,54,72,6,False\n668,Pyroar,Fire,Normal,507,86,68,72,109,66,106,6,False\n669,Flabébé,Fairy,,303,44,38,39,61,79,42,6,False\n670,Floette,Fairy,,371,54,45,47,75,98,52,6,False\n671,Florges,Fairy,,552,78,65,68,112,154,75,6,False\n672,Skiddo,Grass,,350,66,65,48,62,57,52,6,False\n673,Gogoat,Grass,,531,123,100,62,97,81,68,6,False\n674,Pancham,Fighting,,348,67,82,62,46,48,43,6,False\n675,Pangoro,Fighting,Dark,495,95,124,78,69,71,58,6,False\n676,Furfrou,Normal,,472,75,80,60,65,90,102,6,False\n677,Espurr,Psychic,,355,62,48,54,63,60,68,6,False\n678,MeowsticMale,Psychic,,466,74,48,76,83,81,104,6,False\n678,MeowsticFemale,Psychic,,466,74,48,76,83,81,104,6,False\n679,Honedge,Steel,Ghost,325,45,80,100,35,37,28,6,False\n680,Doublade,Steel,Ghost,448,59,110,150,45,49,35,6,False\n681,AegislashBlade Forme,Steel,Ghost,520,60,150,50,150,50,60,6,False\n681,AegislashShield Forme,Steel,Ghost,520,60,50,150,50,150,60,6,False\n682,Spritzee,Fairy,,341,78,52,60,63,65,23,6,False\n683,Aromatisse,Fairy,,462,101,72,72,99,89,29,6,False\n684,Swirlix,Fairy,,341,62,48,66,59,57,49,6,False\n685,Slurpuff,Fairy,,480,82,80,86,85,75,72,6,False\n686,Inkay,Dark,Psychic,288,53,54,53,37,46,45,6,False\n687,Malamar,Dark,Psychic,482,86,92,88,68,75,73,6,False\n688,Binacle,Rock,Water,306,42,52,67,39,56,50,6,False\n689,Barbaracle,Rock,Water,500,72,105,115,54,86,68,6,False\n690,Skrelp,Poison,Water,320,50,60,60,60,60,30,6,False\n691,Dragalge,Poison,Dragon,494,65,75,90,97,123,44,6,False\n692,Clauncher,Water,,330,50,53,62,58,63,44,6,False\n693,Clawitzer,Water,,500,71,73,88,120,89,59,6,False\n694,Helioptile,Electric,Normal,289,44,38,33,61,43,70,6,False\n695,Heliolisk,Electric,Normal,481,62,55,52,109,94,109,6,False\n696,Tyrunt,Rock,Dragon,362,58,89,77,45,45,48,6,False\n697,Tyrantrum,Rock,Dragon,521,82,121,119,69,59,71,6,False\n698,Amaura,Rock,Ice,362,77,59,50,67,63,46,6,False\n699,Aurorus,Rock,Ice,521,123,77,72,99,92,58,6,False\n700,Sylveon,Fairy,,525,95,65,65,110,130,60,6,False\n701,Hawlucha,Fighting,Flying,500,78,92,75,74,63,118,6,False\n702,Dedenne,Electric,Fairy,431,67,58,57,81,67,101,6,False\n703,Carbink,Rock,Fairy,500,50,50,150,50,150,50,6,False\n704,Goomy,Dragon,,300,45,50,35,55,75,40,6,False\n705,Sliggoo,Dragon,,452,68,75,53,83,113,60,6,False\n706,Goodra,Dragon,,600,90,100,70,110,150,80,6,False\n707,Klefki,Steel,Fairy,470,57,80,91,80,87,75,6,False\n708,Phantump,Ghost,Grass,309,43,70,48,50,60,38,6,False\n709,Trevenant,Ghost,Grass,474,85,110,76,65,82,56,6,False\n710,PumpkabooAverage Size,Ghost,Grass,335,49,66,70,44,55,51,6,False\n710,PumpkabooSmall Size,Ghost,Grass,335,44,66,70,44,55,56,6,False\n710,PumpkabooLarge Size,Ghost,Grass,335,54,66,70,44,55,46,6,False\n710,PumpkabooSuper Size,Ghost,Grass,335,59,66,70,44,55,41,6,False\n711,GourgeistAverage Size,Ghost,Grass,494,65,90,122,58,75,84,6,False\n711,GourgeistSmall Size,Ghost,Grass,494,55,85,122,58,75,99,6,False\n711,GourgeistLarge Size,Ghost,Grass,494,75,95,122,58,75,69,6,False\n711,GourgeistSuper Size,Ghost,Grass,494,85,100,122,58,75,54,6,False\n712,Bergmite,Ice,,304,55,69,85,32,35,28,6,False\n713,Avalugg,Ice,,514,95,117,184,44,46,28,6,False\n714,Noibat,Flying,Dragon,245,40,30,35,45,40,55,6,False\n715,Noivern,Flying,Dragon,535,85,70,80,97,80,123,6,False\n716,Xerneas,Fairy,,680,126,131,95,131,98,99,6,True\n717,Yveltal,Dark,Flying,680,126,131,95,131,98,99,6,True\n718,Zygarde50% Forme,Dragon,Ground,600,108,100,121,81,95,95,6,True\n719,Diancie,Rock,Fairy,600,50,100,150,100,150,50,6,True\n719,DiancieMega Diancie,Rock,Fairy,700,50,160,110,160,110,110,6,True\n720,HoopaHoopa Confined,Psychic,Ghost,600,80,110,60,150,130,70,6,True\n720,HoopaHoopa Unbound,Psychic,Dark,680,80,160,60,170,130,80,6,True\n721,Volcanion,Fire,Water,600,80,110,120,130,90,70,6,True\n" }, { "path": "pandas_log/__init__.py", "content": "# -*- coding: utf-8 -*-\n\n\"\"\"Top-level package for pandas-log.\"\"\"\nfrom .pandas_log import *\n\n__author__ = \"\"\"Eyal Trabelsi\"\"\"\n__email__ = \"eyaltrabelsi@gmail.com\"\n__version__ = \"0.0.0\"\n\nORIGINAL_METHOD_PREFIX = \"original_\"\n" }, { "path": "pandas_log/aop_utils.py", "content": "import itertools\nfrom inspect import signature\n\nimport pandas as pd\n\nfrom pandas_log import settings\nfrom pandas_log.settings import DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE\n\n\ndef set_df_attr(df, attr_name, attr_value):\n \"\"\" Hacky way to set attributes in dataframe\n\n :param df: DataFrame\n :param attr_name: Attribute name\n :param attr_value: Attribute value\n :return: None\n \"\"\"\n\n df.__dict__[attr_name] = attr_value\n\n\ndef append_df_attr(df, attr_name, attr_value):\n \"\"\" Hacky way to append a value to dataframe\n\n :param df: DataFrame\n :param attr_name: Attribute name\n :param attr_value: Attribute value\n :return: None\n \"\"\"\n\n df.__dict__[attr_name].append(attr_value)\n\n\ndef get_df_attr(df, attr_name, default_val):\n \"\"\" Get Dataframe attribute if exists otherwise default value\n\n :return: Dataframe attribute\n \"\"\"\n\n return df.__dict__.get(attr_name, default_val)\n\n\ndef get_pandas_func(cls, func, prefix=settings.ORIGINAL_METHOD_PREFIX):\n \"\"\" Get original pandas method\n\n :param cls: pandas class\n :param func: pandas method name\n :param prefix: the prefix used to keep original method\n :return: Original pandas method\n \"\"\"\n\n _raise_on_bad_class(cls)\n return getattr(cls, f\"{prefix}{func.__name__}\")\n\n\ndef get_signature_repr(cls, fn, args, full_signature=True):\n \"\"\" Get the signature for the original pandas method with actual values\n\n :param cls: the pandas class\n :param fn: The pandas method\n :param args: The arguments used when it was applied\n :return: string representation of the signature for the applied pandas method\n \"\"\"\n\n _raise_on_bad_class(cls)\n\n def _get_bold_text(text):\n return f\"\\033[1m{text}\\033[0m\"\n\n def _get_orig_func_params():\n return [\n param_value if full_signature else param_name\n for param_name, param_value in signature(\n get_pandas_func(cls, fn)\n ).parameters.items()\n if param_name not in (\"kwargs\", \"self\")\n ]\n\n def _get_param_value(param_with_default, arg_value):\n res = str(param_with_default)\n if arg_value is not None:\n param_name = res.split(\"=\")[0]\n arg_value = (\n f'\"{arg_value}\"' if isinstance(arg_value, str) else arg_value\n )\n res = (\n param_name\n if isinstance(arg_value, pd.DataFrame)\n or isinstance(arg_value, pd.Series)\n else f\"{param_name}={arg_value}\"\n )\n return res\n\n zip_func = itertools.zip_longest if full_signature else zip\n orig_func_params = _get_orig_func_params()\n args_vals = \", \".join(\n _get_param_value(param_with_default, arg_value)\n for param_with_default, arg_value in zip_func(orig_func_params, args)\n )\n return f\"{_get_bold_text(fn.__name__)}({args_vals}):\"\n\n\ndef _raise_on_bad_class(cls):\n implemented_classes = (pd.DataFrame, pd.Series)\n if cls not in implemented_classes:\n raise TypeError(\"cls must be one of {}\".format(implemented_classes))\n\n\ndef restore_pandas_func_copy(\n cls, func, prefix=settings.ORIGINAL_METHOD_PREFIX\n):\n \"\"\" Restore the original pandas method instead of overridden one\n\n :param cls: class containing the method\n :param func: pandas method name\n :param prefix: the prefix used to keep original method\n :return: None\n \"\"\"\n\n _raise_on_bad_class(cls)\n original_method = getattr(cls, func)\n setattr(cls, func.replace(prefix, \"\"), original_method)\n\n\ndef keep_pandas_func_copy(cls, func, prefix=settings.ORIGINAL_METHOD_PREFIX):\n \"\"\" Saved copy of the pandas method before it overridden\n\n :param cls: class containing the method\n :param func: pandas method name\n :param prefix: the prefix used to keep original method\n :return: None\n \"\"\"\n\n _raise_on_bad_class(cls)\n original_method = getattr(cls, func)\n setattr(cls, f\"{prefix}{func}\", original_method)\n\n\ndef calc_step_number(method_name, input_df):\n # TODO\n step_number = get_df_attr(input_df, \"execution_history\", 0)\n if step_number:\n step_number = step_number[-1].execution_stats.step_number\n\n if method_name not in DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE:\n step_number += 1\n return step_number\n\n\nif __name__ == \"__main__\":\n pass\n" }, { "path": "pandas_log/pandas_execution_stats.py", "content": "import warnings\nfrom collections import namedtuple\nfrom functools import partial\nfrom time import time\n\nimport pandas as pd\n\nfrom pandas_log import patched_logs_functions\nfrom pandas_log.aop_utils import (\n append_df_attr,\n calc_step_number,\n get_df_attr,\n get_pandas_func,\n get_signature_repr,\n set_df_attr,\n)\nfrom pandas_log.settings import (\n DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE,\n PATCHED_LOG_METHOD_PREFIX,\n)\n\nwith warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n import humanize\n\n\ndef get_execution_stats(cls, fn, input_df, fn_args, fn_kwargs, calculate_memory):\n start = time()\n output_df = get_pandas_func(cls, fn)(input_df, *fn_args, **fn_kwargs)\n exec_time = time() - start\n exec_time_pretty = humanize.naturaldelta(exec_time)\n if exec_time_pretty == \"a moment\":\n exec_time_pretty = f\"{round(exec_time,6)} seconds\"\n step_number = calc_step_number(fn.__name__, input_df)\n\n input_memory_size = (\n StepStats.calc_df_series_memory(input_df) if calculate_memory else None\n )\n output_memory_size = (\n StepStats.calc_df_series_memory(output_df) if calculate_memory else None\n )\n\n ExecutionStats = namedtuple(\n \"ExecutionStats\", \"exec_time step_number input_memory_size output_memory_size\",\n )\n execution_stats = ExecutionStats(\n exec_time_pretty, step_number, input_memory_size, output_memory_size\n )\n return output_df, execution_stats\n\n\nclass StepStats:\n def __init__(\n self,\n execution_stats,\n cls,\n fn,\n fn_args,\n fn_kwargs,\n full_signature,\n input_df,\n output_df,\n ):\n \"\"\" Constructor\n :param execution_stats: execution_stats of the pandas operation both in time and memory\n :param cls: The calling object's pandas class\n :param fn: The original pandas method\n :param fn_args: The original pandas method args\n :param fn_kwargs: The original pandas method kwargs\n :param full_signature: adding additional information to function signature\n :param input_df: dataframe before step calculation\n :param output_df: dataframe after step calculation\n \"\"\"\n\n self.execution_stats = execution_stats\n self.full_signature = full_signature\n self.cls = cls\n self.fn = fn\n self.fn_args = fn_args\n self.fn_kwargs = fn_kwargs\n self.input_df = input_df\n self.output_df = output_df\n\n @staticmethod\n def calc_df_series_memory(df_or_series):\n res = None\n if isinstance(df_or_series, pd.Series):\n mem = df_or_series.memory_usage(index=True, deep=True)\n res = humanize.naturalsize(mem)\n elif isinstance(df_or_series, pd.DataFrame):\n mem = df_or_series.memory_usage(index=True, deep=True)\n res = humanize.naturalsize(mem.sum())\n return res\n\n def persist_execution_stats(self):\n prev_exec_history = get_df_attr(self.input_df, \"execution_history\", [])\n set_df_attr(self.output_df, \"execution_history\", prev_exec_history)\n append_df_attr(self.output_df, \"execution_history\", self)\n\n def log_stats_if_needed(self, silent, verbose, copy_ok):\n\n from pandas_log.pandas_log import ALREADY_ENABLED\n\n if silent or not ALREADY_ENABLED:\n return\n\n if verbose or self.fn.__name__ not in DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE:\n s = self.__repr__(verbose, copy_ok)\n if s:\n # If this method isn't patched and verbose is False, __repr__ will give an empty string, which\n # we don't want to print\n print(s)\n\n def get_logs_for_specifc_method(self, verbose, copy_ok):\n self.fn_kwargs[\"kwargs\"] = self.fn_kwargs.copy()\n self.fn_kwargs[\"copy_ok\"] = copy_ok\n try:\n log_method = getattr(\n patched_logs_functions,\n f\"{PATCHED_LOG_METHOD_PREFIX}{self.fn.__name__}\",\n )\n except AttributeError:\n # Method is listed as a method to override, but no patched function exists\n if verbose:\n log_method = getattr(patched_logs_functions, \"log_default\")\n else:\n log_method = getattr(patched_logs_functions, \"log_no_message\")\n\n log_method = partial(log_method, self.output_df, self.input_df)\n logs, tips = log_method(*self.fn_args, **self.fn_kwargs)\n return logs, tips\n\n def _repr_html_(self):\n pass\n\n def __repr__(self, verbose, copy_ok):\n # Step title\n func_sig = get_signature_repr(\n self.cls, self.fn, self.fn_args, self.full_signature\n )\n step_number = (\n \"X\"\n if self.fn.__name__ in DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE\n else self.execution_stats.step_number\n )\n step_title = f\"{step_number}) {func_sig}\"\n\n # Step Metadata stats\n logs, tips = self.get_logs_for_specifc_method(verbose, copy_ok)\n metadata_stats = f\"\\033[4mMetadata\\033[0m:\\n{logs}\" if logs else \"\"\n metadata_tips = f\"\\033[4mTips\\033[0m:\\n{tips}\" if tips else \"\"\n\n # Step Execution stats\n exec_time_humanize = (\n f\"* Execution time: Step Took {self.execution_stats.exec_time}.\"\n )\n exec_stats_raw = [exec_time_humanize]\n if self.execution_stats.input_memory_size is not None:\n exec_stats_raw.append(\n f\"* Input Dataframe size is {self.execution_stats.input_memory_size}.\"\n )\n if self.execution_stats.output_memory_size is not None:\n exec_stats_raw.append(\n f\"* Output Dataframe size is {self.execution_stats.output_memory_size}.\"\n )\n exec_stats_raw_str = \"\\n\\t\".join(exec_stats_raw)\n execution_stats = f\"\\033[4mExecution Stats\\033[0m:\\n\\t{exec_stats_raw_str}\"\n\n all_logs = [metadata_stats, execution_stats, metadata_tips]\n all_logs_str = \"\\n\\t\".join([x for x in all_logs if x])\n\n return f\"\\n{step_title}\\n\\t{all_logs_str}\"\n\n\nif __name__ == \"__main__\":\n pass\n" }, { "path": "pandas_log/pandas_log.py", "content": "# -*- coding: utf-8 -*-\n\n\"\"\"Main module.\"\"\"\n\nimport warnings\nfrom contextlib import contextmanager\nfrom functools import wraps\n\nimport pandas as pd\nimport pandas_flavor as pf\n\nfrom pandas_log import settings\nfrom pandas_log.aop_utils import (\n keep_pandas_func_copy,\n restore_pandas_func_copy,\n)\nfrom pandas_log.pandas_execution_stats import StepStats, get_execution_stats\n\n__all__ = [\"auto_enable\", \"auto_disable\", \"enable\"]\n\n\nALREADY_ENABLED = False\n\n\ndef auto_disable():\n \"\"\" Restore original pandas method without the additional log functionality (statistics)\n Note: we keep the original methods using original_ prefix.\n :return: None\n \"\"\"\n global ALREADY_ENABLED\n if not ALREADY_ENABLED:\n return\n\n for cls in (pd.DataFrame, pd.Series):\n for func in dir(cls):\n if func.startswith(settings.ORIGINAL_METHOD_PREFIX):\n restore_pandas_func_copy(cls, func)\n ALREADY_ENABLED = False\n\n\n@contextmanager\ndef enable(\n verbose=False,\n silent=False,\n full_signature=True,\n copy_ok=True,\n calculate_memory=False,\n):\n \"\"\" Adds the additional logging functionality (statistics) to pandas methods only for the scope of this\n context manager.\n\n :param verbose: Whether some inner functions should be recorded as well.\n For example: when a dataframe being copied\n :param silent: Whether additional the statistics get printed\n :param full_signature: adding additional information to function signature\n :param copy_ok: whether the dataframe is allowed to be copied to calculate more informative metadata logs\n :return: None\n \"\"\"\n\n auto_enable(verbose, silent, full_signature, copy_ok, calculate_memory)\n yield\n auto_disable()\n\n\ndef auto_enable(\n verbose=False,\n silent=False,\n full_signature=True,\n copy_ok=True,\n calculate_memory=False,\n):\n \"\"\" Adds the additional logging functionality (statistics) to pandas methods.\n\n :param verbose: Whether some inner functions should be recorded as well.\n For example: when a dataframe being copied\n :param silent: Whether additional the statistics get printed\n :param full_signature: adding additional information to function signature\n :param copy_ok: whether the dataframe is allowed to be copied to calculate more informative metadata logs\n :return: None\n \"\"\"\n global ALREADY_ENABLED\n if ALREADY_ENABLED:\n return\n\n settings.DATAFRAME_METHODS_TO_OVERIDE.extend(\n settings.DATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE\n )\n\n # Suppressing warning of the fact we override pandas functions.\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\")\n for cls, overrides in [\n (pd.DataFrame, settings.DATAFRAME_METHODS_TO_OVERIDE),\n (pd.Series, settings.SERIES_METHODS_TO_OVERIDE),\n ]:\n for func in dir(cls):\n if func in overrides:\n keep_pandas_func_copy(cls, func)\n create_overide_pandas_func(\n cls,\n func,\n verbose,\n silent,\n full_signature,\n copy_ok,\n calculate_memory,\n )\n ALREADY_ENABLED = True\n\n\ndef create_overide_pandas_func(\n cls, func, verbose, silent, full_signature, copy_ok, calculate_memory\n):\n \"\"\" Create overridden pandas method dynamically with\n additional logging using DataFrameLogger\n\n Note: if we extracting _overide_pandas_method outside we need to implement decorator like here\n https://stackoverflow.com/questions/10176226/how-do-i-pass-extra-arguments-to-a-python-decorator\n\n :param cls: pandas class for which the method should be overriden\n :param func: pandas method name to be overridden\n :param silent: Whether additional the statistics get printed\n :param full_signature: adding additional information to function signature\n :param copy_ok: whether the dataframe is allowed to be copied to calculate more informative metadata logs\n :return: the same function with additional logging capabilities\n \"\"\"\n\n def _run_method_and_calc_stats(\n fn,\n fn_args,\n fn_kwargs,\n input_df,\n full_signature,\n silent,\n verbose,\n copy_ok,\n calculate_memory,\n ):\n\n if copy_ok:\n # If we're ok to make copies, copy the input_df so that we can compare against the output of inplace methods\n try:\n # Will hit infinite recursion if we use the patched copy method so use the original\n original_input_df = getattr(\n input_df, settings.ORIGINAL_METHOD_PREFIX + \"copy\"\n )(deep=True)\n except AttributeError:\n original_input_df = input_df.copy(deep=True)\n output_df, execution_stats = get_execution_stats(\n cls, fn, input_df, fn_args, fn_kwargs, calculate_memory\n )\n if output_df is None:\n # The operation was strictly in place so we just call the dataframe the output_df as well\n output_df = input_df\n if copy_ok:\n # If this isn't true and the method was strictly inplace, input_df and output_df will just\n # point to the same object\n input_df = original_input_df\n\n step_stats = StepStats(\n execution_stats,\n cls,\n fn,\n fn_args,\n fn_kwargs,\n full_signature,\n input_df,\n output_df,\n )\n step_stats.log_stats_if_needed(silent, verbose, copy_ok)\n if isinstance(output_df, pd.DataFrame) or isinstance(output_df, pd.Series):\n step_stats.persist_execution_stats()\n return output_df\n\n def _overide_pandas_method(fn):\n if cls == pd.DataFrame:\n register_method_wrapper = pf.register_dataframe_method\n elif cls == pd.Series:\n register_method_wrapper = pf.register_series_method\n\n @register_method_wrapper\n @wraps(fn)\n def wrapped(*args, **fn_kwargs):\n\n input_df, fn_args = args[0], args[1:]\n output_df = _run_method_and_calc_stats(\n fn,\n fn_args,\n fn_kwargs,\n input_df,\n full_signature,\n silent,\n verbose,\n copy_ok,\n calculate_memory,\n )\n return output_df\n\n return wrapped\n\n return exec(f\"@_overide_pandas_method\\ndef {func}(df, *args, **kwargs): pass\")\n\n\nif __name__ == \"__main__\":\n pass\n" }, { "path": "pandas_log/patched_logs_functions.py", "content": "import warnings\n\nimport numpy as np\nimport pandas as pd\n\n# Values messages\nALTERED_VALUES_MSG = \"\\t* Changed {values_changed} values, {values_unchanged} values were not changed.\"\n\n# Rows messages\nREMOVED_NO_ROWS_MSG = \"\\t* No change in number of rows of input df.\"\nFILTERED_ROWS_MSG = \"\\t* Removed {rows_removed} rows ({rows_removed_pct}%), {rows_remaining} rows remaining.\"\n\n# Cols messages\nREMOVED_NO_COLS_MSG = \"\\t* Removed no columns.\"\nFILTERED_COLS_MSG = (\n \"\\t* Removed the following columns ({cols_removed}) now only have the following columns\"\n \"({cols_remaining}).\"\n)\nASSIGN_EXISTING_MSG = \"\\t* The columns {existing_cols} were reassigned.\"\nASSIGN_NEW_MSG = \"\\t* The columns {new_cols} were created.\"\n\n# Group by messages\nGROUPBY_MSG = \"\\t* Grouping by {} resulted in {} groups like {}.\"\n# N/A messages\nFILLNA_NO_NA_MSG = \"\\t* There are no nulls.\"\nFILLNA_WITHH_NA_MSG = \"\\t* Filled {} with {}.\"\n\n# Mege messages\nJOIN_ROWS_MSG = \"\\t* Number of rows changed, after join is {output_rows} rows.\"\nJOIN_NEW_COLS_MSG = \"\\t* Added {num_new_columns} columns ({new_columns}).\"\nJOIN_TYPE_MSG = (\n \"\\t* Its a {how} join with the following cardinality:\\n\\t\\t> rows only in left is {left_only}.\"\n \"\\n\\t\\t> rows only in right is {right_only}.\\n\\t\\t> rows in both is {both}.\"\n)\n\n# Pick messages\nSAMPLE_MSG = \"\\t* Picked random sample of {output_rows} rows.\"\nNLARGEST_MSG = \"\\t* Picked {n} largest rows by columns ({cols}).\"\nNSMALLEST_MSG = \"\\t* Picked {n} smallest rows by columns ({cols}).\"\nHEAD_MSG = \"\\t* Picked the first {} rows.\"\nTAIL_MSG = \"\\t* Picked the last {} rows.\"\n\n\n# TIPS\nITERROWS_TIPS = \"\\t*iterrows is not recommended, and in the majority of cases will have better alternatives\"\nFILLNA_NO_NA_TIP = (\n \"\\t* There are no nulls in this dataframe, if you are working on the entire dataset you can \"\n \"remove this operation.\"\n)\nSHOULD_REDUCED_ROW_TIP = (\n \"\\t* Number of rows didn't change, if you are working on the entire dataset you can remove \"\n \"this operation.\"\n)\n\n# Others\nSORT_VALUES_MSG = \"\\t* Sorting by columns {} in a {} order.\"\nSORT_INDEX_MSG = \"\\t* Sorting by index in a {'ascending' if ascending else 'descending'} order.\"\nDEFAULT_STRATEGY_USED_MSG = (\n \"\\t* Using default strategy (some metric might not be relevant).\"\n)\nTRANSFORMED_TO_DF_MSG = \"\\t* After transformation we received Series\"\n\nCOPY_WARNING_MSG = (\n \"Some pandas logging may involve copying dataframes, which can be time-/memory-intensive. \"\n \"Consider passing copy_ok=False to the enable/auto_enable functions in pandas_log if issues arise.\"\n)\n\n\ndef _stringify_list(l):\n return [str(x) for x in l]\n\n\ndef rows_removed(input_df, output_df):\n return len(input_df) - len(output_df)\n\n\ndef rows_removed_pct(input_df, output_df):\n return 100 * (rows_removed(input_df, output_df)) / len(input_df)\n\n\ndef rows_remaining(output_df):\n return len(output_df)\n\n\ndef cols_removed(input_df, output_df):\n return \", \".join(\n _stringify_list(set(input_df.columns) - set(output_df.columns))\n )\n\n\ndef cols_remaining(output_df):\n return \", \".join(_stringify_list(set(output_df.columns)))\n\n\ndef is_same_cols(input_df, output_df):\n return len(input_df.columns) == len(output_df.columns)\n\n\ndef columns_changed(df, cols):\n return set(df.columns).intersection(set(cols))\n\n\ndef columns_added(df, cols):\n return set(cols) - set(df.columns)\n\n\ndef is_same_rows(input_df, output_df):\n return len(input_df) == len(output_df)\n\n\ndef num_of_na(df):\n return df.isnull().values.sum()\n\n\ndef str_new_columns(input_df, output_df):\n return \", \".join(\n _stringify_list(set(output_df.columns) - set(input_df.columns))\n )\n\n\ndef num_new_columns(input_df, output_df):\n return len(set(output_df.columns) - set(input_df.columns))\n\n\ndef num_values_changed(input_obj, output_obj):\n if (\n isinstance(input_obj, pd.Series)\n and isinstance(output_obj, pd.Series)\n and input_obj.dtype != output_obj.dtype\n ):\n # Comparing values for equality across dtypes wouldn't be well-defined so we just say they all changed\n values_changed = len(input_obj)\n else:\n values_changed = (\n (output_obj != input_obj)\n & ~(output_obj.isnull() & input_obj.isnull())\n ).sum()\n if isinstance(input_obj, pd.DataFrame):\n # We only summed once so values_changed will be a series, so we sum again\n values_changed = values_changed.sum()\n values_unchanged = (\n input_obj.shape[0] * input_obj.shape[1]\n ) - values_changed\n elif isinstance(input_obj, pd.Series):\n values_unchanged = len(output_obj) - values_changed\n return values_changed, values_unchanged\n\n\ndef get_filter_rows_logs(input_df, output_df):\n tips = \"\"\n if is_same_rows(input_df, output_df):\n logs = REMOVED_NO_ROWS_MSG\n tips = SHOULD_REDUCED_ROW_TIP\n else:\n logs = FILTERED_ROWS_MSG.format(\n rows_removed=rows_removed(input_df, output_df),\n rows_removed_pct=rows_removed_pct(input_df, output_df),\n rows_remaining=rows_remaining(output_df),\n )\n return logs, tips\n\n\ndef log_default(output_df, input_df, *args, **kwargs):\n logs = [DEFAULT_STRATEGY_USED_MSG]\n tips = \"\"\n if isinstance(output_df, pd.DataFrame):\n if not is_same_cols(input_df, output_df):\n logs.append(\n FILTERED_COLS_MSG.format(\n cols_removed=cols_removed(input_df, output_df),\n cols_remaining=cols_remaining(output_df),\n )\n )\n if not is_same_rows(input_df, output_df):\n logs.append(\n FILTERED_ROWS_MSG.format(\n rows_removed=rows_removed(input_df, output_df),\n rows_removed_pct=rows_removed_pct(input_df, output_df),\n rows_remaining=rows_remaining(output_df),\n )\n )\n elif isinstance(output_df, pd.Series):\n logs.append(TRANSFORMED_TO_DF_MSG)\n logs = \"\\n\".join(logs)\n return logs, tips\n\n\ndef log_no_message(output_df, input_df, *args, **kwargs):\n return \"\", \"\"\n\n\ndef log_drop(\n output_df,\n input_df,\n labels=None,\n axis=0,\n index=None,\n columns=None,\n level=None,\n inplace=False,\n errors=\"raise\",\n *args,\n **kwargs,\n):\n logs, tips = get_filter_rows_logs(input_df, output_df)\n tips = \"\"\n if is_same_cols(input_df, output_df):\n logs += f\"\\n{REMOVED_NO_COLS_MSG}\"\n else:\n msg = FILTERED_COLS_MSG.format(\n cols_removed=cols_removed(input_df, output_df),\n cols_remaining=cols_remaining(output_df),\n )\n logs += f\"\\n{msg}\"\n return logs, tips\n\n\ndef log_dropna(\n output_df,\n input_df,\n axis=0,\n how=\"any\",\n thresh=None,\n subset=None,\n inplace=False,\n **kwargs,\n):\n logs, tips = get_filter_rows_logs(input_df, output_df)\n if is_same_cols(input_df, output_df):\n logs += f\"\\n{REMOVED_NO_COLS_MSG}\"\n tips = SHOULD_REDUCED_ROW_TIP\n else:\n msg = FILTERED_COLS_MSG.format(\n cols_removed=cols_removed(input_df, output_df),\n cols_remaining=cols_remaining(output_df),\n )\n logs += f\"\\n{msg}\"\n return logs, tips\n\n\ndef log_assign(output_df, input_df, **kwargs):\n logs = []\n tips = \"\"\n cols = [key for key in kwargs.keys() if key not in [\"kwargs\", \"copy_ok\"]]\n changed_cols = columns_changed(input_df, cols)\n added_cols = columns_added(input_df, cols)\n if changed_cols:\n if kwargs[\"copy_ok\"]:\n warnings.warn(COPY_WARNING_MSG)\n # If copying is ok, we can check how many values actually changed\n for col in changed_cols:\n values_changed, values_unchanged = num_values_changed(\n input_df[col], output_df[col]\n )\n logs.append(\n \"\\t* {}: {}\".format(\n col,\n ALTERED_VALUES_MSG.format(\n values_changed=values_changed,\n values_unchanged=values_unchanged,\n )[3:],\n )\n ) # [3:] to strip the \"\\t* \" from the altered values message\n else:\n # Otherwise just indicated which columns changed\n # (Doing the above calculation would always say zero values changed in this case, since if copying isn't\n # ok then input_df and output_df point to the same object)\n logs.append(\n ASSIGN_EXISTING_MSG.format(\n existing_cols=\", \".join(changed_cols)\n )\n )\n if added_cols:\n logs.append(\n ASSIGN_NEW_MSG.format(\n new_cols=\", \".join(columns_added(input_df, cols))\n )\n )\n logs = \"\\n\".join(logs)\n return logs, tips\n\n\ndef log___setitem__(output_df, input_df, key, value, **kwargs):\n if isinstance(key, str):\n # Only setting one column so can just use the assign logger as is\n kwargs[key] = value\n elif isinstance(key, list):\n # This would be kind of complicated but since we don't actually use the values in kwargs we can just\n # add the new columns as keys each with the full set of assigned values (as a placeholder)\n for subkey in key:\n kwargs[subkey] = value\n return log_assign(output_df, input_df, **kwargs)\n\n\ndef log_query(output_df, input_df, expr, inplace=False, *args, **kwargs):\n logs, tips = get_filter_rows_logs(input_df, output_df)\n return logs, tips\n\n\ndef log_sort_index(\n output_df,\n input_df,\n axis=0,\n level=None,\n ascending=True,\n inplace=False,\n kind=\"quicksort\",\n na_position=\"last\",\n sort_remaining=True,\n by=None,\n **kwargs,\n):\n logs = SORT_INDEX_MSG.format(ascending)\n tips = \"\"\n return logs, tips\n\n\ndef log_sort_values(\n output_df,\n input_df,\n by,\n axis=0,\n ascending=True,\n inplace=False,\n kind=\"quicksort\",\n na_position=\"last\",\n **kwargs,\n):\n logs = SORT_VALUES_MSG.format(\n by, \"ascending\" if ascending else \"descending\"\n )\n tips = \"\"\n return logs, tips\n\n\ndef log_tail(output_df, input_df, n=5, **kwargs):\n logs = TAIL_MSG.format(n)\n tips = SHOULD_REDUCED_ROW_TIP if is_same_rows(input_df, output_df) else \"\"\n return logs, tips\n\n\ndef log_head(output_df, input_df, n=5, **kwargs):\n logs = HEAD_MSG.format(n)\n tips = SHOULD_REDUCED_ROW_TIP if is_same_rows(input_df, output_df) else \"\"\n return logs, tips\n\n\ndef log_merge(\n output_df,\n input_df,\n right,\n how=\"inner\",\n on=None,\n left_on=None,\n right_on=None,\n left_index=False,\n right_index=False,\n sort=False,\n suffixes=(\"_x\", \"_y\"),\n copy=True,\n indicator=False,\n validate=None,\n **kwargs,\n):\n logs = []\n tips = \"\"\n merged = input_df.original_merge(\n right,\n \"outer\",\n on,\n left_on,\n right_on,\n left_index,\n right_index,\n sort,\n suffixes,\n copy,\n True,\n validate,\n )\n logs.append(\n JOIN_TYPE_MSG.format(how=how, **merged._merge.value_counts().to_dict())\n )\n if is_same_rows(input_df, output_df):\n logs.append(REMOVED_NO_ROWS_MSG)\n else:\n logs.append(JOIN_ROWS_MSG.format(output_rows=len(output_df)))\n if not is_same_cols(input_df, output_df):\n logs.append(\n JOIN_NEW_COLS_MSG.format(\n num_new_columns=num_new_columns(input_df, output_df),\n new_columns=str_new_columns(input_df, output_df),\n )\n )\n\n logs = \"\\n\".join(logs)\n return logs, tips\n\n\ndef log_applymap(output_df, input_df, func, **kwargs):\n values_changed, values_unchanged = num_values_changed(input_df, output_df)\n log = ALTERED_VALUES_MSG.format(\n values_changed=values_changed, values_unchanged=values_unchanged\n )\n return log, \"\"\n\n\ndef log_join(\n output_df,\n input_df,\n other,\n on=None,\n how=\"left\",\n lsuffix=\"\",\n rsuffix=\"\",\n sort=False,\n **kwargs,\n):\n logs = []\n tips = \"\"\n merged = input_df.original_merge(\n other, \"outer\", on, input_df.index, other.index, indicator=True\n )\n logs.append(\n JOIN_TYPE_MSG.format(how=how, **merged._merge.value_counts().to_dict())\n )\n\n logs.append(get_filter_rows_logs(input_df, output_df))\n if not is_same_cols(input_df, output_df):\n logs.append(\n JOIN_NEW_COLS_MSG.format(\n num_new_columns=num_new_columns(input_df, output_df),\n new_columns=str_new_columns(input_df, output_df),\n )\n )\n logs = \"\\n\".join(logs)\n return logs, tips\n\n\ndef log_fillna(\n output_df,\n input_df,\n value=None,\n method=None,\n axis=None,\n inplace=False,\n limit=None,\n downcast=None,\n **kwargs,\n):\n tips = \"\"\n value = \"empty string\" if value == \"\" else value\n if num_of_na(input_df) == num_of_na(output_df):\n logs = FILLNA_NO_NA_MSG\n tips = FILLNA_NO_NA_TIP\n else:\n logs = FILLNA_WITHH_NA_MSG.format(num_of_na(input_df), value)\n return logs, tips\n\n\ndef log_mask(\n output_df,\n input_df,\n cond,\n other=np.nan,\n inplace=False,\n axis=None,\n level=None,\n errors=\"raise\",\n try_cast=False,\n *args,\n **kwargs,\n):\n values_changed, values_unchanged = num_values_changed(input_df, output_df)\n logs = []\n logs.append(\n ALTERED_VALUES_MSG.format(\n values_changed=values_changed, values_unchanged=values_unchanged\n )\n )\n if isinstance(cond, pd.Series) and isinstance(input_df, pd.Series):\n # Calculate the values for which the condition was true but the value didn't change only for this simplest case,\n # where we can just & the two series\n true_but_unchanged = (cond & (output_df == input_df)).sum()\n if true_but_unchanged > 0:\n logs.append(\n \"\\t* {} rows met the masking condition and already had the masking value.\".format(\n true_but_unchanged\n )\n )\n logs = \"\\n\".join(logs)\n tips = \"\"\n return logs, tips\n\n\ndef log_where(\n output_df,\n input_df,\n cond,\n other=np.nan,\n inplace=False,\n axis=None,\n level=None,\n errors=\"raise\",\n try_cast=False,\n *args,\n **kwargs,\n):\n return log_mask(\n output_df,\n input_df,\n ~cond, # Important\n other=np.nan,\n inplace=False,\n axis=None,\n level=None,\n errors=\"raise\",\n try_cast=False,\n *args,\n **kwargs,\n )\n\n\ndef log_sample(\n output_df,\n input_df,\n n=None,\n frac=None,\n replace=False,\n weights=None,\n random_state=None,\n axis=None,\n *args,\n **kwargs,\n):\n logs = SAMPLE_MSG.format(output_rows=len(output_df))\n tips = SHOULD_REDUCED_ROW_TIP if is_same_rows(input_df, output_df) else \"\"\n return logs, tips\n\n\ndef log_nlargest(output_df, input_df, n, columns, keep=\"first\", **kwargs):\n # todo maybe wrong\n logs = NLARGEST_MSG.format(n=n, cols=columns)\n tips = SHOULD_REDUCED_ROW_TIP if is_same_rows(input_df, output_df) else \"\"\n return logs, tips\n\n\ndef log_nsmallest(output_df, input_df, n, columns, keep=\"first\", **kwargs):\n logs = NSMALLEST_MSG.format(n=n, cols=columns)\n tips = SHOULD_REDUCED_ROW_TIP if is_same_rows(input_df, output_df) else \"\"\n return logs, tips\n\n\ndef log_groupby(\n output_df,\n input_df,\n by=None,\n axis=0,\n level=None,\n as_index=True,\n sort=True,\n group_keys=True,\n squeeze=False,\n observed=False,\n **kwargs,\n):\n group_by = str(by)\n groups = list(output_df.groups)\n groups_len = len(groups)\n groups_repr = (\n \",\".join([\"\\n\\t\\t\" + str(x) for x in groups[:5]]) + \",\\n\\t and more\"\n )\n tips = \"\"\n logs = GROUPBY_MSG.format(group_by, groups_len, groups_repr)\n return logs, tips\n\n\ndef log__iterrows(output_df, input_df):\n tips = ITERROWS_TIPS\n logs = \"\"\n return logs, tips\n\n\ndef log___getitem__(output_df, input_df, key, *args, **kwargs):\n logs = []\n tips = \"\"\n\n if isinstance(output_df, pd.Series):\n # Naive handle of __getitem__ which return series\n logs = TRANSFORMED_TO_DF_MSG\n return logs, tips\n\n if not is_same_cols(input_df, output_df):\n logs.append(\n FILTERED_COLS_MSG.format(\n cols_removed=cols_removed(input_df, output_df),\n cols_remaining=cols_remaining(output_df),\n )\n )\n if not is_same_rows(input_df, output_df):\n logs.append(\n FILTERED_ROWS_MSG.format(\n rows_removed=rows_removed(input_df, output_df),\n rows_removed_pct=rows_removed_pct(input_df, output_df),\n rows_remaining=rows_remaining(output_df),\n )\n )\n logs = \"\\n\".join(logs)\n return logs, tips\n\n\n# TODO add tip on types+cardinality\n\nif __name__ == \"__main__\":\n pass\n" }, { "path": "pandas_log/settings.py", "content": "ORIGINAL_METHOD_PREFIX = \"original_\"\nPATCHED_LOG_METHOD_PREFIX = \"log_\"\nDATAFRAME_ADDITIONAL_METHODS_TO_OVERIDE = [\n \"copy\",\n \"reset_index\",\n \"__getitem__\",\n \"__setitem__\",\n]\nDATAFRAME_METHODS_TO_OVERIDE = [\n \"query\",\n \"drop\",\n \"dropna\",\n \"assign\",\n \"sort_index\",\n \"sort_values\",\n \"head\",\n \"tail\",\n \"sample\",\n \"fillna\",\n \"merge\",\n \"join\",\n \"nlargest\",\n \"nsmallest\",\n \"apply\",\n \"iterrows\",\n \"applymap\",\n \"pipe\",\n \"rolling\",\n \"groupby\",\n \"rename\",\n \"agg\",\n \"aggregate\",\n \"stack\",\n \"unstack\",\n \"pivot\",\n \"pivot_table\",\n \"mask\",\n \"max\",\n \"mean\",\n \"median\",\n \"melt\",\n \"replace\",\n \"skew\",\n \"notna\",\n \"kurt\",\n \"expanding\",\n \"drop_duplicates\",\n \"bfill\",\n \"corr\",\n \"corrwith\",\n \"droplevel\",\n \"explode\",\n \"ffill\",\n \"filter\",\n \"first\",\n \"kurtosis\",\n \"align\",\n \"transform\",\n \"update\",\n \"squeeze\",\n \"shift\",\n \"rank\",\n \"nunique\",\n \"min\",\n \"mod\",\n \"mode\",\n \"std\",\n]\nSERIES_METHODS_TO_OVERIDE = [\"mask\", \"where\"]\n" }, { "path": "requirements_dev.txt", "content": "pip>=19.2.3\nbump2version>=0.5.11\nwheel>=0.33.6\nwatchdog>=0.9.0\nflake8>=3.7.8\ntox>=3.14.0\ncoverage>=4.5.4\nSphinx>=2.2.0\ntwine>=2.2.0\npytest>=5.1.3\npytest-runner>=5.1\npandas>=0.25.1\npandas_flavor>=0.1.2\nhumanize>=0.5.0\n" }, { "path": "setup.cfg", "content": "[bumpversion]\ncurrent_version = 0.1.7\ncommit = True\ntag = True\n\n[bumpversion:file:setup.py]\nsearch = version='{current_version}'\nreplace = version='{new_version}'\n\n[bumpversion:file:pandas_log/__init__.py]\nsearch = __version__ = '{current_version}'\nreplace = __version__ = '{new_version}'\n\n[bdist_wheel]\nuniversal = 1\n\n[flake8]\nexclude = docs\n\n[aliases]\n# Define setup.py command aliases here\ntest = pytest\n\n[tool:pytest]\ncollect_ignore = ['setup.py']\n\n" }, { "path": "setup.py", "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\n\"\"\"The setup script.\"\"\"\n\nfrom setuptools import setup, find_packages\n\nwith open('README.rst') as readme_file:\n readme = readme_file.read()\n\nrequirements = [\"humanize>=0.5.0\", \"pandas>=0.25.1\", \"pandas_flavor>=0.1.2\"]\n\nsetup_requirements = ['pytest-runner', ]\n\ntest_requirements = ['pytest>=3', ]\n\nsetup(\n name='pandas-log',\n version='0.1.7',\n description=\"pandas-log provides feedback about basic pandas operations. It provides simple wrapper functions for \"\n \"the most common functions, such as apply, map, query and more.\",\n author=\"Eyal Trabelsi\",\n author_email='eyaltrabelsi@gmail.com',\n url='https://github.com/eyaltrabelsi/pandas-log',\n packages=find_packages(include=['pandas_log', 'pandas_log.*']),\n install_requires=requirements,\n python_requires=\">=3.4\",\n license=\"MIT license\",\n long_description=\"pandas-log provides feedback about basic pandas operations. It provides simple wrapper functions \"\n \"for the most common functions, such as apply, map, query and more.\",\n long_description_content_type=\"text/x-rst\",\n setup_requires=setup_requirements,\n test_suite='tests',\n tests_require=test_requirements\n)\n" }, { "path": "tox.ini", "content": "[tox]\nenvlist = py27, py35, py36, py37 flake8\n\n[travis]\npython =\n 3.7: py37\n 3.6: py36\n 3.5: py35\n 2.7: py27\n\n[testenv:flake8]\nbasepython = python\ndeps = flake8\ncommands = flake8 pandas_log\n\n[testenv]\nsetenv =\n PYTHONPATH = {toxinidir}\ndeps =\n -r{toxinidir}/requirements_dev.txt\n; If you want to make tox run the tests with the same versions, create a\n; requirements.txt with the pinned versions and uncomment the following line:\n; -r{toxinidir}/requirements.txt\ncommands =\n pip install -U pip\n pytest --basetemp={envtmpdir}\n\n" } ]