[ { "path": ".devcontainer/Dockerfile", "content": "FROM mcr.microsoft.com/vscode/devcontainers/python:3.10\nCOPY setup.sh /setup.sh\n" }, { "path": ".devcontainer/devcontainer.json", "content": "{\n \"dockerFile\": \"Dockerfile\",\n \"postCreateCommand\": \"sh /setup.sh\",\n\t\"customizations\": {\n\t\t\"vscode\": {\n\t\t\t\"settings\": {\n\t\t\t\t\"python.linting.enabled\": true,\n\t\t\t\t\"python.linting.flake8Enabled\": true,\n\t\t\t\t\"python.linting.pylintEnabled\": false,\n\t\t\t\t\"python.testing.pytestEnabled\": true,\n\t\t\t\t\"editor.formatOnSave\": true,\n\t\t\t\t\"editor.codeActionsOnSave\": {\n\t\t\t\t\t\"source.organizeImports\": true\n\t\t\t\t},\n\t\t\t\t\"[python]\": {\n\t\t\t\t\t\"editor.defaultFormatter\": \"ms-python.black-formatter\"\n\t\t\t\t},\n\t\t\t\t\"editor.rulers\": [\n\t\t\t\t\t80\n\t\t\t\t]\n\t\t\t},\n\t\t\t\"extensions\": [\n\t\t\t\t\"ms-python.python\",\n\t\t\t\t\"ms-python.isort\",\n\t\t\t\t\"ms-python.flake8\",\n\t\t\t\t\"ms-python.black-formatter\"\n\t\t\t]\n\t\t}\n\t},\n \"features\": {\n \"ghcr.io/devcontainers/features/github-cli:1\": {}\n }\n}" }, { "path": ".devcontainer/setup.sh", "content": "sudo pip install --upgrade pip\nsudo pip install -e \".[tests]\"\necho \"sh shell/lint.sh\" > .git/hooks/pre-commit\nchmod a+x .git/hooks/pre-commit" }, { "path": ".dockerignore", "content": ".git\n.github\n*.Dockerfile\n" }, { "path": ".github/CODEOWNERS", "content": "* @haifeng-jin @fchollet\n" }, { "path": ".github/CODE_OF_CONDUCT.md", "content": "# Contributor Covenant Code of Conduct\n\n## Our Pledge\n\nIn the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.\n\n## Our Standards\n\nExamples of behavior that contributes to creating a positive environment include:\n\n* Using welcoming and inclusive language\n* Being respectful of differing viewpoints and experiences\n* Gracefully accepting constructive criticism\n* Focusing on what is best for the community\n* Showing empathy towards other community members\n\nExamples of unacceptable behavior by participants include:\n\n* The use of sexualized language or imagery and unwelcome sexual attention or advances\n* Trolling, insulting/derogatory comments, and personal or political attacks\n* Public or private harassment\n* Publishing others' private information, such as a physical or electronic address, without explicit permission\n* Other conduct which could reasonably be considered inappropriate in a professional setting\n\n## Our Responsibilities\n\nProject maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.\n\nProject maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.\n\n## Scope\n\nThis Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.\n\n## Enforcement\n\nInstances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at jin@tamu.edu. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.\n\nProject maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.\n\n## Attribution\n\nThis Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version]\n\n[homepage]: http://contributor-covenant.org\n[version]: http://contributor-covenant.org/version/1/4/\n" }, { "path": ".github/CONTRIBUTING.md", "content": "# Contributing Guide\n\nContributions are welcome, and greatly appreciated!\nThis page is only a guide of the best practices of contributing code to AutoKeras.\nThe best way to contribute is to join our community by reading [this](https://autokeras.com/#contributing-code).\nWe will get you started right away.\n\nFollow the tag of [good first issue](https://github.com/keras-team/autokeras/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)\nfor the issues for beginner.\n\n## Pull Request Guide\n\n1. Is this the first pull request that you're making with GitHub? If so, read the guide [Making a pull request to an open-source project](https://github.com/gabrieldemarmiesse/getting_started_open_source).\n\n2. Include \"resolves #issue_number\" in the description of the pull request if applicable. Briefly describe your contribution.\n\n3. Submit the pull request from the first day of your development and create it as a [draft pull request](https://github.blog/2019-02-14-introducing-draft-pull-requests/). Click `ready for review` when finished and passed the all the checks.\n\n4. For the case of bug fixes, add new test cases which would fail before your bug fix.\n\n\n## Setup Environment\nWe introduce 3 different options: **GitHub Codespaces**, **VS Code & Dev Containers**, **the general setup**.\nYou can choose base on your preference.\n\n### Option 1: GitHub Codespaces\nYou can simply open the repository in GitHub Codespaces.\nThe environment is already setup there.\n\n### Option 2: VS Code & Dev Containers\nOpen VS Code.\nInstall the `Dev Containers` extension.\nPress `F1` key. Enter `Dev Containers: Open Folder in Container...` to open the repository root folder.\nThe environment is already setup there.\n\n### Option 3: The General Setup\n\nInstall [Virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/).\nCreate a new virtualenv named `ak` based on python3.\n```\nmkvirtualenv -p python3 ak \n```\nPlease use this virtualenv for development.\n\nClone the repo. Go to the repo directory.\nRun the following commands.\n```\nworkon ak\n\npip install -e \".[tests]\"\npip uninstall autokeras\nadd2virtualenv .\n``` \n\n## Run Tests\n\n### GitHub Codespaces or VS Code & Dev Containers\nIf you are using \"GitHub Codespaces\" or \"VS Code & Dev Containers\",\nyou can simply open any `*_test.py` file under the `tests` directory,\nand wait a few seconds, you will see the test tab on the left of the window.\n\n### General Setup\n\nIf you are using the general setup.\n\nActivate the virtualenv.\nGo to the repo directory\nRun the following lines to run the tests.\n\nRun all the tests.\n```\npytest tests\n```\n\nRun all the unit tests.\n```\npytest tests/autokeras\n```\n\nRun all the integration tests.\n```\npytest tests/integration_tests\n```\n\n## Code Style\nYou can run the following manually every time you want to format your code.\n1. Run `shell/format.sh` to format your code.\n2. Run `shell/lint.sh` to check.\n\n## Docstrings\nDocstrings should follow our style.\nJust check the style of other docstrings in AutoKeras.\n" }, { "path": ".github/ISSUE_TEMPLATE/bug_report.md", "content": "---\nname: Bug report\nabout: Create a report to help us improve\nlabels: [\"bug report\"]\ntitle: \"Bug: \"\n\n---\n\n### Bug Description\n\n\n### Bug Reproduction\nCode for reproducing the bug:\n\nData used by the code:\n\n### Expected Behavior\n\n\n### Setup Details\nInclude the details about the versions of:\n - OS type and version:\n - Python:\n - autokeras: \n - keras-tuner:\n - scikit-learn:\n - numpy:\n - pandas:\n - tensorflow:\n\n### Additional context\n\n" }, { "path": ".github/ISSUE_TEMPLATE/custom.md", "content": "---\nname: Custom issue template\nabout: Describe this issue template's purpose here.\n\n---\n\n\n" }, { "path": ".github/ISSUE_TEMPLATE/feature_request.md", "content": "---\nname: Feature request\nabout: Suggest an idea for this project\nlabels: [\"feature request\"]\ntitle: \"Feature: \"\n\n---\n\n### Feature Description\n\n### Code Example\n\n```python\n```\n\n### Reason\n\n\n### Solution\n\n" }, { "path": ".github/ISSUE_TEMPLATE/new-task-template.md", "content": "---\nname: New Task Template\nabout: Add a new machine learning task\nlabels: [\"algorithm\"]\ntitle: \"New Task: \"\n\n---\n\n\n\n### Suggested Name\n\n\n### Task Description\n\n\n### Evaluation Metrics\n\n\n### Benchmark Datasets\n\n\n### Reason\n\n\n### Solution\n\n\n### Additional Context\n\n" }, { "path": ".github/PULL_REQUEST_TEMPLATE.md", "content": "\n### Which issue(s) does this Pull Request fix?\n\nresolves #000\n\n### Details of the Pull Request\n\n\n\n" }, { "path": ".github/dependabot.yml", "content": "version: 2\nupdates:\n- package-ecosystem: pip\n directory: \"/\"\n schedule:\n interval: daily\n time: \"11:00\"\n open-pull-requests-limit: 10\n ignore:\n - dependency-name: sphinx\n versions:\n - \">= 3.1.a, < 3.2\"\n - dependency-name: typeguard\n versions:\n - \">= 2.11.a, < 2.12\"\n - dependency-name: typeguard\n versions:\n - \">= 2.12.a, < 2.13\"\n" }, { "path": ".github/workflows/actions.yml", "content": "name: Tests\n\non:\n push:\n branches: [ master ]\n pull_request:\n release:\n types: [created]\njobs:\n build:\n name: Run tests\n runs-on: ubuntu-latest\n env:\n KERAS_BACKEND: torch\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python 3.10\n uses: actions/setup-python@v4\n with:\n python-version: '3.10'\n - name: Get pip cache dir\n id: pip-cache\n run: |\n python -m pip install --upgrade pip setuptools\n echo \"dir=$(pip cache dir)\" >> $GITHUB_OUTPUT\n - name: pip cache\n uses: actions/cache@v3\n with:\n path: ${{ steps.pip-cache.outputs.dir }}\n key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}\n - name: Install dependencies\n run: |\n pip install torch --index-url https://download.pytorch.org/whl/cpu\n pip install -e \".[tests]\" --progress-bar off\n - name: Test with pytest\n run: |\n pytest --cov=autokeras --cov-report xml:coverage.xml\n - name: Codecov\n uses: codecov/codecov-action@v3\n with:\n token: ${{ secrets.CODECOV_TOKEN }}\n files: ./coverage.xml\n flags: unittests\n fail_ci_if_error: true\n format:\n name: Check the code format\n runs-on: ubuntu-latest\n env:\n KERAS_BACKEND: torch\n steps:\n - uses: actions/checkout@v3\n - name: Run pre-commit\n run: bash shell/pre-commit.sh\n build-docs:\n name: Build the docs\n runs-on: ubuntu-latest\n env:\n KERAS_BACKEND: torch\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python 3.10\n uses: actions/setup-python@v4\n with:\n python-version: '3.10'\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip setuptools\n pip install torch --index-url https://download.pytorch.org/whl/cpu\n pip install -e .\n pip install -r docs/requirements.txt\n - name: Build the docs\n run: |\n cd docs\n python autogen.py\n mkdocs build\n deploy:\n needs: [build, format, build-docs]\n if: github.event_name == 'release' && github.event.action == 'created'\n runs-on: ubuntu-latest\n env:\n KERAS_BACKEND: torch\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python\n uses: actions/setup-python@v4\n with:\n python-version: '3.10'\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip build twine\n - name: Build and publish\n env:\n TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}\n TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}\n run: |\n python -m build\n twine upload dist/*\n" }, { "path": ".gitignore", "content": "# vim swp files\n*.swp\n# caffe/pytorch model files\n*.pth\n\n# Mkdocs\n/docs/sources\n/docs/site\n\n.DS_Store\n.vscode\nsettings.json\n.idea\n.pytest_cache\n/experiments\n\n# resource temp folder\ntests/resources/temp/*\n!tests/resources/temp/.gitkeep\n\n# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n\n# PyInstaller\n# Usually these files are written by a python script from a template\n# before PyInstaller builds the exe, so as to inject date/other infos into it.\n*.manifest\n*.spec\n\n# Installer logs\npip-log.txt\npip-delete-this-directory.txt\n\n# Unit test / coverage reports\nhtmlcov/\n.tox/\n.coverage\n.coverage.*\ncov.xml\n.cache\nnosetests.xml\ncoverage.xml\n*.cover\n.hypothesis/\n\n# Translations\n*.mo\n*.pot\n\n# Django stuff:\n*.log\n.static_storage/\n.media/\nlocal_settings.py\n\n# Flask stuff:\ninstance/\n.webassets-cache\n\n# Scrapy stuff:\n.scrapy\n\n# Sphinx documentation\ndocs/_build/\n\n# PyBuilder\ntarget/\n\n# Jupyter Notebook\n.ipynb_checkpoints\n\n# pyenv\n.python-version\n\n# celery beat schedule file\ncelerybeat-schedule\n\n# SageMath parsed files\n*.sage.py\n\n# Environments\n.env\n.venv\nenv/\nvenv/\nENV/\nenv.bak/\nvenv.bak/\n\n# Spyder project settings\n.spyderproject\n.spyproject\n\n# Rope project settings\n.ropeproject\n\n# mkdocs documentation\n/site\n\n# mypy\n.mypy_cache/\n\nexamples/text_cnn/glove_embedding/\n" }, { "path": "LICENSE", "content": "\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2020 The AutoKeras Authors.\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n\n" }, { "path": "README.md", "content": "

\n \"logo\"\n

\n\n[![](https://github.com/keras-team/autokeras/workflows/Tests/badge.svg)](https://github.com/keras-team/autokeras/actions?query=workflow%3ATests+branch%3Amaster)\n[![codecov](https://codecov.io/gh/keras-team/autokeras/branch/master/graph/badge.svg)](https://codecov.io/gh/keras-team/autokeras)\n[![PyPI version](https://badge.fury.io/py/autokeras.svg)](https://badge.fury.io/py/autokeras)\n[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/keras-team/autokeras/issues)\n\nOfficial Website: [autokeras.com](https://autokeras.com)\n\n##\nAutoKeras: An AutoML system based on Keras.\nIt is developed by DATA Lab at Texas A&M University.\nThe goal of AutoKeras is to make machine learning accessible to everyone.\n\n## Learning resources\n\n* A short example.\n\n```python\nimport autokeras as ak\n\nclf = ak.ImageClassifier()\nclf.fit(x_train, y_train)\nresults = clf.predict(x_test)\n```\n\n* [Official website tutorials](https://autokeras.com/tutorial/overview/).\n* The book of [*Automated Machine Learning in Action*](https://www.manning.com/books/automated-machine-learning-in-action?query=automated&utm_source=jin&utm_medium=affiliate&utm_campaign=affiliate&a_aid=jin).\n* The LiveProjects of [*Image Classification with AutoKeras*](https://www.manning.com/liveprojectseries/autokeras-ser).\n

\n\"drawing\"\n \n \n\"drawing\"\n

\n\n\n## Installation\n\nTo install the package, please use the `pip` installation as follows:\n\n```shell\npip3 install autokeras\n```\n\nPlease follow the [installation guide](https://autokeras.com/install) for more details.\n\n**Note:** Currently, AutoKeras is only compatible with **Python >= 3.7** and **TensorFlow >= 2.8.0**.\n\n## Community\n\nAsk your questions on our [GitHub Discussions](https://github.com/keras-team/autokeras/discussions).\n\n## Contributing Code\n\nHere is how we manage our project.\n\nWe pick the critical issues to work on from [GitHub issues](https://github.com/keras-team/autokeras/issues).\nThey will be added to this [Project](https://github.com/keras-team/autokeras/projects/3).\nSome of the issues will then be added to the [milestones](https://github.com/keras-team/autokeras/milestones),\nwhich are used to plan for the releases.\n\nRefer to our [Contributing Guide](https://autokeras.com/contributing/) to learn the best practices.\n\nThank all the contributors!\n\n[![The contributors](https://autokeras.com/img/contributors.svg)](https://github.com/keras-team/autokeras/graphs/contributors)\n\n## Cite this work\n\nHaifeng Jin, François Chollet, Qingquan Song, and Xia Hu. \"AutoKeras: An AutoML Library for Deep Learning.\" *the Journal of machine Learning research* 6 (2023): 1-6. ([Download](http://jmlr.org/papers/v24/20-1355.html))\n\nBiblatex entry:\n\n```bibtex\n@article{JMLR:v24:20-1355,\n author = {Haifeng Jin and François Chollet and Qingquan Song and Xia Hu},\n title = {AutoKeras: An AutoML Library for Deep Learning},\n journal = {Journal of Machine Learning Research},\n year = {2023},\n volume = {24},\n number = {6},\n pages = {1--6},\n url = {http://jmlr.org/papers/v24/20-1355.html}\n}\n```\n\n## Acknowledgements\n\nThe authors gratefully acknowledge the D3M program of the Defense Advanced Research Projects Agency (DARPA) administered through AFRL contract FA8750-17-2-0116; the Texas A&M College of Engineering, and Texas A&M University.\n" }, { "path": "RELEASE.md", "content": "# Release v2.0.0\n\n## Breaking changes\n\n* Requires `keras>=3.0.0` instead of `tf.keras`.\n* Removed the structured data related tasks by removing the following public\n APIs:\n * `CategoricalToNumerical`\n * `MultiCategoryEncoding`\n * `StructuredDataInput`\n * `StructuredDataBlock`\n * `StructuredDataClassifier`\n * `StructuredDataRegressor`\n* Removed the Time series related tasks by removing the following public APIs:\n * `TimeseriesInput`\n * `TimeseriesForecaster`\n* Reduced search space of Text related tasks by removing the following blocks.\n * `Embedding`\n * `TextToIntSequence`\n * `TextToNgramVector`\n * `Transformer`\n\n# Release v1.1.0\n\n## Breaking changes\n\n* This only affect you if you use `BertTokenizer` or `BertEncoder` in AutoKeras\n explicity. You are not affected if you only use `BertBlock`, `TextClassifier`\n or `TextRegressor`. Removed the AutoKeras implementation of `BertTokenizer`\n and `BertEncoder`. Use `keras-nlp` implementation instead.\n\n## New features\n\n## Bug fixes\n* Now also support `numpy>=1.24`.\n" }, { "path": "autokeras/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.auto_model import AutoModel\nfrom autokeras.blocks import ClassificationHead\nfrom autokeras.blocks import ConvBlock\nfrom autokeras.blocks import DenseBlock\nfrom autokeras.blocks import EfficientNetBlock\nfrom autokeras.blocks import Embedding\nfrom autokeras.blocks import Flatten\nfrom autokeras.blocks import ImageAugmentation\nfrom autokeras.blocks import ImageBlock\nfrom autokeras.blocks import Merge\nfrom autokeras.blocks import Normalization\nfrom autokeras.blocks import RegressionHead\nfrom autokeras.blocks import ResNetBlock\nfrom autokeras.blocks import RNNBlock\nfrom autokeras.blocks import SpatialReduction\nfrom autokeras.blocks import StructuredDataBlock\nfrom autokeras.blocks import TemporalReduction\nfrom autokeras.blocks import TextBlock\nfrom autokeras.blocks import XceptionBlock\nfrom autokeras.engine.block import Block\nfrom autokeras.engine.head import Head\nfrom autokeras.engine.node import Node\nfrom autokeras.keras_layers import CastToFloat32\nfrom autokeras.keras_layers import ExpandLastDim\nfrom autokeras.nodes import ImageInput\nfrom autokeras.nodes import Input\nfrom autokeras.nodes import StructuredDataInput\nfrom autokeras.nodes import TextInput\nfrom autokeras.tasks import ImageClassifier\nfrom autokeras.tasks import ImageRegressor\nfrom autokeras.tasks import StructuredDataClassifier\nfrom autokeras.tasks import StructuredDataRegressor\nfrom autokeras.tasks import TextClassifier\nfrom autokeras.tasks import TextRegressor\nfrom autokeras.tuners import BayesianOptimization\nfrom autokeras.tuners import Greedy\nfrom autokeras.tuners import Hyperband\nfrom autokeras.tuners import RandomSearch\n\n__version__ = \"3.0.0\"\n\nCUSTOM_OBJECTS = {\n \"CastToFloat32\": CastToFloat32,\n \"ExpandLastDim\": ExpandLastDim,\n}\n" }, { "path": "autokeras/adapters/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.adapters.input_adapters import ImageAdapter\nfrom autokeras.adapters.input_adapters import InputAdapter\nfrom autokeras.adapters.input_adapters import StructuredDataAdapter\nfrom autokeras.adapters.input_adapters import TextAdapter\nfrom autokeras.adapters.output_adapters import ClassificationAdapter\nfrom autokeras.adapters.output_adapters import RegressionAdapter\n" }, { "path": "autokeras/adapters/input_adapters.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras.engine import adapter as adapter_module\n\n\nclass InputAdapter(adapter_module.Adapter):\n def check(self, x):\n \"\"\"Record any information needed by transform.\"\"\"\n if not isinstance(x, np.ndarray):\n raise TypeError(\n \"Expect the data to Input to be numpy.ndarray, \"\n \"but got {type}.\".format(type=type(x))\n )\n if isinstance(x, np.ndarray) and not np.issubdtype(x.dtype, np.number):\n raise TypeError(\n \"Expect the data to Input to be numerical, but got \"\n \"{type}.\".format(type=x.dtype)\n )\n\n\nclass ImageAdapter(adapter_module.Adapter):\n def check(self, x):\n \"\"\"Record any information needed by transform.\"\"\"\n if not isinstance(x, np.ndarray):\n raise TypeError(\n \"Expect the data to ImageInput to be numpy.ndarray, \"\n \"but got {type}.\".format(type=type(x))\n )\n if isinstance(x, np.ndarray) and not np.issubdtype(x.dtype, np.number):\n raise TypeError(\n \"Expect the data to ImageInput to be numerical, but got \"\n \"{type}.\".format(type=x.dtype)\n )\n\n\nclass TextAdapter(adapter_module.Adapter):\n def check(self, x):\n \"\"\"Record any information needed by transform.\"\"\"\n if not isinstance(x, np.ndarray):\n raise TypeError(\n \"Expect the data to TextInput to be numpy.ndarray, \"\n \"but got {type}.\".format(type=type(x))\n )\n\n\nclass StructuredDataAdapter(adapter_module.Adapter):\n def check(self, x):\n if not isinstance(x, np.ndarray):\n raise TypeError(\n \"Unsupported type {type} for \"\n \"{name}.\".format(type=type(x), name=self.__class__.__name__)\n )\n" }, { "path": "autokeras/adapters/input_adapters_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom autokeras import test_utils\nfrom autokeras.adapters import input_adapters\nfrom autokeras.utils import data_utils\n\n\ndef test_image_input_adapter_transform_to_dataset():\n x = test_utils.generate_data()\n adapter = input_adapters.ImageAdapter()\n assert isinstance(adapter.adapt(x), np.ndarray)\n\n\ndef test_image_input_unsupported_type():\n x = \"unknown\"\n adapter = input_adapters.ImageAdapter()\n with pytest.raises(TypeError) as info:\n x = adapter.adapt(x)\n assert \"Expect the data to ImageInput to be numpy\" in str(info.value)\n\n\ndef test_image_input_numerical():\n x = np.array([[[\"unknown\"]]])\n adapter = input_adapters.ImageAdapter()\n with pytest.raises(TypeError) as info:\n x = adapter.adapt(x)\n assert \"Expect the data to ImageInput to be numerical\" in str(info.value)\n\n\ndef test_input_type_error():\n x = \"unknown\"\n adapter = input_adapters.InputAdapter()\n with pytest.raises(TypeError) as info:\n x = adapter.adapt(x)\n assert \"Expect the data to Input to be numpy\" in str(info.value)\n\n\ndef test_input_numerical():\n x = np.array([[[\"unknown\"]]])\n adapter = input_adapters.InputAdapter()\n with pytest.raises(TypeError) as info:\n x = adapter.adapt(x)\n assert \"Expect the data to Input to be numerical\" in str(info.value)\n\n\ndef test_text_adapt_np():\n x = np.array([\"a b c\", \"b b c\"])\n adapter = input_adapters.TextAdapter()\n x = adapter.adapt(x)\n\n assert data_utils.dataset_shape(x) == [2]\n assert isinstance(x, np.ndarray)\n\n\ndef test_text_input_type_error():\n x = \"unknown\"\n adapter = input_adapters.TextAdapter()\n with pytest.raises(TypeError) as info:\n x = adapter.adapt(x)\n assert \"Expect the data to TextInput to be numpy\" in str(info.value)\n\n\ndef test_structured_data_input_unsupported_type_error():\n with pytest.raises(TypeError) as info:\n adapter = input_adapters.StructuredDataAdapter()\n adapter.adapt(\"unknown\")\n\n assert \"Unsupported type\" in str(info.value)\n\n\ndef test_structured_data_input_transform_to_dataset():\n x = pd.read_csv(test_utils.TRAIN_CSV_PATH).to_numpy().astype(str)\n adapter = input_adapters.StructuredDataAdapter()\n\n x = adapter.adapt(x)\n\n assert isinstance(x, np.ndarray)\n" }, { "path": "autokeras/adapters/output_adapters.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras.engine import adapter as adapter_module\n\n\nclass HeadAdapter(adapter_module.Adapter):\n def __init__(self, name, **kwargs):\n super().__init__(**kwargs)\n self.name = name\n\n def check(self, dataset):\n if not isinstance(dataset, np.ndarray):\n raise TypeError(\n f\"Expect the target data of {self.name} to be\"\n f\" np.ndarray, but got {type(dataset)}.\"\n )\n\n\nclass ClassificationAdapter(HeadAdapter):\n pass\n\n\nclass RegressionAdapter(HeadAdapter):\n pass\n" }, { "path": "autokeras/adapters/output_adapters_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport pytest\n\nfrom autokeras.adapters import output_adapters\n\n\ndef test_unsupported_types_error():\n adapter = output_adapters.ClassificationAdapter(name=\"a\")\n\n with pytest.raises(TypeError) as info:\n adapter.adapt(1)\n\n assert \"Expect the target data of a to be\" in str(info.value)\n\n\ndef test_reg_head_transform_1d_np():\n adapter = output_adapters.RegressionAdapter(name=\"a\")\n\n y = adapter.adapt(np.random.rand(10))\n\n assert isinstance(y, np.ndarray)\n" }, { "path": "autokeras/analysers/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.analysers.input_analysers import CATEGORICAL\nfrom autokeras.analysers.input_analysers import NUMERICAL\nfrom autokeras.analysers.input_analysers import ImageAnalyser\nfrom autokeras.analysers.input_analysers import InputAnalyser\nfrom autokeras.analysers.input_analysers import StructuredDataAnalyser\nfrom autokeras.analysers.input_analysers import TextAnalyser\nfrom autokeras.analysers.output_analysers import ClassificationAnalyser\nfrom autokeras.analysers.output_analysers import RegressionAnalyser\n" }, { "path": "autokeras/analysers/input_analysers.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport numpy as np\n\nfrom autokeras.engine import analyser\n\nCATEGORICAL = \"categorical\"\nNUMERICAL = \"numerical\"\n\n\nclass InputAnalyser(analyser.Analyser):\n def finalize(self):\n return\n\n\nclass ImageAnalyser(InputAnalyser):\n def __init__(self, **kwargs):\n super().__init__(**kwargs)\n\n def finalize(self):\n if len(self.shape) not in [3, 4]:\n raise ValueError(\n \"Expect the data to ImageInput to have shape (batch_size, \"\n \"height, width, channels) or (batch_size, height, width) \"\n \"dimensions, but got input shape {shape}\".format(\n shape=self.shape\n )\n )\n\n\nclass TextAnalyser(InputAnalyser):\n def correct_shape(self):\n if len(self.shape) == 1:\n return True\n return len(self.shape) == 2 and self.shape[1] == 1\n\n def finalize(self):\n if not self.correct_shape():\n raise ValueError(\n \"Expect the data to TextInput to have shape \"\n \"(batch_size, 1), but \"\n \"got input shape {shape}.\".format(shape=self.shape)\n )\n if not (self.dtype == \"string\"):\n raise TypeError(\n \"Expect the data to TextInput to be strings, but got \"\n \"{type}.\".format(type=self.dtype)\n )\n\n\nclass StructuredDataAnalyser(InputAnalyser):\n def __init__(self, column_names=None, column_types=None, **kwargs):\n super().__init__(**kwargs)\n self.column_names = column_names\n self.column_types = column_types\n # Variables for inferring column types.\n self.count_numerical = None\n self.count_categorical = None\n self.count_unique_numerical = []\n self.num_col = None\n\n def update(self, data):\n super().update(data)\n # The super class set self.dtype to \"string\" based on the input numpy\n # array. However, the preprocessor will encode it to float32. So, set\n # self.dtype to \"float32\", which would be propagated to the Keras Input\n # node.\n self.dtype = \"float32\"\n if len(self.shape) != 2:\n return\n # data is a numpy array containing all the data.\n self.num_col = data.shape[1]\n self.count_numerical = np.zeros(self.num_col)\n self.count_categorical = np.zeros(self.num_col)\n self.count_unique_numerical = [{} for _ in range(self.num_col)]\n for i in range(self.num_col):\n self._update_column(data[:, i], i)\n\n def _update_column(self, column_data, i):\n # Vectorized check for numerical values\n def is_numerical(x):\n try:\n float(x)\n return True\n except (ValueError, TypeError):\n return False\n\n numerical_mask = np.vectorize(is_numerical)(column_data)\n self.count_numerical[i] = np.sum(numerical_mask)\n self.count_categorical[i] = len(column_data) - np.sum(numerical_mask)\n\n if np.any(numerical_mask):\n # Get numerical values\n numerical_values = column_data[numerical_mask]\n # Handle bytes\n numerical_values = np.array(\n [\n x.decode(\"utf-8\") if isinstance(x, bytes) else x\n for x in numerical_values\n ]\n )\n numerical_floats = numerical_values.astype(float)\n unique_vals, counts = np.unique(\n numerical_floats, return_counts=True\n )\n self.count_unique_numerical[i] = dict(zip(unique_vals, counts))\n\n def finalize(self):\n self.check()\n self.infer_column_types()\n\n def get_input_name(self):\n return \"StructuredDataInput\"\n\n def check(self):\n if len(self.shape) != 2:\n raise ValueError(\n \"Expect the data to {input_name} to have shape \"\n \"(batch_size, num_features), but \"\n \"got input shape {shape}.\".format(\n input_name=self.get_input_name(), shape=self.shape\n )\n )\n\n # Fill in the column_names\n if self.column_names is None:\n if self.column_types:\n raise ValueError(\n \"column_names must be specified, if \"\n \"column_types is specified.\"\n )\n self.column_names = [str(index) for index in range(self.shape[1])]\n\n # Check if column_names has the correct length.\n if len(self.column_names) != self.shape[1]:\n raise ValueError(\n \"Expect column_names to have length {expect} \"\n \"but got {actual}.\".format(\n expect=self.shape[1], actual=len(self.column_names)\n )\n )\n\n def infer_column_types(self):\n column_types = {}\n\n for i in range(self.num_col):\n if self.count_categorical[i] > 0:\n column_types[self.column_names[i]] = CATEGORICAL\n elif (\n len(self.count_unique_numerical[i]) / self.count_numerical[i]\n < 0.05\n ):\n column_types[self.column_names[i]] = CATEGORICAL\n else:\n column_types[self.column_names[i]] = NUMERICAL\n\n # Partial column_types is provided.\n if self.column_types is None:\n self.column_types = {}\n for key, value in column_types.items():\n if key not in self.column_types:\n self.column_types[key] = value\n" }, { "path": "autokeras/analysers/input_analysers_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport copy\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nfrom autokeras import test_utils\nfrom autokeras.analysers import input_analysers\n\n\ndef test_structured_data_input_less_col_name_error():\n with pytest.raises(ValueError) as info:\n analyser = input_analysers.StructuredDataAnalyser(\n column_names=list(range(8))\n )\n dataset = np.random.rand(20, 10)\n analyser.update(dataset)\n\n analyser.finalize()\n\n assert \"Expect column_names to have length\" in str(info.value)\n\n\ndef test_structured_data_infer_col_types():\n analyser = input_analysers.StructuredDataAnalyser(\n column_names=test_utils.COLUMN_NAMES,\n column_types=None,\n )\n x = pd.read_csv(test_utils.TRAIN_CSV_PATH)\n x.pop(\"survived\")\n dataset = x.values.astype(str)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.column_types == test_utils.COLUMN_TYPES\n\n\ndef test_dont_infer_specified_column_types():\n column_types = copy.copy(test_utils.COLUMN_TYPES)\n column_types.pop(\"sex\")\n column_types[\"age\"] = \"categorical\"\n\n analyser = input_analysers.StructuredDataAnalyser(\n column_names=test_utils.COLUMN_NAMES,\n column_types=column_types,\n )\n x = pd.read_csv(test_utils.TRAIN_CSV_PATH)\n x.pop(\"survived\")\n dataset = x.values.astype(str)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.column_types[\"age\"] == \"categorical\"\n\n\ndef test_structured_data_input_with_illegal_dim():\n analyser = input_analysers.StructuredDataAnalyser(\n column_names=test_utils.COLUMN_NAMES,\n column_types=None,\n )\n dataset = np.random.rand(100, 32, 32)\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the data to StructuredDataInput to have shape\" in str(\n info.value\n )\n\n\ndef test_image_input_analyser_shape_is_list_of_int():\n analyser = input_analysers.ImageAnalyser()\n dataset = np.random.rand(100, 32, 32, 3)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert isinstance(analyser.shape, list)\n assert all(map(lambda x: isinstance(x, int), analyser.shape))\n\n\ndef test_image_input_with_three_dim():\n analyser = input_analysers.ImageAnalyser()\n dataset = np.random.rand(100, 32, 32)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert len(analyser.shape) == 3\n\n\ndef test_image_input_with_illegal_dim():\n analyser = input_analysers.ImageAnalyser()\n dataset = np.random.rand(100, 32)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the data to ImageInput to have shape\" in str(info.value)\n\n\ndef test_text_input_with_illegal_dim():\n analyser = input_analysers.TextAnalyser()\n dataset = np.random.rand(100, 32)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the data to TextInput to have shape\" in str(info.value)\n\n\ndef test_text_analyzer_with_one_dim_doesnt_crash():\n analyser = input_analysers.TextAnalyser()\n dataset = np.array([\"a b c\", \"b b c\"])\n\n analyser.update(dataset)\n analyser.finalize()\n\n\ndef test_text_illegal_type_error():\n analyser = input_analysers.TextAnalyser()\n dataset = np.random.rand(100, 1)\n\n with pytest.raises(TypeError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the data to TextInput to be strings\" in str(info.value)\n" }, { "path": "autokeras/analysers/output_analysers.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport numpy as np\n\nfrom autokeras.engine import analyser\n\n\nclass TargetAnalyser(analyser.Analyser):\n def __init__(self, name=None, **kwargs):\n super().__init__(**kwargs)\n self.name = name\n\n\nclass ClassificationAnalyser(TargetAnalyser):\n def __init__(self, num_classes=None, multi_label=False, **kwargs):\n super().__init__(**kwargs)\n self.num_classes = num_classes\n self.label_encoder = None\n self.multi_label = multi_label\n self.labels = set()\n\n def update(self, data):\n super().update(data)\n if len(self.shape) > 2:\n raise ValueError(\n \"Expect the target data for {name} to have shape \"\n \"(batch_size, num_classes), \"\n \"but got {shape}.\".format(name=self.name, shape=self.shape)\n )\n if len(self.shape) > 1 and self.shape[1] > 1:\n return\n self.labels = self.labels.union(set(np.unique(data)))\n\n def finalize(self):\n # TODO: support raw string labels for multi-label.\n self.labels = sorted(list(self.labels))\n\n # Infer the num_classes if not specified.\n if not self.num_classes:\n if self.encoded:\n # Single column with 0s and 1s.\n if len(self.shape) == 1 or self.shape[1:] == [1]:\n self.num_classes = 2\n else:\n self.num_classes = self.shape[1]\n else:\n self.num_classes = len(self.labels)\n\n if self.num_classes < 2:\n raise ValueError(\n \"Expect the target data for {name} to have \"\n \"at least 2 classes, but got {num_classes}.\".format(\n name=self.name, num_classes=self.num_classes\n )\n )\n\n # Check shape equals expected shape.\n expected = self.get_expected_shape()\n actual = self.shape[1:]\n if len(actual) == 0:\n actual = [1]\n if self.encoded and actual != expected:\n raise ValueError(\n \"Expect the target data for {name} to have \"\n \"shape {expected}, but got {actual}.\".format(\n name=self.name, expected=expected, actual=self.shape[1:]\n )\n )\n\n def get_expected_shape(self):\n # Compute expected shape from num_classes.\n if self.num_classes == 2 and not self.multi_label:\n return [1]\n return [self.num_classes]\n\n @property\n def encoded(self):\n return self.encoded_for_sigmoid or self.encoded_for_softmax\n\n @property\n def encoded_for_sigmoid(self):\n if len(self.labels) != 2:\n return False\n return sorted(self.labels) == [0, 1]\n\n @property\n def encoded_for_softmax(self):\n return len(self.shape) > 1 and self.shape[1] > 1\n\n\nclass RegressionAnalyser(TargetAnalyser):\n def __init__(self, output_dim=None, **kwargs):\n super().__init__(**kwargs)\n self.output_dim = output_dim\n\n def finalize(self):\n if self.output_dim and (self.expected_dim() != self.output_dim):\n raise ValueError(\n \"Expect the target data for {name} to have shape \"\n \"(batch_size, {output_dim}), \"\n \"but got {shape}.\".format(\n name=self.name, output_dim=self.output_dim, shape=self.shape\n )\n )\n\n def expected_dim(self):\n if len(self.shape) == 1:\n return 1\n return self.shape[1]\n" }, { "path": "autokeras/analysers/output_analysers_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport pytest\n\nfrom autokeras import test_utils\nfrom autokeras.analysers import output_analysers\n\n\ndef test_clf_head_one_hot_shape_error():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\", num_classes=9)\n dataset = test_utils.generate_one_hot_labels(num_classes=10)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the target data for a to have shape\" in str(info.value)\n\n\ndef test_clf_head_more_dim_error():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\", num_classes=9)\n dataset = np.random.rand(100, 32, 32, 3)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the target data for a to have shape\" in str(info.value)\n\n\ndef test_wrong_num_classes_error():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\", num_classes=5)\n dataset = np.random.rand(10, 3)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the target data for a to have shape\" in str(info.value)\n\n\ndef test_one_class_error():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\")\n dataset = np.array([\"a\", \"a\", \"a\"])\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the target data for a to have at least 2 classes\" in str(\n info.value\n )\n\n\ndef test_infer_ten_classes():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\")\n dataset = test_utils.generate_one_hot_labels(num_classes=10)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.num_classes == 10\n\n\ndef test_infer_single_column_two_classes():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\")\n dataset = np.random.randint(0, 2, 10)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.num_classes == 2\n\n\ndef test_specify_five_classes():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\", num_classes=5)\n dataset = np.random.rand(10, 5)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.num_classes == 5\n\n\ndef test_specify_two_classes_fit_single_column():\n analyser = output_analysers.ClassificationAnalyser(name=\"a\", num_classes=2)\n dataset = np.random.rand(10, 1)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.num_classes == 2\n\n\ndef test_multi_label_two_classes_has_two_columns():\n analyser = output_analysers.ClassificationAnalyser(\n name=\"a\", multi_label=True\n )\n dataset = np.random.rand(10, 2)\n\n analyser.update(dataset)\n analyser.finalize()\n\n assert analyser.encoded\n\n\ndef test_reg_with_specified_output_dim_error():\n analyser = output_analysers.RegressionAnalyser(name=\"a\", output_dim=3)\n dataset = np.random.rand(10, 2)\n\n with pytest.raises(ValueError) as info:\n analyser.update(dataset)\n analyser.finalize()\n\n assert \"Expect the target data for a to have shape\" in str(info.value)\n\n\ndef test_reg_with_specified_output_dim_and_single_column_doesnt_crash():\n analyser = output_analysers.RegressionAnalyser(name=\"a\", output_dim=1)\n dataset = np.random.rand(10, 1)\n\n analyser.update(dataset)\n analyser.finalize()\n\n\ndef test_regression_analyser_expected_dim_1d():\n analyser = output_analysers.RegressionAnalyser()\n analyser.shape = [10] # 1D shape\n assert analyser.expected_dim() == 1\n\n\ndef test_regression_analyser_expected_dim_2d():\n analyser = output_analysers.RegressionAnalyser()\n analyser.shape = [10, 3] # 2D shape\n assert analyser.expected_dim() == 3\n" }, { "path": "autokeras/auto_model.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom pathlib import Path\nfrom typing import List\nfrom typing import Optional\nfrom typing import Type\nfrom typing import Union\n\nimport keras\nimport numpy as np\nimport tree\n\nfrom autokeras import blocks\nfrom autokeras import graph as graph_module\nfrom autokeras import pipeline\nfrom autokeras import tuners\nfrom autokeras.engine import head as head_module\nfrom autokeras.engine import node as node_module\nfrom autokeras.engine import tuner\nfrom autokeras.nodes import Input\nfrom autokeras.utils import data_utils\nfrom autokeras.utils import utils\n\nTUNER_CLASSES = {\n \"bayesian\": tuners.BayesianOptimization,\n \"random\": tuners.RandomSearch,\n \"hyperband\": tuners.Hyperband,\n \"greedy\": tuners.Greedy,\n}\n\n\ndef get_tuner_class(tuner):\n if isinstance(tuner, str) and tuner in TUNER_CLASSES:\n return TUNER_CLASSES.get(tuner)\n else:\n raise ValueError(\n 'Expected the tuner argument to be one of \"greedy\", '\n '\"random\", \"hyperband\", or \"bayesian\", '\n \"but got {tuner}\".format(tuner=tuner)\n )\n\n\nclass AutoModel(object):\n \"\"\"A Model defined by inputs and outputs.\n AutoModel combines a HyperModel and a Tuner to tune the HyperModel.\n The user can use it in a similar way to a Keras model since it\n also has `fit()` and `predict()` methods.\n\n The AutoModel has two use cases. In the first case, the user only specifies\n the input nodes and output heads of the AutoModel. The AutoModel infers the\n rest part of the model. In the second case, user can specify the high-level\n architecture of the AutoModel by connecting the Blocks with the functional\n API, which is the same as the Keras\n [functional\n API](https://keras.io/api/models/model/#with-the-functional-api).\n\n # Example\n ```python\n # The user only specifies the input nodes and output heads.\n import autokeras as ak\n ak.AutoModel(\n inputs=[ak.ImageInput(), ak.TextInput()],\n outputs=[ak.ClassificationHead(), ak.RegressionHead()]\n )\n ```\n ```python\n # The user specifies the high-level architecture.\n import autokeras as ak\n image_input = ak.ImageInput()\n image_output = ak.ImageBlock()(image_input)\n text_input = ak.TextInput()\n text_output = ak.TextBlock()(text_input)\n output = ak.Merge()([image_output, text_output])\n classification_output = ak.ClassificationHead()(output)\n regression_output = ak.RegressionHead()(output)\n ak.AutoModel(\n inputs=[image_input, text_input],\n outputs=[classification_output, regression_output]\n )\n ```\n\n # Arguments\n inputs: A list of Node instances.\n The input node(s) of the AutoModel.\n outputs: A list of Node or Head instances.\n The output node(s) or head(s) of the AutoModel.\n project_name: String. The name of the AutoModel. Defaults to\n 'auto_model'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. Defaults to 'greedy'.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by keras_tuner.Tuner.\n \"\"\"\n\n def __init__(\n self,\n inputs: Union[Input, List[Input]],\n outputs: Union[head_module.Head, node_module.Node, list],\n project_name: str = \"auto_model\",\n max_trials: int = 100,\n directory: Union[str, Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = \"greedy\",\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n self.inputs = tree.flatten(inputs)\n self.outputs = tree.flatten(outputs)\n self.seed = seed\n if seed:\n np.random.seed(seed)\n # TODO: Support passing a tuner instance.\n # Initialize the hyper_graph.\n graph = self._build_graph()\n if isinstance(tuner, str):\n tuner = get_tuner_class(tuner)\n self.tuner = tuner(\n hypermodel=graph,\n overwrite=overwrite,\n objective=objective,\n max_trials=max_trials,\n directory=directory,\n seed=self.seed,\n project_name=project_name,\n max_model_size=max_model_size,\n **kwargs\n )\n self.overwrite = overwrite\n self._heads = [output_node.in_blocks[0] for output_node in self.outputs]\n\n @property\n def objective(self):\n return self.tuner.objective\n\n @property\n def max_trials(self):\n return self.tuner.max_trials\n\n @property\n def directory(self):\n return self.tuner.directory\n\n @property\n def project_name(self):\n return self.tuner.project_name\n\n def _assemble(self):\n \"\"\"Assemble the Blocks based on the input output nodes.\"\"\"\n inputs = tree.flatten(self.inputs)\n outputs = tree.flatten(self.outputs)\n\n middle_nodes = [\n input_node.get_block()(input_node) for input_node in inputs\n ]\n\n # Merge the middle nodes.\n if len(middle_nodes) > 1:\n output_node = blocks.Merge()(middle_nodes)\n else:\n output_node = middle_nodes[0]\n\n outputs = tree.flatten(\n [output_blocks(output_node) for output_blocks in outputs]\n )\n return graph_module.Graph(inputs=inputs, outputs=outputs)\n\n def _build_graph(self):\n # Using functional API.\n if all(\n [isinstance(output, node_module.Node) for output in self.outputs]\n ):\n graph = graph_module.Graph(inputs=self.inputs, outputs=self.outputs)\n # Using input/output API.\n elif all(\n [isinstance(output, head_module.Head) for output in self.outputs]\n ):\n # Clear session to reset get_uid(). The names of the blocks will\n # start to count from 1 for new blocks in a new AutoModel\n # afterwards. When initializing multiple AutoModel with Task API,\n # if not counting from 1 for each of the AutoModel, the predefined\n # hp values in task specifiec tuners would not match the names.\n keras.backend.clear_session()\n graph = self._assemble()\n self.outputs = graph.outputs\n keras.backend.clear_session()\n\n return graph\n\n def fit(\n self,\n x=None,\n y=None,\n batch_size=32,\n epochs=None,\n callbacks=None,\n validation_split=0.2,\n validation_data=None,\n verbose=1,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n It will search for the best model based on the performances on\n validation data.\n\n # Arguments\n x: numpy.ndarray. Training data x.\n y: numpy.ndarray. Training data y.\n batch_size: Int. Number of samples per gradient update. Defaults to\n 32.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, by default we train for a maximum of\n 1000 epochs, but we stop training if the validation loss stops\n improving for 10 epochs (unless you specified an EarlyStopping\n callback as part of the callbacks argument, in which case the\n EarlyStopping callback you specified will determine early\n stopping).\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2.\n Fraction of the training data to be used as validation data.\n The model will set apart this fraction of the training data,\n will not train on it, and will evaluate the loss and any model\n metrics on this data at the end of each epoch. The validation\n data is selected from the last samples in the `x` and `y` data\n provided, before shuffling. This argument is not supported when\n `x` is a dataset. The best model found would be fit on the\n entire dataset including the validation data.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n verbose: 0, 1, or 2. Verbosity mode. 0 = silent, 1 = progress bar,\n 2 = one line per epoch. Note that the progress bar is not\n particularly useful when logged to a file, so verbose=2 is\n recommended when not running interactively (eg, in a production\n environment). Controls the verbosity of both KerasTuner search\n and\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method)\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n # Check validation information.\n if not validation_data and not validation_split:\n raise ValueError(\n \"Either validation_data or a non-zero validation_split \"\n \"should be provided.\"\n )\n\n if validation_data:\n validation_split = 0\n\n dataset, validation_data = self._check_and_adapt(\n x=x, y=y, validation_data=validation_data\n )\n self._analyze_data(dataset)\n self._build_hyper_pipeline()\n\n # Split the data with validation_split.\n if validation_data is None and validation_split:\n dataset, validation_data = data_utils.split_dataset(\n dataset, validation_split\n )\n\n x, y = dataset\n history = self.tuner.search(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_data=validation_data,\n validation_split=validation_split,\n verbose=verbose,\n batch_size=batch_size,\n **kwargs\n )\n\n return history\n\n def _adapt(self, dataset, hms):\n sources = tree.flatten(dataset)\n adapted = []\n for source, hm in zip(sources, hms):\n source = hm.get_adapter().adapt(source)\n adapted.append(source)\n if len(adapted) == 1:\n return adapted[0]\n return tuple(adapted)\n\n def _check_numpy_arrays(self, data, name, in_val=\"\"):\n \"\"\"Check if all elements in the nested structure are numpy arrays.\"\"\"\n if not all([isinstance(a, np.ndarray) for a in tree.flatten(data)]):\n raise ValueError(\n \"Expected \"\n \"{name}{in_val} to be a numpy array, got {type}\".format(\n name=name,\n in_val=in_val,\n type=[type(a) for a in tree.flatten(data)],\n )\n )\n\n def _check_array_count(self, actual, expected, name, in_val):\n \"\"\"Check if the number of arrays matches the expected count.\"\"\"\n if actual != expected:\n raise ValueError(\n \"Expected {name}{in_val} to have {expected} arrays, \"\n \"but got {actual}\".format(\n name=name,\n in_val=in_val,\n expected=expected,\n actual=actual,\n )\n )\n\n def _check_data_format(self, x, y, validation=False, predict=False):\n \"\"\"Check if the dataset has the same number of IOs with the model.\"\"\"\n if validation:\n in_val = \" in validation_data\"\n else:\n in_val = \"\"\n\n self._check_numpy_arrays(x, \"x\", in_val)\n if y is not None:\n self._check_numpy_arrays(y, \"y\", in_val)\n\n self._check_array_count(\n len(tree.flatten(x)), len(self.inputs), \"x\", in_val\n )\n # When predicting, y is not required.\n if not predict and y is not None:\n self._check_array_count(\n len(tree.flatten(y)), len(self.outputs), \"y\", in_val\n )\n\n def _analyze_data(self, dataset):\n input_analysers = [node.get_analyser() for node in self.inputs]\n output_analysers = [head.get_analyser() for head in self._heads]\n analysers = input_analysers + output_analysers\n np_arrays = tree.flatten(dataset)\n for array, analyser in zip(np_arrays, analysers):\n analyser.update(array)\n\n for analyser in analysers:\n analyser.finalize()\n\n for hm, analyser in zip(self.inputs + self._heads, analysers):\n hm.config_from_analyser(analyser)\n\n def _build_hyper_pipeline(self):\n self.tuner.hyper_pipeline = pipeline.HyperPipeline(\n inputs=[node.get_hyper_preprocessors() for node in self.inputs],\n outputs=[head.get_hyper_preprocessors() for head in self._heads],\n )\n self.tuner.hypermodel.hyper_pipeline = self.tuner.hyper_pipeline\n\n def _check_and_adapt(self, x, y, validation_data):\n # Convert training data.\n self._check_data_format(x, y)\n x = self._adapt(x, self.inputs)\n y = self._adapt(y, self._heads)\n\n # Convert validation data\n if validation_data:\n self._check_data_format(*validation_data, validation=True)\n x_val, y_val = validation_data\n x_val = self._adapt(x_val, self.inputs)\n y_val = self._adapt(y_val, self._heads)\n validation_data = (x_val, y_val)\n\n return (x, y), validation_data\n\n def predict(self, x, batch_size=32, verbose=1, **kwargs):\n \"\"\"Predict the output for a given testing data.\n\n # Arguments\n x: Any allowed types according to the input node. Testing data.\n batch_size: Number of samples per batch.\n If unspecified, batch_size will default to 32.\n verbose: Verbosity mode. 0 = silent, 1 = progress bar.\n Controls the verbosity of\n [keras.Model.predict](https://keras.io/api/models/model_training_apis/#predict-method)\n **kwargs: Any arguments supported by keras.Model.predict.\n\n # Returns\n A list of numpy.ndarray objects or a single numpy.ndarray.\n The predicted results.\n \"\"\"\n self._check_data_format(x, None, predict=True)\n dataset = self._adapt(x, self.inputs)\n pipeline = self.tuner.get_best_pipeline()\n model = self.tuner.get_best_model()\n dataset = pipeline.transform_x(dataset)\n y = utils.predict_with_adaptive_batch_size(\n model=model,\n batch_size=batch_size,\n x=dataset,\n verbose=verbose,\n **kwargs\n )\n return pipeline.postprocess(y)\n\n def evaluate(self, x, y=None, batch_size=32, verbose=1, **kwargs):\n \"\"\"Evaluate the best model for the given data.\n\n # Arguments\n x: Any allowed types according to the input node. Testing data.\n y: Any allowed types according to the head. Testing targets.\n Defaults to None.\n batch_size: Number of samples per batch.\n If unspecified, batch_size will default to 32.\n verbose: Verbosity mode. 0 = silent, 1 = progress bar.\n Controls the verbosity of\n [keras.Model.evaluate](https://keras.io/api/models/model_training_apis/#evaluate-method)\n **kwargs: Any arguments supported by keras.Model.evaluate.\n\n # Returns\n Scalar test loss (if the model has a single output and no metrics)\n or list of scalars (if the model has multiple outputs and/or\n metrics). The attribute model.metrics_names will give you the\n display labels for the scalar outputs.\n \"\"\"\n self._check_data_format(x, y)\n x = self._adapt(x, self.inputs)\n y = self._adapt(y, self._heads)\n pipeline = self.tuner.get_best_pipeline()\n x, y = pipeline.transform((x, y))\n model = self.tuner.get_best_model()\n return utils.evaluate_with_adaptive_batch_size(\n model=model,\n batch_size=batch_size,\n x=x,\n y=y,\n verbose=verbose,\n **kwargs\n )\n\n def export_model(self):\n \"\"\"Export the best Keras Model.\n\n # Returns\n keras.Model instance. The best model found during the search, loaded\n with trained weights.\n \"\"\"\n return self.tuner.get_best_model()\n" }, { "path": "autokeras/auto_model_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport keras_tuner\nimport numpy as np\nimport pytest\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\ndef get_tuner_class(*args, **kwargs):\n pipeline = mock.Mock()\n pipeline.transform_x.side_effect = lambda x: x\n tuner = mock.Mock()\n tuner.get_best_pipeline.return_value = pipeline\n tuner_class = mock.Mock()\n tuner_class.return_value = tuner\n return tuner_class\n\n\ndef test_auto_model_objective_is_kt_objective(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n\n assert isinstance(auto_model.objective, keras_tuner.Objective)\n\n\ndef test_auto_model_max_trial_field_as_specified(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path, max_trials=10\n )\n\n assert auto_model.max_trials == 10\n\n\ndef test_auto_model_directory_field_as_specified(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n\n assert auto_model.directory == tmp_path\n\n\ndef test_auto_model_project_name_field_as_specified(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(),\n ak.RegressionHead(),\n directory=tmp_path,\n project_name=\"auto_model\",\n )\n\n assert auto_model.project_name == \"auto_model\"\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\")\ndef test_evaluate(tuner_fn, tmp_path):\n x_train = np.random.rand(100, 32)\n y_train = np.random.rand(100, 1)\n\n input_node = ak.Input()\n output_node = input_node\n output_node = ak.DenseBlock()(output_node)\n output_node = ak.RegressionHead()(output_node)\n\n auto_model = ak.AutoModel(\n input_node, output_node, directory=tmp_path, max_trials=1\n )\n auto_model.fit(\n x_train, y_train, epochs=1, validation_data=(x_train, y_train)\n )\n pipeline = tuner_fn.return_value.return_value.get_best_pipeline.return_value\n pipeline.transform.return_value = (x_train, y_train)\n auto_model.evaluate(x_train, y_train)\n assert tuner_fn.called\n\n\ndef get_single_io_auto_model(tmp_path):\n return ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path, max_trials=2\n )\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\", side_effect=get_tuner_class)\ndef test_auto_model_predict(tuner_fn, tmp_path):\n x_train = np.random.rand(100, 32, 32, 3)\n y_train = np.random.rand(100, 1)\n\n auto_model = get_single_io_auto_model(tmp_path)\n auto_model.fit(x_train, y_train, epochs=2, validation_split=0.2)\n auto_model.predict(x_train)\n assert tuner_fn.called\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\")\ndef test_final_fit_concat(tuner_fn, tmp_path):\n tuner = tuner_fn.return_value.return_value\n\n x_train = np.random.rand(100, 32, 32, 3)\n y_train = np.random.rand(100, 1)\n\n auto_model = get_single_io_auto_model(tmp_path)\n auto_model.fit(x_train, y_train, epochs=2, validation_split=0.2)\n assert tuner.search.call_args_list[0][1][\"validation_split\"]\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\")\ndef test_final_fit_not_concat(tuner_fn, tmp_path):\n tuner = tuner_fn.return_value.return_value\n\n x_train = np.random.rand(100, 32, 32, 3)\n y_train = np.random.rand(100, 1)\n\n auto_model = get_single_io_auto_model(tmp_path)\n auto_model.fit(\n x_train, y_train, epochs=2, validation_data=(x_train, y_train)\n )\n assert not tuner.search.call_args_list[0][1][\"validation_split\"]\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\")\ndef test_overwrite(tuner_fn, tmp_path):\n tuner_class = tuner_fn.return_value\n\n x_train = np.random.rand(100, 32, 32, 3)\n y_train = np.random.rand(100, 1)\n\n auto_model = get_single_io_auto_model(tmp_path)\n auto_model.fit(\n x_train, y_train, epochs=2, validation_data=(x_train, y_train)\n )\n assert not tuner_class.call_args_list[0][1][\"overwrite\"]\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\")\ndef test_export_model(tuner_fn, tmp_path):\n tuner_class = tuner_fn.return_value\n tuner = tuner_class.return_value\n\n x_train = np.random.rand(100, 32, 32, 3)\n y_train = np.random.rand(100, 1)\n\n auto_model = get_single_io_auto_model(tmp_path)\n auto_model.fit(\n x_train, y_train, epochs=2, validation_data=(x_train, y_train)\n )\n auto_model.export_model()\n assert tuner.get_best_model.called\n\n\ndef get_multi_io_auto_model(tmp_path):\n return ak.AutoModel(\n [ak.ImageInput(), ak.ImageInput()],\n [ak.RegressionHead(), ak.RegressionHead()],\n directory=tmp_path,\n max_trials=2,\n overwrite=False,\n )\n\n\ndef dataset_error(x, y, validation_data, message, tmp_path):\n auto_model = get_multi_io_auto_model(tmp_path)\n with pytest.raises(ValueError) as info:\n auto_model.fit(x, y, epochs=2, validation_data=validation_data)\n assert message in str(info.value)\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\", side_effect=get_tuner_class)\ndef test_multi_input_predict(tuner_fn, tmp_path):\n auto_model = get_multi_io_auto_model(tmp_path)\n x1 = test_utils.generate_data()\n y1 = test_utils.generate_data(shape=(1,))\n auto_model.fit(\n x=(x1, x1), y=(y1, y1), epochs=2, validation_data=((x1, x1), (y1, y1))\n )\n auto_model.predict(x=((x1, x1),))\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\", side_effect=get_tuner_class)\ndef test_multi_input_predict2(tuner_fn, tmp_path):\n auto_model = get_multi_io_auto_model(tmp_path)\n x1 = test_utils.generate_data()\n y1 = test_utils.generate_data(shape=(1,))\n auto_model.fit(\n x=(x1, x1), y=(y1, y1), epochs=2, validation_data=((x1, x1), (y1, y1))\n )\n auto_model.predict(x=(x1, x1))\n\n\ndef test_invalid_tuner_name_error(tmp_path):\n with pytest.raises(ValueError) as info:\n ak.AutoModel(\n ak.ImageInput(),\n ak.RegressionHead(),\n directory=tmp_path,\n tuner=\"unknown\",\n )\n\n assert \"Expected the tuner argument to be one of\" in str(info.value)\n\n\ndef test_no_validation_data_nor_split_error(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n with pytest.raises(ValueError) as info:\n auto_model.fit(\n x=np.random.rand(100, 32, 32, 3),\n y=np.random.rand(100, 1),\n validation_split=0,\n )\n\n assert \"Either validation_data or a non-zero\" in str(info.value)\n\n\n@mock.patch(\"autokeras.auto_model.get_tuner_class\", side_effect=get_tuner_class)\ndef test_predict_tuple_x_and_tuple_y_predict_doesnt_crash(tuner_fn, tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n x, y = (np.random.rand(100, 32, 32, 3),), (np.random.rand(100, 1),)\n auto_model.fit(x, y)\n auto_model.predict(x)\n\n\ndef test_fit_with_non_numpy_data_raises_error(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n x = [[[1, 2, 3]] * 32] * 32 # list, not np.ndarray\n y = [1]\n match_str = \"Expected x to be a numpy array\"\n with pytest.raises(ValueError, match=match_str):\n auto_model.fit(x, y)\n\n\ndef test_fit_with_wrong_array_count_raises_error(tmp_path):\n auto_model = ak.AutoModel(\n ak.ImageInput(), ak.RegressionHead(), directory=tmp_path\n )\n x = (\n np.random.rand(10, 32, 32, 3),\n np.random.rand(10, 32, 32, 3),\n ) # 2 arrays\n y = np.random.rand(10, 1)\n match_str = \"Expected x to have 1 arrays, but got 2\"\n with pytest.raises(ValueError, match=match_str):\n auto_model.fit(x, y)\n" }, { "path": "autokeras/blocks/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\n\nfrom autokeras.blocks.basic import ConvBlock\nfrom autokeras.blocks.basic import DenseBlock\nfrom autokeras.blocks.basic import EfficientNetBlock\nfrom autokeras.blocks.basic import Embedding\nfrom autokeras.blocks.basic import ResNetBlock\nfrom autokeras.blocks.basic import RNNBlock\nfrom autokeras.blocks.basic import XceptionBlock\nfrom autokeras.blocks.heads import ClassificationHead\nfrom autokeras.blocks.heads import RegressionHead\nfrom autokeras.blocks.preprocessing import ImageAugmentation\nfrom autokeras.blocks.preprocessing import Normalization\nfrom autokeras.blocks.reduction import Flatten\nfrom autokeras.blocks.reduction import Merge\nfrom autokeras.blocks.reduction import SpatialReduction\nfrom autokeras.blocks.reduction import TemporalReduction\nfrom autokeras.blocks.wrapper import GeneralBlock\nfrom autokeras.blocks.wrapper import ImageBlock\nfrom autokeras.blocks.wrapper import StructuredDataBlock\nfrom autokeras.blocks.wrapper import TextBlock\nfrom autokeras.utils import utils\n\n\ndef serialize(obj):\n return utils.serialize_keras_object(obj)\n\n\ndef deserialize(config, custom_objects=None):\n return utils.deserialize_keras_object(\n config,\n module_objects=globals(),\n custom_objects=custom_objects,\n )\n" }, { "path": "autokeras/blocks/basic.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Optional\nfrom typing import Union\n\nimport keras\nimport tree\nfrom keras import applications\nfrom keras import layers\nfrom keras_tuner.engine import hyperparameters\n\nfrom autokeras.blocks import reduction\nfrom autokeras.engine import block as block_module\nfrom autokeras.utils import io_utils\nfrom autokeras.utils import layer_utils\nfrom autokeras.utils import utils\n\nRESNET_V1 = {\n \"resnet50\": applications.ResNet50,\n \"resnet101\": applications.ResNet101,\n \"resnet152\": applications.ResNet152,\n}\n\nRESNET_V2 = {\n \"resnet50_v2\": applications.ResNet50V2,\n \"resnet101_v2\": applications.ResNet101V2,\n \"resnet152_v2\": applications.ResNet152V2,\n}\n\nEFFICIENT_VERSIONS = {\n \"b0\": applications.EfficientNetB0,\n \"b1\": applications.EfficientNetB1,\n \"b2\": applications.EfficientNetB2,\n \"b3\": applications.EfficientNetB3,\n \"b4\": applications.EfficientNetB4,\n \"b5\": applications.EfficientNetB5,\n \"b6\": applications.EfficientNetB6,\n \"b7\": applications.EfficientNetB7,\n}\n\nPRETRAINED = \"pretrained\"\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass DenseBlock(block_module.Block):\n \"\"\"Block for Dense layers.\n\n # Arguments\n num_layers: Int or keras_tuner.engine.hyperparameters.Choice.\n The number of Dense layers in the block.\n If left unspecified, it will be tuned automatically.\n num_units: Int or keras_tuner.engine.hyperparameters.Choice.\n The number of units in each dense layer.\n If left unspecified, it will be tuned automatically.\n use_bn: Boolean. Whether to use BatchNormalization layers.\n If left unspecified, it will be tuned automatically.\n dropout: Float or keras_tuner.engine.hyperparameters.Choice.\n The dropout rate for the layers.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n num_layers: Optional[Union[int, hyperparameters.Choice]] = None,\n num_units: Optional[Union[int, hyperparameters.Choice]] = None,\n use_batchnorm: Optional[bool] = None,\n dropout: Optional[Union[float, hyperparameters.Choice]] = None,\n **kwargs,\n ):\n super().__init__(**kwargs)\n self.num_layers = utils.get_hyperparameter(\n num_layers,\n hyperparameters.Choice(\"num_layers\", [1, 2, 3], default=2),\n int,\n )\n self.num_units = utils.get_hyperparameter(\n num_units,\n hyperparameters.Choice(\n \"num_units\", [16, 32, 64, 128, 256, 512, 1024], default=32\n ),\n int,\n )\n self.use_batchnorm = use_batchnorm\n self.dropout = utils.get_hyperparameter(\n dropout,\n hyperparameters.Choice(\"dropout\", [0.0, 0.25, 0.5], default=0.0),\n float,\n )\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"num_layers\": io_utils.serialize_block_arg(self.num_layers),\n \"num_units\": io_utils.serialize_block_arg(self.num_units),\n \"use_batchnorm\": self.use_batchnorm,\n \"dropout\": io_utils.serialize_block_arg(self.dropout),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"num_layers\"] = io_utils.deserialize_block_arg(\n config[\"num_layers\"]\n )\n config[\"num_units\"] = io_utils.deserialize_block_arg(\n config[\"num_units\"]\n )\n config[\"dropout\"] = io_utils.deserialize_block_arg(config[\"dropout\"])\n return cls(**config)\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n output_node = reduction.Flatten().build(hp, output_node)\n\n use_batchnorm = self.use_batchnorm\n if use_batchnorm is None:\n use_batchnorm = hp.Boolean(\"use_batchnorm\", default=False)\n\n for i in range(utils.add_to_hp(self.num_layers, hp)):\n units = utils.add_to_hp(self.num_units, hp, \"units_{i}\".format(i=i))\n output_node = layers.Dense(units)(output_node)\n if use_batchnorm:\n output_node = layers.BatchNormalization()(output_node)\n output_node = layers.ReLU()(output_node)\n if utils.add_to_hp(self.dropout, hp) > 0:\n output_node = layers.Dropout(utils.add_to_hp(self.dropout, hp))(\n output_node\n )\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass RNNBlock(block_module.Block):\n \"\"\"An RNN Block.\n\n # Arguments\n return_sequences: Boolean. Whether to return the last output in the\n output sequence, or the full sequence. Defaults to False.\n bidirectional: Boolean or keras_tuner.engine.hyperparameters.Boolean.\n Bidirectional RNN. If left unspecified, it will be\n tuned automatically.\n num_layers: Int or keras_tuner.engine.hyperparameters.Choice.\n The number of layers in RNN. If left unspecified, it will\n be tuned automatically.\n layer_type: String or or keras_tuner.engine.hyperparameters.Choice.\n 'gru' or 'lstm'. If left unspecified, it will be tuned\n automatically.\n \"\"\"\n\n def __init__(\n self,\n return_sequences: bool = False,\n bidirectional: Optional[Union[bool, hyperparameters.Boolean]] = None,\n num_layers: Optional[Union[int, hyperparameters.Choice]] = None,\n layer_type: Optional[Union[str, hyperparameters.Choice]] = None,\n **kwargs,\n ):\n super().__init__(**kwargs)\n self.return_sequences = return_sequences\n self.bidirectional = utils.get_hyperparameter(\n bidirectional,\n hyperparameters.Boolean(\"bidirectional\", default=True),\n bool,\n )\n self.num_layers = utils.get_hyperparameter(\n num_layers,\n hyperparameters.Choice(\"num_layers\", [1, 2, 3], default=2),\n int,\n )\n self.layer_type = utils.get_hyperparameter(\n layer_type,\n hyperparameters.Choice(\n \"layer_type\", [\"gru\", \"lstm\"], default=\"lstm\"\n ),\n str,\n )\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"return_sequences\": self.return_sequences,\n \"bidirectional\": io_utils.serialize_block_arg(\n self.bidirectional\n ),\n \"num_layers\": io_utils.serialize_block_arg(self.num_layers),\n \"layer_type\": io_utils.serialize_block_arg(self.layer_type),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"bidirectional\"] = io_utils.deserialize_block_arg(\n config[\"bidirectional\"]\n )\n config[\"num_layers\"] = io_utils.deserialize_block_arg(\n config[\"num_layers\"]\n )\n config[\"layer_type\"] = io_utils.deserialize_block_arg(\n config[\"layer_type\"]\n )\n return cls(**config)\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n shape = list(input_node.shape)\n if len(shape) != 3:\n raise ValueError(\n \"Expect the input tensor of RNNBlock to have dimensions of \"\n \"[batch_size, time_steps, vec_len], \"\n \"but got {shape}\".format(shape=input_node.shape)\n )\n\n feature_size = shape[-1]\n output_node = input_node\n\n bidirectional = utils.add_to_hp(self.bidirectional, hp)\n layer_type = utils.add_to_hp(self.layer_type, hp)\n num_layers = utils.add_to_hp(self.num_layers, hp)\n rnn_layers = {\"gru\": layers.GRU, \"lstm\": layers.LSTM}\n in_layer = rnn_layers[layer_type]\n for i in range(num_layers):\n return_sequences = True\n if i == num_layers - 1:\n return_sequences = self.return_sequences\n if bidirectional:\n output_node = layers.Bidirectional( # pragma: no cover\n in_layer(feature_size, return_sequences=return_sequences)\n )(output_node)\n else:\n output_node = in_layer(\n feature_size, return_sequences=return_sequences\n )(output_node)\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass ConvBlock(block_module.Block):\n \"\"\"Block for vanilla ConvNets.\n\n # Arguments\n kernel_size: Int or keras_tuner.engine.hyperparameters.Choice.\n The size of the kernel.\n If left unspecified, it will be tuned automatically.\n num_blocks: Int or keras_tuner.engine.hyperparameters.Choice.\n The number of conv blocks, each of which may contain\n convolutional, max pooling, dropout, and activation. If left\n unspecified, it will be tuned automatically.\n num_layers: Int or hyperparameters.Choice.\n The number of convolutional layers in each block. If left\n unspecified, it will be tuned automatically.\n filters: Int or keras_tuner.engine.hyperparameters.Choice. The number of\n filters in the convolutional layers. If left unspecified, it will\n be tuned automatically.\n max_pooling: Boolean. Whether to use max pooling layer in each block. If\n left unspecified, it will be tuned automatically.\n separable: Boolean. Whether to use separable conv layers.\n If left unspecified, it will be tuned automatically.\n dropout: Float or kerastuner.engine.hyperparameters.\n Choice range Between 0 and 1.\n The dropout rate after convolutional layers.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n kernel_size: Optional[Union[int, hyperparameters.Choice]] = None,\n num_blocks: Optional[Union[int, hyperparameters.Choice]] = None,\n num_layers: Optional[Union[int, hyperparameters.Choice]] = None,\n filters: Optional[Union[int, hyperparameters.Choice]] = None,\n max_pooling: Optional[bool] = None,\n separable: Optional[bool] = None,\n dropout: Optional[Union[float, hyperparameters.Choice]] = None,\n **kwargs,\n ):\n super().__init__(**kwargs)\n self.kernel_size = utils.get_hyperparameter(\n kernel_size,\n hyperparameters.Choice(\"kernel_size\", [3, 5, 7], default=3),\n int,\n )\n self.num_blocks = utils.get_hyperparameter(\n num_blocks,\n hyperparameters.Choice(\"num_blocks\", [1, 2, 3], default=2),\n int,\n )\n self.num_layers = utils.get_hyperparameter(\n num_layers,\n hyperparameters.Choice(\"num_layers\", [1, 2], default=2),\n int,\n )\n self.filters = utils.get_hyperparameter(\n filters,\n hyperparameters.Choice(\n \"filters\", [16, 32, 64, 128, 256, 512], default=32\n ),\n int,\n )\n self.max_pooling = max_pooling\n self.separable = separable\n self.dropout = utils.get_hyperparameter(\n dropout,\n hyperparameters.Choice(\"dropout\", [0.0, 0.25, 0.5], default=0.0),\n float,\n )\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"kernel_size\": io_utils.serialize_block_arg(self.kernel_size),\n \"num_blocks\": io_utils.serialize_block_arg(self.num_blocks),\n \"num_layers\": io_utils.serialize_block_arg(self.num_layers),\n \"filters\": io_utils.serialize_block_arg(self.filters),\n \"max_pooling\": self.max_pooling,\n \"separable\": self.separable,\n \"dropout\": io_utils.serialize_block_arg(self.dropout),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"kernel_size\"] = io_utils.deserialize_block_arg(\n config[\"kernel_size\"]\n )\n config[\"num_blocks\"] = io_utils.deserialize_block_arg(\n config[\"num_blocks\"]\n )\n config[\"num_layers\"] = io_utils.deserialize_block_arg(\n config[\"num_layers\"]\n )\n config[\"filters\"] = io_utils.deserialize_block_arg(config[\"filters\"])\n config[\"dropout\"] = io_utils.deserialize_block_arg(config[\"dropout\"])\n return cls(**config)\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n\n kernel_size = utils.add_to_hp(self.kernel_size, hp)\n\n separable = self.separable\n if separable is None:\n separable = hp.Boolean(\"separable\", default=False)\n\n if separable:\n conv = layer_utils.get_sep_conv(\n input_node.shape\n ) # pragma: no cover\n else:\n conv = layer_utils.get_conv(input_node.shape)\n\n max_pooling = self.max_pooling\n if max_pooling is None:\n max_pooling = hp.Boolean(\"max_pooling\", default=True)\n pool = layer_utils.get_max_pooling(input_node.shape)\n\n for i in range(utils.add_to_hp(self.num_blocks, hp)):\n for j in range(utils.add_to_hp(self.num_layers, hp)):\n output_node = conv(\n utils.add_to_hp(\n self.filters, hp, \"filters_{i}_{j}\".format(i=i, j=j)\n ),\n kernel_size,\n padding=self._get_padding(kernel_size, output_node),\n activation=\"relu\",\n )(output_node)\n if max_pooling:\n output_node = pool(\n kernel_size - 1,\n padding=self._get_padding(kernel_size - 1, output_node),\n )(output_node)\n if utils.add_to_hp(self.dropout, hp) > 0:\n output_node = layers.Dropout(utils.add_to_hp(self.dropout, hp))(\n output_node\n )\n return output_node\n\n @staticmethod\n def _get_padding(kernel_size, output_node):\n if all(kernel_size * 2 <= length for length in output_node.shape[1:-1]):\n return \"valid\"\n return \"same\"\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass KerasApplicationBlock(block_module.Block):\n \"\"\"Blocks extending Keras applications.\"\"\"\n\n def __init__(self, pretrained, models, min_size, **kwargs):\n super().__init__(**kwargs)\n self.pretrained = pretrained\n self.models = models\n self.min_size = min_size\n\n def get_config(self):\n config = super().get_config()\n config.update({\"pretrained\": self.pretrained})\n return config\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n\n pretrained = self.pretrained\n if input_node.shape[3] not in [1, 3]:\n if self.pretrained:\n raise ValueError(\n \"When pretrained is set to True, expect input to \"\n \"have 1 or 3 channels, bug got \"\n \"{channels}.\".format(channels=input_node.shape[3])\n )\n pretrained = False\n if pretrained is None:\n pretrained = hp.Boolean(PRETRAINED, default=False)\n if pretrained:\n with hp.conditional_scope(PRETRAINED, [True]):\n trainable = hp.Boolean(\"trainable\", default=False)\n elif pretrained:\n trainable = hp.Boolean(\"trainable\", default=False)\n\n if len(self.models) > 1:\n version = hp.Choice(\"version\", list(self.models.keys()))\n else:\n version = list(self.models.keys())[0]\n\n min_size = self.min_size\n if hp.Boolean(\"imagenet_size\", default=False):\n min_size = 224\n if input_node.shape[1] < min_size or input_node.shape[2] < min_size:\n input_node = layers.Resizing(\n max(min_size, input_node.shape[1]),\n max(min_size, input_node.shape[2]),\n )(input_node)\n if input_node.shape[3] == 1:\n input_node = layers.Concatenate()([input_node] * 3)\n if input_node.shape[3] != 3:\n input_node = layers.Conv2D(\n filters=3, kernel_size=1, padding=\"same\"\n )(input_node)\n\n if pretrained:\n model = self.models[version](weights=\"imagenet\", include_top=False)\n model.trainable = trainable\n else:\n model = self.models[version](\n weights=None,\n include_top=False,\n input_shape=input_node.shape[1:],\n )\n\n return model(input_node)\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass ResNetBlock(KerasApplicationBlock):\n \"\"\"Block for ResNet.\n\n # Arguments\n version: String. 'v1', 'v2'. The type of ResNet to use.\n If left unspecified, it will be tuned automatically.\n pretrained: Boolean. Whether to use ImageNet pretrained weights.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n version: Optional[str] = None,\n pretrained: Optional[bool] = None,\n **kwargs,\n ):\n if version is None:\n models = {**RESNET_V1, **RESNET_V2}\n elif version == \"v1\":\n models = RESNET_V1\n elif version == \"v2\":\n models = RESNET_V2\n else:\n raise ValueError(\n 'Expect version to be \"v1\", or \"v2\", but got '\n \"{version}.\".format(version=version)\n )\n super().__init__(\n pretrained=pretrained, models=models, min_size=32, **kwargs\n )\n self.version = version\n\n def get_config(self):\n config = super().get_config()\n config.update({\"version\": self.version})\n return config\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass XceptionBlock(KerasApplicationBlock):\n \"\"\"Block for XceptionNet.\n\n An Xception structure, used for specifying your model with specific\n datasets.\n\n The original Xception architecture is from\n [https://arxiv.org/abs/1610.02357](https://arxiv.org/abs/1610.02357).\n The data first goes through the entry flow, then through the middle flow\n which is repeated eight times, and finally through the exit flow.\n\n This XceptionBlock returns a similar architecture as Xception except without\n the last (optional) fully connected layer(s) and logistic regression.\n The size of this architecture could be decided by `HyperParameters`, to get\n an architecture with a half, an identical, or a double size of the original\n one.\n\n # Arguments\n pretrained: Boolean. Whether to use ImageNet pretrained weights.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(self, pretrained: Optional[bool] = None, **kwargs):\n super().__init__(\n pretrained=pretrained,\n models={\"xception\": applications.Xception},\n min_size=71,\n **kwargs,\n )\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass EfficientNetBlock(KerasApplicationBlock):\n \"\"\"Block for EfficientNet.\n\n # Arguments\n version: String. The value should be one of 'b0', 'b1', ..., 'b7'. The\n type of EfficientNet to use. If left unspecified, it will be tuned\n automatically.\n pretrained: Boolean. Whether to use ImageNet pretrained weights.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n version: Optional[str] = None,\n pretrained: Optional[bool] = None,\n **kwargs,\n ):\n if version is None:\n models = EFFICIENT_VERSIONS\n elif version in EFFICIENT_VERSIONS.keys():\n models = {version: EFFICIENT_VERSIONS[version]}\n else:\n raise ValueError(\n \"Expect version to be in {expect}, but got \"\n \"{version}.\".format(\n expect=list(EFFICIENT_VERSIONS.keys()), version=version\n )\n )\n super().__init__(\n pretrained=pretrained,\n models=models,\n min_size=32,\n **kwargs,\n )\n self.version = version\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Embedding(block_module.Block):\n \"\"\"Word embedding block for sequences.\n\n The input should be tokenized sequences with the same length, where each\n element of a sequence should be the index of the word.\n\n # Arguments\n max_features: Int. Size of the vocabulary. Must be set if not using\n TextToIntSequence before this block. Defaults to 20001.\n embedding_dim: Int or keras_tuner.engine.hyperparameters.Choice.\n Output dimension of the Attention block.\n If left unspecified, it will be tuned automatically.\n dropout: Float or keras_tuner.engine.hyperparameters.Choice.\n The dropout rate for the layers.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n max_features: int = 20001,\n embedding_dim: Optional[Union[int, hyperparameters.Choice]] = None,\n dropout: Optional[Union[float, hyperparameters.Choice]] = None,\n **kwargs,\n ):\n super().__init__(**kwargs)\n self.max_features = max_features\n self.embedding_dim = utils.get_hyperparameter(\n embedding_dim,\n hyperparameters.Choice(\n \"embedding_dim\", [32, 64, 128, 256, 512], default=128\n ),\n int,\n )\n self.dropout = utils.get_hyperparameter(\n dropout,\n hyperparameters.Choice(\"dropout\", [0.0, 0.25, 0.5], default=0.25),\n float,\n )\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"max_features\": self.max_features,\n \"embedding_dim\": io_utils.serialize_block_arg(\n self.embedding_dim\n ),\n \"dropout\": io_utils.serialize_block_arg(self.dropout),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"dropout\"] = io_utils.deserialize_block_arg(config[\"dropout\"])\n config[\"embedding_dim\"] = io_utils.deserialize_block_arg(\n config[\"embedding_dim\"]\n )\n return cls(**config)\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n embedding_dim = utils.add_to_hp(self.embedding_dim, hp)\n layer = layers.Embedding(\n input_dim=self.max_features, output_dim=embedding_dim\n )\n output_node = layer(input_node)\n dropout = utils.add_to_hp(self.dropout, hp)\n if dropout > 0:\n output_node = layers.Dropout(dropout)(output_node)\n return output_node\n" }, { "path": "autokeras/blocks/basic_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport pytest\nimport tree\n\nfrom autokeras import blocks\nfrom autokeras import test_utils\n\n\ndef test_resnet_build_return_tensor():\n block = blocks.ResNetBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_resnet_v1_return_tensor():\n block = blocks.ResNetBlock(version=\"v1\")\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_efficientnet_b0_return_tensor():\n block = blocks.EfficientNetBlock(version=\"b0\", pretrained=False)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_resnet_pretrained_build_return_tensor():\n block = blocks.ResNetBlock(pretrained=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_resnet_pretrained_with_one_channel_input():\n block = blocks.ResNetBlock(pretrained=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(28, 28, 1), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_resnet_pretrained_error_with_two_channels():\n block = blocks.ResNetBlock(pretrained=True)\n\n with pytest.raises(ValueError) as info:\n block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(224, 224, 2), dtype=\"float32\"),\n )\n\n assert \"When pretrained is set to True\" in str(info.value)\n\n\ndef test_resnet_deserialize_to_resnet():\n serialized_block = blocks.serialize(blocks.ResNetBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.ResNetBlock)\n\n\ndef test_resnet_get_config_has_all_attributes():\n block = blocks.ResNetBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.ResNetBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_resnet_wrong_version_error():\n with pytest.raises(ValueError) as info:\n blocks.ResNetBlock(version=\"abc\")\n\n assert \"Expect version to be\" in str(info.value)\n\n\ndef test_efficientnet_wrong_version_error():\n with pytest.raises(ValueError) as info:\n blocks.EfficientNetBlock(version=\"abc\")\n\n assert \"Expect version to be\" in str(info.value)\n\n\ndef test_xception_build_return_tensor():\n block = blocks.XceptionBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 2), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_xception_pretrained_build_return_tensor():\n block = blocks.XceptionBlock(pretrained=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_xception_pretrained_with_one_channel_input():\n block = blocks.XceptionBlock(pretrained=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(224, 224, 1), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_xception_pretrained_error_with_two_channels():\n block = blocks.XceptionBlock(pretrained=True)\n\n with pytest.raises(ValueError) as info:\n block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(224, 224, 2), dtype=\"float32\"),\n )\n\n assert \"When pretrained is set to True\" in str(info.value)\n\n\ndef test_xception_deserialize_to_xception():\n serialized_block = blocks.serialize(blocks.XceptionBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.XceptionBlock)\n\n\ndef test_xception_get_config_has_all_attributes():\n block = blocks.XceptionBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.XceptionBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_conv_build_return_tensor():\n block = blocks.ConvBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_conv_with_small_image_size_return_tensor():\n block = blocks.ConvBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(10, 10, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_conv_build_with_dropout_return_tensor():\n block = blocks.ConvBlock(dropout=0.5)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_conv_deserialize_to_conv():\n serialized_block = blocks.serialize(blocks.ConvBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.ConvBlock)\n\n\ndef test_conv_get_config_has_all_attributes():\n block = blocks.ConvBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.ConvBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_rnn_build_return_tensor():\n block = blocks.RNNBlock(bidirectional=False)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 10), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_rnn_input_shape_one_dim_error():\n block = blocks.RNNBlock()\n\n with pytest.raises(ValueError) as info:\n block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert \"Expect the input tensor of RNNBlock\" in str(info.value)\n\n\ndef test_rnn_deserialize_to_rnn():\n serialized_block = blocks.serialize(blocks.RNNBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.RNNBlock)\n\n\ndef test_rnn_get_config_has_all_attributes():\n block = blocks.RNNBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.RNNBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_dense_build_return_tensor():\n block = blocks.DenseBlock(\n num_units=keras_tuner.engine.hyperparameters.Choice(\n \"num_units\", [10, 20]\n )\n )\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_dense_build_with_dropout_return_tensor():\n block = blocks.DenseBlock(dropout=0.5)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_dense_build_with_bn_return_tensor():\n block = blocks.DenseBlock(use_batchnorm=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_dense_deserialize_to_dense():\n serialized_block = blocks.serialize(blocks.DenseBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.DenseBlock)\n\n\ndef test_dense_get_config_has_all_attributes():\n block = blocks.DenseBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.DenseBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_embed_build_return_tensor():\n block = blocks.Embedding()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_embed_deserialize_to_embed():\n serialized_block = blocks.serialize(blocks.Embedding())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.Embedding)\n\n\ndef test_embed_get_config_has_all_attributes():\n block = blocks.Embedding()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.Embedding.__init__).issubset(\n config.keys()\n )\n" }, { "path": "autokeras/blocks/heads.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Optional\n\nimport keras\nimport tree\nfrom keras import activations\nfrom keras import layers\nfrom keras import losses\n\nfrom autokeras import adapters\nfrom autokeras import analysers\nfrom autokeras import hyper_preprocessors as hpps_module\nfrom autokeras import preprocessors\nfrom autokeras.blocks import reduction\nfrom autokeras.engine import head as head_module\nfrom autokeras.utils import types\nfrom autokeras.utils import utils\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass ClassificationHead(head_module.Head):\n \"\"\"Classification Dense layers.\n\n Use sigmoid and binary crossentropy for binary classification and\n multi-label classification. Use softmax and categorical crossentropy for\n multi-class (more than 2) classification. Use Accuracy as metrics by\n default.\n\n The targets passing to the head would have to be np.ndarray. It can be raw\n labels, one-hot encoded if more than two classes, or binary encoded for\n binary classification.\n\n The raw labels will be encoded to one column if two classes were found,\n or one-hot encoded if more than two classes were found.\n\n # Arguments\n num_classes: Int. Defaults to None. If None, it will be inferred from\n the data.\n multi_label: Boolean. Defaults to False.\n loss: A Keras loss function. Defaults to use `binary_crossentropy` or\n `categorical_crossentropy` based on the number of classes.\n metrics: A list of Keras metrics. Defaults to use 'accuracy'.\n dropout: Float. The dropout rate for the layers.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n num_classes: Optional[int] = None,\n multi_label: bool = False,\n loss: Optional[types.LossType] = None,\n metrics: Optional[types.MetricsType] = None,\n dropout: Optional[float] = None,\n **kwargs\n ):\n self.num_classes = num_classes\n self.multi_label = multi_label\n self.dropout = dropout\n if metrics is None:\n metrics = [\"accuracy\"]\n if loss is None:\n loss = self.infer_loss()\n super().__init__(loss=loss, metrics=metrics, **kwargs)\n # Infered from analyser.\n self._encoded = None\n self._encoded_for_sigmoid = None\n self._encoded_for_softmax = None\n self._add_one_dimension = False\n self._labels = None\n\n def infer_loss(self):\n if not self.num_classes:\n return None\n if self.num_classes == 2 or self.multi_label:\n return losses.BinaryCrossentropy()\n return losses.CategoricalCrossentropy()\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"num_classes\": self.num_classes,\n \"multi_label\": self.multi_label,\n \"dropout\": self.dropout,\n }\n )\n return config\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n\n # Reduce the tensor to a vector.\n if len(output_node.shape) > 2:\n output_node = reduction.SpatialReduction().build(hp, output_node)\n\n if self.dropout is not None:\n dropout = self.dropout\n else:\n dropout = hp.Choice(\"dropout\", [0.0, 0.25, 0.5], default=0)\n\n if dropout > 0:\n output_node = layers.Dropout(dropout)(output_node)\n output_node = layers.Dense(self.shape[-1])(output_node)\n if isinstance(self.loss, keras.losses.BinaryCrossentropy):\n output_node = layers.Activation(\n activations.sigmoid, name=self.name\n )(output_node)\n else:\n output_node = layers.Softmax(name=self.name)(output_node)\n return output_node\n\n def get_adapter(self):\n return adapters.ClassificationAdapter(name=self.name)\n\n def get_analyser(self):\n return analysers.ClassificationAnalyser(\n name=self.name, multi_label=self.multi_label\n )\n\n def config_from_analyser(self, analyser):\n super().config_from_analyser(analyser)\n self.num_classes = analyser.num_classes\n self.loss = self.infer_loss()\n self._encoded = analyser.encoded\n self._encoded_for_sigmoid = analyser.encoded_for_sigmoid\n self._encoded_for_softmax = analyser.encoded_for_softmax\n self._add_one_dimension = len(analyser.shape) == 1\n self._labels = analyser.labels\n\n def get_hyper_preprocessors(self):\n hyper_preprocessors = []\n\n if self._add_one_dimension:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.AddOneDimension()\n )\n )\n\n if self.dtype in [\"uint8\", \"uint16\", \"uint32\", \"uint64\"]:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.CastToInt32()\n )\n )\n\n if not self._encoded and self.dtype != \"string\":\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.CastToString()\n )\n )\n\n if self._encoded_for_sigmoid:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.SigmoidPostprocessor()\n )\n )\n elif self._encoded_for_softmax:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.SoftmaxPostprocessor()\n )\n )\n elif self.num_classes == 2:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.LabelEncoder(self._labels)\n )\n )\n else:\n hyper_preprocessors.append(\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.OneHotEncoder(self._labels)\n )\n )\n return hyper_preprocessors\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass RegressionHead(head_module.Head):\n \"\"\"Regression Dense layers.\n\n The targets passing to the head would have to be np.ndarray. It can be\n single-column or multi-column. The values should all be numerical.\n\n # Arguments\n output_dim: Int. The number of output dimensions. Defaults to None.\n If None, it will be inferred from the data.\n multi_label: Boolean. Defaults to False.\n loss: A Keras loss function. Defaults to use `mean_squared_error`.\n metrics: A list of Keras metrics. Defaults to use `mean_squared_error`.\n dropout: Float. The dropout rate for the layers.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n output_dim: Optional[int] = None,\n loss: types.LossType = \"mean_squared_error\",\n metrics: Optional[types.MetricsType] = None,\n dropout: Optional[float] = None,\n **kwargs\n ):\n if metrics is None:\n metrics = [\"mean_squared_error\"]\n super().__init__(loss=loss, metrics=metrics, **kwargs)\n self.output_dim = output_dim\n self.dropout = dropout\n\n def get_config(self):\n config = super().get_config()\n config.update({\"output_dim\": self.output_dim, \"dropout\": self.dropout})\n return config\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n\n if self.dropout is not None:\n dropout = self.dropout\n else:\n dropout = hp.Choice(\"dropout\", [0.0, 0.25, 0.5], default=0)\n\n if dropout > 0:\n output_node = layers.Dropout(dropout)(output_node)\n output_node = reduction.Flatten().build(hp, output_node)\n output_node = layers.Dense(self.shape[-1], name=self.name)(output_node)\n return output_node\n\n def config_from_analyser(self, analyser):\n super().config_from_analyser(analyser)\n self._add_one_dimension = len(analyser.shape) == 1\n\n def get_adapter(self):\n return adapters.RegressionAdapter(name=self.name)\n\n def get_analyser(self):\n return analysers.RegressionAnalyser(\n name=self.name, output_dim=self.output_dim\n )\n\n def get_hyper_preprocessors(self):\n hyper_preprocessors = []\n if self._add_one_dimension:\n hyper_preprocessors.append( # pragma: no cover\n hpps_module.DefaultHyperPreprocessor(\n preprocessors.AddOneDimension()\n )\n )\n return hyper_preprocessors\n" }, { "path": "autokeras/blocks/heads_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport numpy as np\nimport tree\n\nfrom autokeras import hyper_preprocessors\nfrom autokeras import nodes as input_module\nfrom autokeras import preprocessors\nfrom autokeras import test_utils\nfrom autokeras.blocks import heads as head_module\n\n\ndef test_two_classes_infer_binary_crossentropy():\n dataset = np.array([\"a\", \"a\", \"a\", \"b\"])\n head = head_module.ClassificationHead(name=\"a\", shape=(1,))\n adapter = head.get_adapter()\n dataset = adapter.adapt(dataset)\n analyser = head.get_analyser()\n analyser.update(dataset)\n analyser.finalize()\n head.config_from_analyser(analyser)\n head.build(\n keras_tuner.HyperParameters(),\n input_module.Input(shape=(32,)).build_node(\n keras_tuner.HyperParameters()\n ),\n )\n assert head.loss.name == \"binary_crossentropy\"\n\n\ndef test_three_classes_infer_categorical_crossentropy():\n dataset = np.array([\"a\", \"a\", \"c\", \"b\"])\n head = head_module.ClassificationHead(name=\"a\", shape=(1,))\n adapter = head.get_adapter()\n dataset = adapter.adapt(dataset)\n analyser = head.get_analyser()\n analyser.update(dataset)\n analyser.finalize()\n head.config_from_analyser(analyser)\n head.build(\n keras_tuner.HyperParameters(),\n input_module.Input(shape=(32,)).build_node(\n keras_tuner.HyperParameters()\n ),\n )\n assert head.loss.name == \"categorical_crossentropy\"\n\n\ndef test_multi_label_loss():\n head = head_module.ClassificationHead(\n name=\"a\", multi_label=True, num_classes=8, shape=(8,)\n )\n input_node = keras.Input(shape=(5,))\n output_node = head.build(keras_tuner.HyperParameters(), input_node)\n model = keras.Model(input_node, output_node)\n assert model.layers[-1].activation.__name__ == \"sigmoid\"\n assert head.loss.name == \"binary_crossentropy\"\n\n\ndef test_clf_head_get_sigmoid_postprocessor():\n head = head_module.ClassificationHead(name=\"a\", multi_label=True)\n head._encoded = True\n head._encoded_for_sigmoid = True\n assert isinstance(\n head.get_hyper_preprocessors()[0].preprocessor,\n preprocessors.SigmoidPostprocessor,\n )\n\n\ndef test_clf_head_with_2_clases_get_label_encoder():\n head = head_module.ClassificationHead(name=\"a\", num_classes=2)\n head._encoded = False\n head._labels = [\"a\", \"b\"]\n assert isinstance(\n head.get_hyper_preprocessors()[-1].preprocessor,\n preprocessors.LabelEncoder,\n )\n\n\ndef test_clf_head_build_with_zero_dropout_return_tensor():\n block = head_module.ClassificationHead(dropout=0, shape=(8,))\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(5,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_clf_head_hpps_with_uint8_contain_cast_to_int32():\n dataset = test_utils.generate_one_hot_labels(100, 10)\n dataset = dataset.astype(\"uint8\")\n head = head_module.ClassificationHead(shape=(8,))\n analyser = head.get_analyser()\n for data in dataset:\n analyser.update(data)\n analyser.finalize()\n head.config_from_analyser(analyser)\n\n assert any(\n [\n isinstance(hpp, hyper_preprocessors.DefaultHyperPreprocessor)\n and isinstance(hpp.preprocessor, preprocessors.CastToInt32)\n for hpp in head.get_hyper_preprocessors()\n ]\n )\n\n\ndef test_reg_head_build_with_zero_dropout_return_tensor():\n block = head_module.RegressionHead(dropout=0, shape=(8,))\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(5,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n" }, { "path": "autokeras/blocks/preprocessing.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Blocks for data preprocessing.\n\nThey are built into keras preprocessing layers and will be part of the Keras\nmodel.\n\n\"\"\"\nfrom typing import Optional\nfrom typing import Tuple\nfrom typing import Union\n\nimport keras\nimport tree\nfrom keras import layers\nfrom keras_tuner.engine import hyperparameters\n\nfrom autokeras.engine import block as block_module\nfrom autokeras.utils import io_utils\nfrom autokeras.utils import utils\n\n\nclass Normalization(block_module.Block):\n \"\"\"Perform feature-wise normalization on data.\n\n Refer to Normalization layer in keras preprocessing layers for more\n information.\n\n # Arguments\n axis: Integer or tuple of integers, the axis or axes that should be\n normalized (typically the features axis). We will normalize each\n element in the specified axis. The default is '-1' (the innermost\n axis); 0 (the batch axis) is not allowed.\n \"\"\"\n\n def __init__(self, axis: int = -1, **kwargs):\n super().__init__(**kwargs)\n self.axis = axis\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n return layers.Normalization(axis=self.axis)(input_node)\n\n def get_config(self):\n config = super().get_config()\n config.update({\"axis\": self.axis})\n return config\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass ImageAugmentation(block_module.Block):\n \"\"\"Collection of various image augmentation methods.\n\n # Arguments\n translation_factor: A positive float represented as fraction value, or a\n tuple of 2 representing fraction for translation vertically and\n horizontally, or a kerastuner.engine.hyperparameters.Choice range\n of positive floats. For instance, `translation_factor=0.2` result\n in a random translation factor within 20% of the width and height.\n If left unspecified, it will be tuned automatically.\n vertical_flip: Boolean. Whether to flip the image vertically.\n If left unspecified, it will be tuned automatically.\n horizontal_flip: Boolean. Whether to flip the image horizontally.\n If left unspecified, it will be tuned automatically.\n rotation_factor: Float or kerastuner.engine.hyperparameters.Choice range\n between [0, 1]. A positive float represented as fraction of 2pi\n upper bound for rotating clockwise and counter-clockwise. When\n represented as a single float, lower = upper.\n If left unspecified, it will be tuned automatically.\n zoom_factor: A positive float represented as fraction value, or a tuple\n of 2 representing fraction for zooming vertically and horizontally,\n or a kerastuner.engine.hyperparameters.Choice range of positive\n floats. For instance, `zoom_factor=0.2` result in a random zoom\n factor from 80% to 120%. If left unspecified, it will be tuned\n automatically.\n contrast_factor: A positive float represented as fraction of value, or a\n tuple of size 2 representing lower and upper bound, or a\n kerastuner.engine.hyperparameters.Choice range of floats to find the\n optimal value. When represented as a single float, lower = upper.\n The contrast factor will be randomly picked\n between [1.0 - lower, 1.0 + upper]. If left unspecified, it will be\n tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n translation_factor: Optional[\n Union[float, Tuple[float, float], hyperparameters.Choice]\n ] = None,\n vertical_flip: Optional[bool] = None,\n horizontal_flip: Optional[bool] = None,\n rotation_factor: Optional[Union[float, hyperparameters.Choice]] = None,\n zoom_factor: Optional[\n Union[float, Tuple[float, float], hyperparameters.Choice]\n ] = None,\n contrast_factor: Optional[\n Union[float, Tuple[float, float], hyperparameters.Choice]\n ] = None,\n **kwargs\n ):\n super().__init__(**kwargs)\n self.translation_factor = utils.get_hyperparameter(\n translation_factor,\n hyperparameters.Choice(\"translation_factor\", [0.0, 0.1]),\n Union[float, Tuple[float, float]],\n )\n self.horizontal_flip = horizontal_flip\n self.vertical_flip = vertical_flip\n self.rotation_factor = utils.get_hyperparameter(\n rotation_factor,\n hyperparameters.Choice(\"rotation_factor\", [0.0, 0.1]),\n float,\n )\n self.zoom_factor = utils.get_hyperparameter(\n zoom_factor,\n hyperparameters.Choice(\"zoom_factor\", [0.0, 0.1]),\n Union[float, Tuple[float, float]],\n )\n self.contrast_factor = utils.get_hyperparameter(\n contrast_factor,\n hyperparameters.Choice(\"contrast_factor\", [0.0, 0.1]),\n Union[float, Tuple[float, float]],\n )\n\n @staticmethod\n def _get_fraction_value(value):\n if isinstance(value, tuple):\n return value\n return value, value\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n output_node = input_node\n\n # Translate\n translation_factor = utils.add_to_hp(self.translation_factor, hp)\n if translation_factor not in [0, (0, 0)]:\n height_factor, width_factor = self._get_fraction_value(\n translation_factor\n )\n output_node = layers.RandomTranslation(height_factor, width_factor)(\n output_node\n )\n\n # Flip\n horizontal_flip = self.horizontal_flip\n if horizontal_flip is None:\n horizontal_flip = hp.Boolean(\"horizontal_flip\", default=True)\n vertical_flip = self.vertical_flip\n if self.vertical_flip is None:\n vertical_flip = hp.Boolean(\"vertical_flip\", default=True)\n if not horizontal_flip and not vertical_flip:\n flip_mode = \"\"\n elif horizontal_flip and vertical_flip:\n flip_mode = \"horizontal_and_vertical\"\n elif horizontal_flip and not vertical_flip:\n flip_mode = \"horizontal\"\n elif not horizontal_flip and vertical_flip:\n flip_mode = \"vertical\"\n if flip_mode != \"\":\n output_node = layers.RandomFlip(mode=flip_mode)(output_node)\n\n # Rotate\n rotation_factor = utils.add_to_hp(self.rotation_factor, hp)\n if rotation_factor != 0:\n output_node = layers.RandomRotation(rotation_factor)(output_node)\n\n # Zoom\n zoom_factor = utils.add_to_hp(self.zoom_factor, hp)\n if zoom_factor not in [0, (0, 0)]:\n height_factor, width_factor = self._get_fraction_value(zoom_factor)\n # TODO: Add back RandomZoom when it is ready.\n # output_node = layers.RandomZoom(\n # height_factor, width_factor)(output_node)\n\n # Contrast\n contrast_factor = utils.add_to_hp(self.contrast_factor, hp)\n if contrast_factor not in [0, (0, 0)]:\n output_node = layers.RandomContrast(contrast_factor)(output_node)\n\n return output_node\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"translation_factor\": io_utils.serialize_block_arg(\n self.translation_factor\n ),\n \"horizontal_flip\": self.horizontal_flip,\n \"vertical_flip\": self.vertical_flip,\n \"rotation_factor\": io_utils.serialize_block_arg(\n self.rotation_factor\n ),\n \"zoom_factor\": io_utils.serialize_block_arg(self.zoom_factor),\n \"contrast_factor\": io_utils.serialize_block_arg(\n self.contrast_factor\n ),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"translation_factor\"] = io_utils.deserialize_block_arg(\n config[\"translation_factor\"]\n )\n config[\"rotation_factor\"] = io_utils.deserialize_block_arg(\n config[\"rotation_factor\"]\n )\n config[\"zoom_factor\"] = io_utils.deserialize_block_arg(\n config[\"zoom_factor\"]\n )\n config[\"contrast_factor\"] = io_utils.deserialize_block_arg(\n config[\"contrast_factor\"]\n )\n return cls(**config)\n" }, { "path": "autokeras/blocks/preprocessing_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport tree\nfrom keras_tuner.engine import hyperparameters\n\nfrom autokeras import blocks\nfrom autokeras import test_utils\n\n\ndef test_augment_build_return_tensor():\n block = blocks.ImageAugmentation(rotation_factor=0.2)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_build_with_translation_factor_range_return_tensor():\n block = blocks.ImageAugmentation(translation_factor=(0, 0.1))\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_build_with_no_flip_return_tensor():\n block = blocks.ImageAugmentation(vertical_flip=False, horizontal_flip=False)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_build_with_vflip_only_return_tensor():\n block = blocks.ImageAugmentation(vertical_flip=True, horizontal_flip=False)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_build_with_zoom_factor_return_tensor():\n block = blocks.ImageAugmentation(zoom_factor=0.1)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_build_with_contrast_factor_return_tensor():\n block = blocks.ImageAugmentation(contrast_factor=0.1)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_augment_deserialize_to_augment():\n serialized_block = blocks.serialize(\n blocks.ImageAugmentation(\n zoom_factor=0.1,\n contrast_factor=hyperparameters.Float(\"contrast_factor\", 0.1, 0.5),\n )\n )\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.ImageAugmentation)\n assert block.zoom_factor == 0.1\n assert isinstance(block.contrast_factor, hyperparameters.Float)\n\n\ndef test_augment_get_config_has_all_attributes():\n block = blocks.ImageAugmentation()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.ImageAugmentation.__init__).issubset(\n config.keys()\n )\n" }, { "path": "autokeras/blocks/reduction.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Optional\n\nimport keras\nimport tree\nfrom keras import layers\nfrom keras import ops\n\nfrom autokeras.engine import block as block_module\nfrom autokeras.utils import layer_utils\nfrom autokeras.utils import utils\n\nREDUCTION_TYPE = \"reduction_type\"\nFLATTEN = \"flatten\"\nGLOBAL_MAX = \"global_max\"\nGLOBAL_AVG = \"global_avg\"\n\n\ndef shape_compatible(shape1, shape2):\n if len(shape1) != len(shape2):\n return False\n # TODO: If they can be the same after passing through any layer,\n # they are compatible. e.g. (32, 32, 3), (16, 16, 2) are compatible\n return shape1[:-1] == shape2[:-1]\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Merge(block_module.Block):\n \"\"\"Merge block to merge multiple nodes into one.\n\n # Arguments\n merge_type: String. 'add' or 'concatenate'. If left unspecified, it will\n be tuned automatically.\n \"\"\"\n\n def __init__(self, merge_type: Optional[str] = None, **kwargs):\n super().__init__(**kwargs)\n self.merge_type = merge_type\n\n def get_config(self):\n config = super().get_config()\n config.update({\"merge_type\": self.merge_type})\n return config\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n if len(inputs) == 1:\n return inputs\n\n if not all(\n [\n shape_compatible(input_node.shape, inputs[0].shape)\n for input_node in inputs\n ]\n ):\n inputs = [Flatten().build(hp, input_node) for input_node in inputs]\n\n # TODO: Even inputs have different shape[-1], they can still be Add(\n # ) after another layer. Check if the inputs are all of the same\n # shape\n if self._inputs_same_shape(inputs):\n merge_type = self.merge_type or hp.Choice(\n \"merge_type\", [\"add\", \"concatenate\"], default=\"add\"\n )\n if merge_type == \"add\":\n return layers.Add()(inputs)\n\n return layers.Concatenate()(inputs)\n\n def _inputs_same_shape(self, inputs):\n return all(input_node.shape == inputs[0].shape for input_node in inputs)\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Flatten(block_module.Block):\n \"\"\"Flatten the input tensor with Keras Flatten layer.\"\"\"\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n if len(input_node.shape) > 2:\n return layers.Flatten()(input_node)\n return input_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Reduction(block_module.Block):\n def __init__(self, reduction_type: Optional[str] = None, **kwargs):\n super().__init__(**kwargs)\n self.reduction_type = reduction_type\n\n def get_config(self):\n config = super().get_config()\n config.update({REDUCTION_TYPE: self.reduction_type})\n return config\n\n def global_max(self, input_node):\n raise NotImplementedError\n\n def global_avg(self, input_node):\n raise NotImplementedError\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n\n # No need to reduce.\n if len(output_node.shape) <= 2:\n return output_node\n\n if self.reduction_type is not None:\n return self._build_block(hp, output_node, self.reduction_type)\n\n reduction_type = hp.Choice(\n REDUCTION_TYPE, [FLATTEN, GLOBAL_MAX, GLOBAL_AVG]\n )\n with hp.conditional_scope(REDUCTION_TYPE, [reduction_type]):\n return self._build_block(hp, output_node, reduction_type)\n\n def _build_block(self, hp, output_node, reduction_type):\n if reduction_type == FLATTEN:\n output_node = Flatten().build(hp, output_node)\n elif reduction_type == GLOBAL_MAX:\n output_node = self.global_max(output_node)\n elif reduction_type == GLOBAL_AVG:\n output_node = self.global_avg(output_node)\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass SpatialReduction(Reduction):\n \"\"\"Reduce the dimension of a spatial tensor, e.g. image, to a vector.\n\n # Arguments\n reduction_type: String. 'flatten', 'global_max' or 'global_avg'.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(self, reduction_type: Optional[str] = None, **kwargs):\n super().__init__(reduction_type, **kwargs)\n\n def global_max(self, input_node):\n return layer_utils.get_global_max_pooling(input_node.shape)()(\n input_node\n )\n\n def global_avg(self, input_node):\n return layer_utils.get_global_average_pooling(input_node.shape)()(\n input_node\n )\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass TemporalReduction(Reduction):\n \"\"\"Reduce the dim of a temporal tensor, e.g. output of RNN, to a vector.\n\n # Arguments\n reduction_type: String. 'flatten', 'global_max' or 'global_avg'. If left\n unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(self, reduction_type: Optional[str] = None, **kwargs):\n super().__init__(reduction_type, **kwargs)\n\n def global_max(self, input_node):\n return ops.max(input_node, axis=-2)\n\n def global_avg(self, input_node):\n return ops.mean(input_node, axis=-2)\n" }, { "path": "autokeras/blocks/reduction_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport tree\n\nfrom autokeras import blocks\nfrom autokeras import test_utils\n\n\ndef test_merge_build_return_tensor():\n block = blocks.Merge()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n [\n keras.Input(shape=(32,), dtype=\"float32\"),\n keras.Input(shape=(4, 8), dtype=\"float32\"),\n ],\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_merge_single_input_return_tensor():\n block = blocks.Merge()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32,), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_merge_inputs_with_same_shape_return_tensor():\n block = blocks.Merge()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n [\n keras.Input(shape=(32,), dtype=\"float32\"),\n keras.Input(shape=(32,), dtype=\"float32\"),\n ],\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_merge_deserialize_to_merge():\n serialized_block = blocks.serialize(blocks.Merge())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.Merge)\n\n\ndef test_merge_get_config_has_all_attributes():\n block = blocks.Merge()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.Merge.__init__).issubset(\n config.keys()\n )\n\n\ndef test_temporal_build_return_tensor():\n block = blocks.TemporalReduction()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 10), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_temporal_global_max_return_tensor():\n block = blocks.TemporalReduction(reduction_type=\"global_max\")\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 10), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_temporal_global_avg_return_tensor():\n block = blocks.TemporalReduction(reduction_type=\"global_avg\")\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 10), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_reduction_2d_tensor_return_input_node():\n block = blocks.TemporalReduction()\n input_node = keras.Input(shape=(32,), dtype=\"float32\")\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n input_node,\n )\n\n assert len(tree.flatten(outputs)) == 1\n assert tree.flatten(outputs)[0] is input_node\n\n\ndef test_temporal_deserialize_to_temporal():\n serialized_block = blocks.serialize(blocks.TemporalReduction())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.TemporalReduction)\n\n\ndef test_temporal_get_config_has_all_attributes():\n block = blocks.TemporalReduction()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.TemporalReduction.__init__).issubset(\n config.keys()\n )\n\n\ndef test_spatial_build_return_tensor():\n block = blocks.SpatialReduction()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_spatial_deserialize_to_spatial():\n serialized_block = blocks.serialize(blocks.SpatialReduction())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.SpatialReduction)\n\n\ndef test_spatial_get_config_has_all_attributes():\n block = blocks.SpatialReduction()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.SpatialReduction.__init__).issubset(\n config.keys()\n )\n" }, { "path": "autokeras/blocks/wrapper.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Optional\n\nimport keras\nimport tree\n\nfrom autokeras.blocks import basic\nfrom autokeras.blocks import preprocessing\nfrom autokeras.blocks import reduction\nfrom autokeras.engine import block as block_module\nfrom autokeras.utils import utils\n\nBLOCK_TYPE = \"block_type\"\nRESNET = \"resnet\"\nXCEPTION = \"xception\"\nVANILLA = \"vanilla\"\nEFFICIENT = \"efficient\"\nNORMALIZE = \"normalize\"\nAUGMENT = \"augment\"\nMAX_TOKENS = \"max_tokens\"\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass ImageBlock(block_module.Block):\n \"\"\"Block for image data.\n\n The image blocks is a block choosing from ResNetBlock, XceptionBlock,\n ConvBlock, which is controlled by a hyperparameter, 'block_type'.\n\n # Arguments\n block_type: String. 'resnet', 'xception', 'vanilla'. The type of Block\n to use. If unspecified, it will be tuned automatically.\n normalize: Boolean. Whether to channel-wise normalize the images.\n If unspecified, it will be tuned automatically.\n augment: Boolean. Whether to do image augmentation. If unspecified,\n it will be tuned automatically.\n \"\"\"\n\n def __init__(\n self,\n block_type: Optional[str] = None,\n normalize: Optional[bool] = None,\n augment: Optional[bool] = None,\n **kwargs\n ):\n super().__init__(**kwargs)\n self.block_type = block_type\n self.normalize = normalize\n self.augment = augment\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n BLOCK_TYPE: self.block_type,\n NORMALIZE: self.normalize,\n AUGMENT: self.augment,\n }\n )\n return config\n\n def _build_block(self, hp, output_node, block_type):\n if block_type == RESNET:\n return basic.ResNetBlock().build(hp, output_node)\n elif block_type == XCEPTION:\n return basic.XceptionBlock().build(hp, output_node)\n elif block_type == VANILLA:\n return basic.ConvBlock().build(hp, output_node)\n elif block_type == EFFICIENT:\n return basic.EfficientNetBlock().build(hp, output_node)\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n output_node = input_node\n\n if self.normalize is None and hp.Boolean(NORMALIZE):\n with hp.conditional_scope(NORMALIZE, [True]):\n output_node = preprocessing.Normalization().build(\n hp, output_node\n )\n elif self.normalize:\n output_node = preprocessing.Normalization().build(hp, output_node)\n\n if self.augment is None and hp.Boolean(AUGMENT):\n with hp.conditional_scope(AUGMENT, [True]):\n output_node = preprocessing.ImageAugmentation().build(\n hp, output_node\n )\n elif self.augment:\n output_node = preprocessing.ImageAugmentation().build(\n hp, output_node\n )\n\n if self.block_type is None:\n block_type = hp.Choice(\n BLOCK_TYPE, [RESNET, XCEPTION, VANILLA, EFFICIENT]\n )\n with hp.conditional_scope(BLOCK_TYPE, [block_type]):\n output_node = self._build_block(hp, output_node, block_type)\n else:\n output_node = self._build_block(hp, output_node, self.block_type)\n\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass TextBlock(block_module.Block):\n \"\"\"Block for text data.\n\n # Arguments\n max_tokens: Int. The maximum size of the vocabulary.\n If left unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(self, max_tokens=None, **kwargs):\n super().__init__(**kwargs)\n self.max_tokens = max_tokens\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n output_node = input_node\n output_node = self._build_block(hp, output_node)\n return output_node\n\n def get_config(self):\n config = super().get_config()\n config.update({\"max_tokens\": self.max_tokens})\n return config\n\n def _build_block(self, hp, output_node):\n # Use Embedding and dense layers for tokenized text\n max_tokens = self.max_tokens or hp.Choice(\n MAX_TOKENS, [500, 5000, 20000], default=5000\n )\n output_node = basic.Embedding(\n max_features=max_tokens + 1,\n ).build(hp, output_node)\n output_node = basic.ConvBlock().build(hp, output_node)\n output_node = reduction.SpatialReduction().build(hp, output_node)\n output_node = basic.DenseBlock().build(hp, output_node)\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass GeneralBlock(block_module.Block):\n \"\"\"A general neural network block when the input type is unknown.\n\n When the input type is unknown. The GeneralBlock would search in a large\n space for a good model.\n\n # Arguments\n name: String.\n \"\"\"\n\n def build(self, hp, inputs=None):\n inputs = tree.flatten(inputs)\n utils.validate_num_inputs(inputs, 1)\n input_node = inputs[0]\n output_node = input_node\n\n output_node = reduction.Flatten().build(hp, output_node)\n output_node = basic.DenseBlock().build(hp, output_node)\n return output_node\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass StructuredDataBlock(block_module.Block):\n \"\"\"Block for structured data.\n\n # Arguments\n categorical_encoding: Boolean. Whether to use the CategoricalToNumerical\n to encode the categorical features to numerical features. Defaults\n to True.\n normalize: Boolean. Whether to normalize the features.\n If unspecified, it will be tuned automatically.\n \"\"\"\n\n def __init__(self, normalize: Optional[bool] = None, **kwargs):\n super().__init__(**kwargs)\n self.normalize = normalize\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"normalize\": self.normalize,\n }\n )\n return config\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n output_node = input_node\n\n if self.normalize is None and hp.Boolean(NORMALIZE):\n with hp.conditional_scope(NORMALIZE, [True]):\n output_node = preprocessing.Normalization().build(\n hp, output_node\n )\n elif self.normalize:\n output_node = preprocessing.Normalization().build(hp, output_node)\n\n output_node = basic.DenseBlock().build(hp, output_node)\n return output_node\n" }, { "path": "autokeras/blocks/wrapper_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport tree\n\nfrom autokeras import analysers\nfrom autokeras import blocks\nfrom autokeras import test_utils\n\n\ndef test_image_build_return_tensor():\n block = blocks.ImageBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_general_build_return_tensor():\n block = blocks.GeneralBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_image_block_xception_return_tensor():\n block = blocks.ImageBlock(block_type=\"xception\")\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_image_block_normalize_return_tensor():\n block = blocks.ImageBlock(normalize=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_image_block_augment_return_tensor():\n block = blocks.ImageBlock(augment=True)\n\n outputs = block.build(\n keras_tuner.HyperParameters(),\n keras.Input(shape=(32, 32, 3), dtype=\"float32\"),\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_image_deserialize_to_image():\n serialized_block = blocks.serialize(blocks.ImageBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.ImageBlock)\n\n\ndef test_image_get_config_has_all_attributes():\n block = blocks.ImageBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.ImageBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_text_build_return_tensor():\n block = blocks.TextBlock()\n\n outputs = block.build(\n keras_tuner.HyperParameters(), keras.Input(shape=(1,), dtype=\"string\")\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_text_deserialize_to_text():\n serialized_block = blocks.serialize(blocks.TextBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.TextBlock)\n\n\ndef test_text_get_config_has_all_attributes():\n block = blocks.TextBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(blocks.TextBlock.__init__).issubset(\n config.keys()\n )\n\n\ndef test_structured_build_return_tensor():\n block = blocks.StructuredDataBlock()\n block.column_names = [\"0\", \"1\"]\n block.column_types = {\"0\": analysers.NUMERICAL, \"1\": analysers.NUMERICAL}\n\n outputs = block.build(\n keras_tuner.HyperParameters(), keras.Input(shape=(2,), dtype=\"string\")\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_structured_block_normalize_return_tensor():\n block = blocks.StructuredDataBlock(normalize=True)\n block.column_names = [\"0\", \"1\"]\n block.column_types = {\"0\": analysers.NUMERICAL, \"1\": analysers.NUMERICAL}\n\n outputs = block.build(\n keras_tuner.HyperParameters(), keras.Input(shape=(2,), dtype=\"string\")\n )\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_structured_block_search_normalize_return_tensor():\n block = blocks.StructuredDataBlock(name=\"a\")\n block.column_names = [\"0\", \"1\"]\n block.column_types = {\"0\": analysers.NUMERICAL, \"1\": analysers.NUMERICAL}\n hp = keras_tuner.HyperParameters()\n hp.values[\"a/\" + blocks.wrapper.NORMALIZE] = True\n\n outputs = block.build(hp, keras.Input(shape=(2,), dtype=\"string\"))\n\n assert len(tree.flatten(outputs)) == 1\n\n\ndef test_structured_deserialize_to_structured():\n serialized_block = blocks.serialize(blocks.StructuredDataBlock())\n\n block = blocks.deserialize(serialized_block)\n\n assert isinstance(block, blocks.StructuredDataBlock)\n\n\ndef test_structured_get_config_has_all_attributes():\n block = blocks.StructuredDataBlock()\n\n config = block.get_config()\n\n assert test_utils.get_func_args(\n blocks.StructuredDataBlock.__init__\n ).issubset(config.keys())\n" }, { "path": "autokeras/conftest.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport shutil\n\nimport keras\nimport pytest\n\n\n@pytest.fixture(autouse=True)\ndef clear_session():\n keras.backend.clear_session()\n yield\n keras.backend.clear_session()\n\n\n@pytest.fixture(autouse=True)\ndef remove_tmp_path(tmp_path):\n yield\n shutil.rmtree(tmp_path)\n\n\n@pytest.fixture(autouse=True)\ndef disable_traceback_filtering():\n keras.config.disable_traceback_filtering()\n yield\n" }, { "path": "autokeras/engine/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/engine/adapter.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nclass Adapter(object):\n \"\"\"Adpat the input and output format for Keras Model.\n\n Adapter is used by the input nodes and the heads of the hypermodel. It do\n some type checking for the data and converts it to compatible format.\n \"\"\"\n\n def check(self, dataset):\n \"\"\"Check if the dataset is valid for the input node.\n\n # Arguments\n dataset: numpy.ndarray. The dataset to be checked.\n\n # Returns\n Boolean. Whether the dataset is in compatible format.\n \"\"\"\n return True\n\n def adapt(self, dataset):\n \"\"\"Check, convert and batch the dataset.\n\n # Arguments\n dataset: Usually numpy.ndarray. The dataset to be converted.\n\n # Returns\n numpy.ndarray. The converted dataset.\n \"\"\"\n self.check(dataset)\n return dataset\n" }, { "path": "autokeras/engine/adapter_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.engine import adapter as adapter_module\n\n\ndef test_adapter_check_return_true():\n adapter = adapter_module.Adapter()\n\n assert adapter.check(None)\n" }, { "path": "autokeras/engine/analyser.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport numpy as np\n\n\nclass Analyser(object):\n \"\"\"Analyze the dataset for useful information.\n\n Analyser is used by the input nodes and the heads of the hypermodel. It\n analyzes the dataset to get useful information, e.g., the shape of the\n data, the data type of the dataset. The information will be used by the\n input nodes and heads to construct the data pipeline and to build the Keras\n Model.\n \"\"\"\n\n def __init__(self, **kwargs):\n super().__init__(**kwargs)\n # Shape is a list of integers\n self.shape = None\n self.dtype = None\n self.num_samples = 0\n self.batch_size = None\n\n def update(self, data):\n \"\"\"Update the statistics with a batch of data.\n\n # Arguments\n data: np.ndarray. The entire dataset.\n \"\"\"\n if self.dtype is None:\n if np.issubdtype(data.dtype, np.str_) or np.issubdtype(\n data.dtype, np.bytes_\n ):\n self.dtype = \"string\"\n else:\n self.dtype = str(data.dtype)\n if self.shape is None:\n self.shape = list(data.shape)\n if self.batch_size is None:\n self.batch_size = data.shape[0]\n self.num_samples += data.shape[0]\n\n def finalize(self):\n \"\"\"Process recorded information after all updates.\"\"\"\n raise NotImplementedError\n" }, { "path": "autokeras/engine/analyser_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport pytest\n\nfrom autokeras.engine.analyser import Analyser\n\n\ndef test_analyser_update_unicode_string_dtype():\n analyser = Analyser()\n data = np.array([\"hello\", \"world\"], dtype=\"U10\")\n\n analyser.update(data)\n\n assert analyser.dtype == \"string\"\n assert analyser.shape == [2]\n assert analyser.batch_size == 2\n assert analyser.num_samples == 2\n\n\ndef test_analyser_update_byte_string_dtype():\n analyser = Analyser()\n data = np.array([b\"hello\", b\"world\"], dtype=\"S10\")\n\n analyser.update(data)\n\n assert analyser.dtype == \"string\"\n assert analyser.shape == [2]\n assert analyser.batch_size == 2\n assert analyser.num_samples == 2\n\n\ndef test_analyser_update_numeric_dtype():\n analyser = Analyser()\n data = np.array([1, 2, 3], dtype=np.int32)\n\n analyser.update(data)\n\n assert analyser.dtype == \"int32\"\n assert analyser.shape == [3]\n assert analyser.batch_size == 3\n assert analyser.num_samples == 3\n\n\ndef test_analyser_update_float_dtype():\n analyser = Analyser()\n data = np.array([1.0, 2.0, 3.0], dtype=np.float64)\n\n analyser.update(data)\n\n assert analyser.dtype == \"float64\"\n assert analyser.shape == [3]\n assert analyser.batch_size == 3\n assert analyser.num_samples == 3\n\n\ndef test_analyser_finalize_not_implemented():\n analyser = Analyser()\n\n with pytest.raises(NotImplementedError):\n analyser.finalize()\n" }, { "path": "autokeras/engine/block.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport tree\n\nfrom autokeras.engine import named_hypermodel\nfrom autokeras.engine import node as node_module\n\n\nclass Block(named_hypermodel.NamedHyperModel):\n \"\"\"The base class for different Block.\n\n The Block can be connected together to build the search space for an\n AutoModel. Notably, many args in the __init__ function are defaults to be a\n tunable variable when not specified by the user.\n\n \"\"\"\n\n def __init__(self, **kwargs):\n super().__init__(**kwargs)\n self.inputs = None\n self.outputs = None\n self._num_output_node = 1\n\n def _build_wrapper(self, hp, *args, **kwargs):\n with hp.name_scope(self.name):\n return super()._build_wrapper(hp, *args, **kwargs)\n\n def __call__(self, inputs):\n \"\"\"Functional API.\n\n # Arguments\n inputs: A list of input node(s) or a single input node for the\n block.\n\n # Returns\n list: A list of output node(s) of the Block.\n \"\"\"\n self.inputs = tree.flatten(inputs)\n for input_node in self.inputs:\n if not isinstance(input_node, node_module.Node):\n raise TypeError(\n \"Expect the inputs to block {name} to be \"\n \"a Node, but got {type}.\".format(\n name=self.name, type=type(input_node)\n )\n )\n input_node.add_out_block(self)\n self.outputs = []\n for _ in range(self._num_output_node):\n output_node = node_module.Node()\n output_node.add_in_block(self)\n self.outputs.append(output_node)\n return self.outputs\n\n def build(self, hp, inputs=None):\n \"\"\"Build the Block into a real Keras Model.\n\n The subclasses should override this function and return the output node.\n\n # Arguments\n hp: HyperParameters. The hyperparameters for building the model.\n inputs: A list of input node(s).\n \"\"\"\n raise NotImplementedError\n" }, { "path": "autokeras/engine/block_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport pytest\n\nfrom autokeras.engine import block as block_module\n\n\ndef test_block_call_raise_inputs_type_error():\n block = block_module.Block()\n\n with pytest.raises(TypeError) as info:\n block(None)\n\n assert \"Expect the inputs to block\" in str(info.value)\n" }, { "path": "autokeras/engine/head.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nfrom typing import Optional\n\nimport keras\n\nfrom autokeras.engine import io_hypermodel\nfrom autokeras.utils import types\n\n\ndef serialize_metrics(metrics):\n serialized = []\n for metric in metrics:\n if isinstance(metric, str):\n serialized.append([metric])\n else:\n serialized.append(keras.metrics.serialize(metric))\n return serialized\n\n\ndef deserialize_metrics(metrics):\n deserialized = []\n for metric in metrics:\n if isinstance(metric, list):\n deserialized.append(metric[0])\n else:\n deserialized.append(keras.metrics.deserialize(metric))\n return deserialized\n\n\ndef serialize_loss(loss):\n if isinstance(loss, str):\n return [loss]\n return keras.losses.serialize(loss)\n\n\ndef deserialize_loss(loss):\n if isinstance(loss, list):\n return loss[0]\n return keras.losses.deserialize(loss)\n\n\nclass Head(io_hypermodel.IOHyperModel):\n \"\"\"Base class for the heads, e.g. classification, regression.\n\n # Arguments\n loss: A Keras loss function. Defaults to None. If None, the loss will be\n inferred from the AutoModel.\n metrics: A list of Keras metrics. Defaults to None. If None, the metrics\n will be inferred from the AutoModel.\n \"\"\"\n\n def __init__(\n self,\n loss: Optional[types.LossType] = None,\n metrics: Optional[types.MetricsType] = None,\n **kwargs\n ):\n super().__init__(**kwargs)\n self.loss = loss\n if metrics is None:\n metrics = []\n self.metrics = metrics\n # Mark if the head should directly output the input tensor.\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"loss\": serialize_loss(self.loss),\n \"metrics\": serialize_metrics(self.metrics),\n }\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"loss\"] = deserialize_loss(config[\"loss\"])\n config[\"metrics\"] = deserialize_metrics(config[\"metrics\"])\n return super().from_config(config)\n\n def build(self, hp, inputs=None):\n raise NotImplementedError\n" }, { "path": "autokeras/engine/head_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.engine import head as head_module\n\n\ndef test_init_head_none_metrics():\n assert isinstance(head_module.Head().metrics, list)\n" }, { "path": "autokeras/engine/hyper_preprocessor.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.engine import named_hypermodel\n\n\nclass HyperPreprocessor(named_hypermodel.NamedHyperModel):\n \"\"\"Input data preprocessor search space.\n\n This class defines the search space for a Preprocessor.\n \"\"\"\n\n def build(self, hp, dataset):\n \"\"\"Build the input preprocessor.\n\n # Arguments\n hp: `HyperParameters` instance. The hyperparameters for building the\n a Preprocessor.\n dataset: np.ndarray. The dataset to be preprocessed.\n\n # Returns\n an instance of Preprocessor.\n \"\"\"\n raise NotImplementedError\n" }, { "path": "autokeras/engine/io_hypermodel.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.engine import block as block_module\n\n\nclass IOHyperModel(block_module.Block):\n \"\"\"A mixin class connecting the input nodes and heads with the adapters.\n\n This class is extended by the input nodes and the heads. The AutoModel calls\n the functions to get the corresponding adapters and pass the information\n back to the input nodes and heads.\n \"\"\"\n\n def __init__(self, shape=None, **kwargs):\n super().__init__(**kwargs)\n self.shape = shape\n self.data_shape = None\n self.dtype = None\n self.batch_size = None\n self.num_samples = None\n\n def get_analyser(self):\n \"\"\"Get the corresponding Analyser.\n\n # Returns\n An instance of a subclass of autokeras.engine.Analyser.\n \"\"\"\n raise NotImplementedError\n\n def get_adapter(self):\n \"\"\"Get the corresponding Adapter.\n\n # Returns\n An instance of a subclass of autokeras.engine.Adapter.\n \"\"\"\n raise NotImplementedError\n\n def config_from_analyser(self, analyser):\n \"\"\"Load the learned information on dataset from the Analyser.\n\n # Arguments\n adapter: An instance of a subclass of autokeras.engine.Adapter.\n \"\"\"\n self.data_shape = analyser.shape\n self.dtype = analyser.dtype\n self.batch_size = analyser.batch_size\n self.num_samples = analyser.num_samples\n\n def get_hyper_preprocessors(self):\n \"\"\"Construct a list of HyperPreprocessors based on learned information.\n\n # Returns\n A list of HyperPreprocessors for the corresponding data.\n \"\"\"\n raise NotImplementedError\n\n def get_config(self):\n config = super().get_config()\n config.update({\"shape\": self.shape})\n return config\n" }, { "path": "autokeras/engine/named_hypermodel.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\n\nfrom autokeras.engine import serializable\nfrom autokeras.utils import utils\n\n\nclass NamedHyperModel(keras_tuner.HyperModel, serializable.Serializable):\n \"\"\"\n\n # Arguments\n name: String. The name of the HyperModel. If unspecified, it will be set\n automatically with the class name.\n \"\"\"\n\n def __init__(self, name: str = None, **kwargs):\n if not name:\n prefix = self.__class__.__name__\n name = prefix + \"_\" + str(keras.backend.get_uid(prefix))\n name = utils.to_snake_case(name)\n super().__init__(name=name, **kwargs)\n\n def get_config(self):\n \"\"\"Get the configuration of the preprocessor.\n\n # Returns\n A dictionary of configurations of the preprocessor.\n \"\"\"\n return {\"name\": self.name, \"tunable\": self.tunable}\n" }, { "path": "autokeras/engine/node.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nclass Node(object):\n \"\"\"The nodes in a network connecting the blocks.\"\"\"\n\n def __init__(self, **kwargs):\n super().__init__(**kwargs)\n self.in_blocks = []\n self.out_blocks = []\n\n def add_in_block(self, hypermodel):\n self.in_blocks.append(hypermodel)\n\n def add_out_block(self, hypermodel):\n self.out_blocks.append(hypermodel)\n" }, { "path": "autokeras/engine/node_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/engine/preprocessor.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.engine import serializable\n\n\nclass Preprocessor(serializable.Serializable):\n \"\"\"A preprocessor for np.ndarray.\n\n A preprocessor transforms the dataset (numpy.ndarray).\n \"\"\"\n\n def fit(self, dataset):\n \"\"\"Fit the preprocessor with the dataset.\n\n # Arguments\n dataset: an instance of `np.ndarray`.\n \"\"\"\n # TODO: may need to change to a streaming way of fit to reduce the\n # number of iterations through the dataset for speed. Need to be\n # decided when we have more use cases for this fit.\n pass\n\n def transform(self, dataset):\n \"\"\"Transform the dataset wth the preprocessor.\n\n # Arguments\n dataset: an instance of `np.ndarray`.\n\n # Returns\n The transformed dataset.\n \"\"\"\n raise NotImplementedError\n\n def get_config(self):\n return {}\n\n\nclass TargetPreprocessor(Preprocessor):\n \"\"\"Preprocessor for target data.\"\"\"\n\n def postprocess(self, dataset):\n \"\"\"Postprocess the output of the Keras model.\n\n # Arguments\n dataset: numpy.ndarray. The corresponding output of the model.\n\n # Returns\n numpy.ndarray. The postprocessed data.\n \"\"\"\n raise NotImplementedError\n" }, { "path": "autokeras/engine/serializable.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nclass Serializable(object):\n \"\"\"Serializable from and to JSON with same mechanism as Keras Layer.\"\"\"\n\n def get_config(self):\n \"\"\"Returns the current config of this object.\n\n # Returns\n Dictionary.\n \"\"\"\n raise NotImplementedError\n\n @classmethod\n def from_config(cls, config):\n \"\"\"Build an instance from the config of this object.\n\n # Arguments\n config: Dict. The config of the object.\n \"\"\"\n return cls(**config)\n" }, { "path": "autokeras/engine/tuner.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport collections\nimport copy\nimport os\n\nimport keras\nimport keras_tuner\nimport numpy as np\nimport tree\nfrom keras import callbacks as callbacks_module\n\nfrom autokeras import keras_layers\nfrom autokeras import pipeline as pipeline_module\nfrom autokeras.utils import data_utils\nfrom autokeras.utils import utils\n\n\nclass AutoTuner(keras_tuner.engine.tuner.Tuner):\n \"\"\"A Tuner class based on KerasTuner for AutoKeras.\n\n Different from KerasTuner's Tuner class. AutoTuner's not only tunes the\n Hypermodel which can be directly built into a Keras model, but also the\n preprocessors. Therefore, a HyperGraph stores the overall search space\n containing both the Preprocessors and Hypermodel. For every trial, the\n HyperGraph builds the PreprocessGraph and KerasGraph with the provided\n HyperParameters.\n\n The AutoTuner uses EarlyStopping for acceleration during the search and\n fully trains the model with full epochs and with both training and\n validation data. The fully trained model is the best model to be used by\n AutoModel.\n\n # Arguments\n oracle: keras_tuner Oracle.\n hypermodel: keras_tuner HyperModel.\n **kwargs: The args supported by KerasTuner.\n \"\"\"\n\n def __init__(self, oracle, hypermodel, **kwargs):\n # Initialize before super() for reload to work.\n self._finished = False\n super().__init__(oracle, hypermodel, **kwargs)\n # Save or load the HyperModel.\n self.hypermodel.save(os.path.join(self.project_dir, \"graph\"))\n self.hyper_pipeline = None\n\n def _populate_initial_space(self):\n # Override the function to prevent building the model during\n # initialization.\n return\n\n def get_best_model(self):\n return self.get_best_models()[0]\n\n def get_best_pipeline(self):\n return pipeline_module.load_pipeline(self.best_pipeline_path)\n\n def _pipeline_path(self, trial_id):\n return os.path.join(self.get_trial_dir(trial_id), \"pipeline\")\n\n def _prepare_model_build(self, hp, **kwargs):\n \"\"\"Prepare for building the Keras model.\n\n It builds the Pipeline from HyperPipeline, transforms the dataset to set\n the input shapes and output shapes of the HyperModel.\n \"\"\"\n x = kwargs[\"x\"]\n y = kwargs[\"y\"]\n pipeline = self.hyper_pipeline.build(hp, (x, y))\n pipeline.fit((x, y))\n (x, y) = pipeline.transform((x, y))\n self.hypermodel.set_io_shapes(data_utils.dataset_shape((x, y)))\n\n if \"validation_data\" in kwargs:\n validation_data = pipeline.transform(kwargs[\"validation_data\"])\n else:\n validation_data = None\n return pipeline, (x, y), validation_data\n\n def _build_and_fit_model(self, trial, *args, **kwargs):\n model = self._try_build(trial.hyperparameters)\n (\n pipeline,\n (\n kwargs[\"x\"],\n kwargs[\"y\"],\n ),\n kwargs[\"validation_data\"],\n ) = self._prepare_model_build(trial.hyperparameters, **kwargs)\n pipeline.save(self._pipeline_path(trial.trial_id))\n keras.src.backend.compute_output_spec(model, kwargs[\"x\"])\n\n self.adapt(model, kwargs[\"x\"])\n\n _, history = utils.fit_with_adaptive_batch_size(model, **kwargs)\n return history\n\n @staticmethod\n def adapt(model, dataset):\n \"\"\"Adapt the preprocessing layers in the model.\"\"\"\n # Currently, only support using the original dataset to adapt all the\n # preprocessing layers before the first non-preprocessing layer.\n # TODO: Use PreprocessingStage for preprocessing layers adapt.\n # TODO: Use Keras Tuner for preprocessing layers adapt.\n x = tree.flatten(dataset)\n\n def get_output_layers(tensor):\n output_layers = []\n tensor = tree.flatten(tensor)[0]\n for layer in model.layers:\n if isinstance(layer, keras.layers.InputLayer):\n continue\n input_node = tree.flatten(layer.input)[0]\n if input_node is tensor:\n if isinstance(\n layer,\n keras_layers.PreprocessingLayer,\n ) or hasattr(layer, \"adapt\"):\n output_layers.append(layer)\n return output_layers\n\n dq = collections.deque()\n\n for index, input_node in enumerate(tree.flatten(model.input)):\n in_x = x[index]\n for layer in get_output_layers(input_node):\n dq.append((layer, in_x))\n\n while len(dq):\n layer, in_x = dq.popleft()\n layer.adapt(in_x)\n out_x = layer(in_x)\n for next_layer in get_output_layers(layer.output):\n dq.append((next_layer, out_x))\n\n return model\n\n def search(\n self,\n epochs=None,\n callbacks=None,\n validation_split=0,\n verbose=1,\n **fit_kwargs\n ):\n \"\"\"Search for the best HyperParameters.\n\n If there is not early-stopping in the callbacks, the early-stopping\n callback is injected to accelerate the search process. At the end of the\n search, the best model will be fully trained with the specified number\n of epochs.\n\n # Arguments\n callbacks: A list of callback functions. Defaults to None.\n validation_split: Float.\n \"\"\"\n if self._finished:\n return\n\n if callbacks is None:\n callbacks = []\n\n self.hypermodel.set_fit_args(validation_split, epochs=epochs)\n\n # Insert early-stopping for adaptive number of epochs.\n epochs_provided = True\n if epochs is None:\n epochs_provided = False\n epochs = 1000\n if not utils.contain_instance(\n callbacks, callbacks_module.EarlyStopping\n ):\n callbacks.append(\n callbacks_module.EarlyStopping(patience=10, min_delta=1e-4)\n )\n\n # Insert early-stopping for acceleration.\n early_stopping_inserted = False\n new_callbacks = self._deepcopy_callbacks(callbacks)\n if not utils.contain_instance(\n callbacks, callbacks_module.EarlyStopping\n ):\n early_stopping_inserted = True\n new_callbacks.append(\n callbacks_module.EarlyStopping(patience=10, min_delta=1e-4)\n )\n\n # Populate initial search space.\n hp = self.oracle.get_space()\n self._prepare_model_build(hp, **fit_kwargs)\n self._try_build(hp)\n self.oracle.update_space(hp)\n super().search(\n epochs=epochs,\n callbacks=new_callbacks,\n verbose=verbose,\n **fit_kwargs\n )\n\n # Train the best model use validation data.\n # Train the best model with enough number of epochs.\n if validation_split > 0 or early_stopping_inserted:\n copied_fit_kwargs = copy.copy(fit_kwargs)\n\n # Remove early-stopping since no validation data.\n # Remove early-stopping since it is inserted.\n copied_fit_kwargs[\"callbacks\"] = self._remove_early_stopping(\n callbacks\n )\n\n # Decide the number of epochs.\n copied_fit_kwargs[\"epochs\"] = epochs\n if not epochs_provided:\n copied_fit_kwargs[\"epochs\"] = self._get_best_trial_epochs()\n\n # Concatenate training and validation data.\n if validation_split > 0:\n x, y = copied_fit_kwargs[\"x\"], copied_fit_kwargs[\"y\"]\n x_val, y_val = fit_kwargs[\"validation_data\"]\n copied_fit_kwargs[\"x\"] = tree.map_structure(\n lambda train, val: np.concatenate([train, val], axis=0),\n x,\n x_val,\n )\n copied_fit_kwargs[\"y\"] = tree.map_structure(\n lambda train, val: np.concatenate([train, val], axis=0),\n y,\n y_val,\n )\n copied_fit_kwargs.pop(\"validation_data\")\n\n self.hypermodel.set_fit_args(0, epochs=copied_fit_kwargs[\"epochs\"])\n copied_fit_kwargs[\"verbose\"] = verbose\n pipeline, model, history = self.final_fit(**copied_fit_kwargs)\n else:\n # TODO: Add return history functionality in Keras Tuner\n model = self.get_best_model()\n history = None\n pipeline = pipeline_module.load_pipeline(\n self._pipeline_path(self.oracle.get_best_trials(1)[0].trial_id)\n )\n\n model.save(self.best_model_path)\n pipeline.save(self.best_pipeline_path)\n self._finished = True\n return history\n\n def get_state(self):\n state = super().get_state()\n state.update({\"finished\": self._finished})\n return state\n\n def set_state(self, state):\n super().set_state(state)\n self._finished = state.get(\"finished\")\n\n @staticmethod\n def _remove_early_stopping(callbacks):\n return [\n copy.deepcopy(callbacks)\n for callback in callbacks\n if not isinstance(callback, callbacks_module.EarlyStopping)\n ]\n\n def _get_best_trial_epochs(self):\n best_trial = self.oracle.get_best_trials(1)[0]\n # steps counts from 0, so epochs = step + 1.\n return self.oracle.get_trial(best_trial.trial_id).best_step + 1\n\n def _build_best_model(self):\n best_trial = self.oracle.get_best_trials(1)[0]\n best_hp = best_trial.hyperparameters\n return self._try_build(best_hp)\n\n def final_fit(self, **kwargs):\n best_trial = self.oracle.get_best_trials(1)[0]\n best_hp = best_trial.hyperparameters\n (\n pipeline,\n (kwargs[\"x\"], kwargs[\"y\"]),\n kwargs[\"validation_data\"],\n ) = self._prepare_model_build(best_hp, **kwargs)\n\n model = self._build_best_model()\n self.adapt(model, kwargs[\"x\"])\n model, history = utils.fit_with_adaptive_batch_size(model, **kwargs)\n return pipeline, model, history\n\n @property\n def best_model_path(self):\n return os.path.join(self.project_dir, \"best_model.keras\")\n\n @property\n def best_pipeline_path(self):\n return os.path.join(self.project_dir, \"best_pipeline\")\n\n @property\n def objective(self):\n return self.oracle.objective\n\n @property\n def max_trials(self):\n return self.oracle.max_trials\n" }, { "path": "autokeras/engine/tuner_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport keras\nimport numpy as np\nimport tree\n\nimport autokeras as ak\nfrom autokeras import test_utils\nfrom autokeras.tuners import greedy\n\n\ndef called_with_early_stopping(func):\n callbacks = func.call_args_list[0][1][\"callbacks\"]\n return any(\n [\n isinstance(callback, keras.callbacks.EarlyStopping)\n for callback in callbacks\n ]\n )\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\ndef test_final_fit_with_specified_epochs(_, final_fit, super_search, tmp_path):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n final_fit.return_value = mock.Mock(), mock.Mock(), mock.Mock()\n\n tuner.search(x=None, epochs=10, validation_data=None)\n\n assert final_fit.call_args_list[0][1][\"epochs\"] == 10\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\ndef test_tuner_call_super_with_early_stopping(\n _, final_fit, super_search, tmp_path\n):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n final_fit.return_value = mock.Mock(), mock.Mock(), mock.Mock()\n\n tuner.search(x=None, epochs=10, validation_data=None)\n\n assert called_with_early_stopping(super_search)\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\n \"autokeras.engine.tuner.AutoTuner.get_best_models\",\n return_value=[mock.Mock()],\n)\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\n@mock.patch(\"autokeras.pipeline.load_pipeline\")\n@mock.patch(\"keras_tuner.Oracle.get_best_trials\", return_value=[mock.Mock()])\ndef test_no_final_fit_without_epochs_and_fov(\n _, _1, _2, get_best_models, final_fit, super_search, tmp_path\n):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n\n tuner.search(x=None, epochs=None, validation_data=None)\n\n final_fit.assert_not_called()\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\n \"autokeras.engine.tuner.AutoTuner._get_best_trial_epochs\", return_value=2\n)\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\ndef test_final_fit_best_epochs_if_epoch_unspecified(\n _, best_epochs, final_fit, super_search, tmp_path\n):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n final_fit.return_value = mock.Mock(), mock.Mock(), mock.Mock()\n\n tuner.search(\n x=np.random.rand(100, 32, 32, 3),\n y=np.random.rand(100, 1),\n epochs=None,\n validation_split=0.2,\n validation_data=(np.random.rand(20, 32, 32, 3), np.random.rand(20, 1)),\n )\n\n assert final_fit.call_args_list[0][1][\"epochs\"] == 2\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\n \"autokeras.engine.tuner.AutoTuner._get_best_trial_epochs\", return_value=2\n)\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\ndef test_super_with_1k_epochs_if_epoch_unspecified(\n _, best_epochs, final_fit, super_search, tmp_path\n):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n final_fit.return_value = mock.Mock(), mock.Mock(), mock.Mock()\n\n # TODO: try to use x, and y instead of tuple input across the lib.\n tuner.search(\n x=np.random.rand(100, 32, 32, 3),\n y=np.random.rand(100, 1),\n epochs=None,\n validation_split=0.2,\n validation_data=(np.random.rand(20, 32, 32, 3), np.random.rand(20, 1)),\n )\n\n assert super_search.call_args_list[0][1][\"epochs\"] == 1000\n assert called_with_early_stopping(super_search)\n\n\n@mock.patch(\"keras_tuner.engine.base_tuner.BaseTuner.search\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner.final_fit\")\n@mock.patch(\"autokeras.engine.tuner.AutoTuner._prepare_model_build\")\ndef test_tuner_not_call_super_search_with_overwrite(\n _, final_fit, super_search, tmp_path\n):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n final_fit.return_value = mock.Mock(), mock.Mock(), mock.Mock()\n\n tuner.search(x=None, epochs=10, validation_data=None)\n tuner.save()\n super_search.reset_mock()\n\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(), directory=tmp_path\n )\n tuner.search(x=None, epochs=10, validation_data=None)\n\n super_search.assert_not_called()\n\n\ndef test_tuner_does_not_crash_with_distribution_strategy(tmp_path):\n tuner = greedy.Greedy(\n hypermodel=test_utils.build_graph(),\n directory=tmp_path,\n )\n tuner.hypermodel.build(tuner.oracle.hyperparameters)\n\n\ndef test_adapt_with_model_with_preprocessing_layer_only():\n input_node = keras.Input(shape=(10,))\n output_node = keras.layers.Normalization()(input_node)\n model = keras.Model(input_node, output_node)\n greedy.Greedy.adapt(\n model,\n (np.random.rand(100, 10), np.random.rand(100, 10)),\n )\n\n\ndef test_build_block_in_blocks_with_same_name(tmp_path):\n class Block1(ak.Block):\n def build(self, hp, inputs):\n hp.Boolean(\"a\")\n return keras.layers.Dense(3)(tree.flatten(inputs)[0])\n\n class Block2(ak.Block):\n def build(self, hp, inputs):\n hp.Boolean(\"b\")\n return Block1().build(hp, inputs)\n\n inputs = ak.Input()\n outputs = Block2()(inputs)\n outputs = ak.RegressionHead()(outputs)\n auto_model = ak.AutoModel(inputs, outputs, max_trials=5, directory=tmp_path)\n auto_model.fit(np.random.rand(100, 5), np.random.rand(100, 1), epochs=1)\n\n trials = [\n trial for trial_id, trial in auto_model.tuner.oracle.trials.items()\n ]\n for trial in trials:\n assert len(trial.hyperparameters.values) == len(\n trials[0].hyperparameters.values\n )\n" }, { "path": "autokeras/graph.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport keras_tuner\nimport tree\n\nfrom autokeras import blocks as blocks_module\nfrom autokeras import nodes as nodes_module\nfrom autokeras.engine import head as head_module\nfrom autokeras.engine import serializable\nfrom autokeras.utils import io_utils\n\n\ndef feature_encoding_input(block):\n \"\"\"Fetch the column_types and column_names.\n\n The values are fetched for FeatureEncoding from StructuredDataInput.\n \"\"\"\n block.column_types = block.inputs[0].column_types\n block.column_names = block.inputs[0].column_names\n\n\n# Compile the graph.\nCOMPILE_FUNCTIONS = {\n blocks_module.StructuredDataBlock: [feature_encoding_input],\n}\n\n\ndef load_graph(filepath, custom_objects=None):\n if custom_objects is None:\n custom_objects = {}\n with keras.utils.custom_object_scope(custom_objects):\n return Graph.from_config(io_utils.load_json(filepath))\n\n\nclass Graph(keras_tuner.HyperModel, serializable.Serializable):\n \"\"\"A graph consists of connected Blocks, or Heads.\n\n # Arguments\n inputs: A list of input node(s) for the Graph.\n outputs: A list of output node(s) for the Graph.\n \"\"\"\n\n def __init__(self, inputs=None, outputs=None, **kwargs):\n super().__init__(**kwargs)\n self.inputs = tree.flatten(inputs)\n self.outputs = tree.flatten(outputs)\n self._node_to_id = {}\n self._nodes = []\n self.blocks = []\n self._block_to_id = {}\n if inputs and outputs:\n self._build_network()\n\n # Temporary attributes\n self.epochs = None\n self.num_samples = None\n\n def _build_network(self):\n self._node_to_id = {}\n\n # Recursively find all the interested nodes.\n for input_node in self.inputs:\n self._search_network(input_node, self.outputs, set(), set())\n self._nodes = sorted(\n list(self._node_to_id.keys()), key=lambda x: self._node_to_id[x]\n )\n\n for node in self.inputs + self.outputs:\n if node not in self._node_to_id:\n raise ValueError(\"Inputs and outputs not connected.\")\n\n # Find the blocks.\n blocks = []\n for input_node in self._nodes:\n for block in input_node.out_blocks:\n if (\n any(\n [\n output_node in self._node_to_id\n for output_node in block.outputs\n ]\n )\n and block not in blocks\n ):\n blocks.append(block)\n\n # Check if all the inputs of the blocks are set as inputs.\n for block in blocks:\n for input_node in block.inputs:\n if input_node not in self._node_to_id:\n raise ValueError(\n \"A required input is missing for HyperModel \"\n \"{name}.\".format(name=block.name)\n )\n\n # Calculate the in degree of all the nodes\n in_degree = [0] * len(self._nodes)\n for node_id, node in enumerate(self._nodes):\n in_degree[node_id] = len(\n [block for block in node.in_blocks if block in blocks]\n )\n\n # Add the blocks in topological order.\n self.blocks = []\n self._block_to_id = {}\n while len(blocks) != 0:\n new_added = []\n\n # Collect blocks with in degree 0.\n for block in blocks:\n if any(\n [in_degree[self._node_to_id[node]] for node in block.inputs]\n ):\n continue\n new_added.append(block)\n\n # Remove the collected blocks from blocks.\n for block in new_added:\n blocks.remove(block)\n\n for block in new_added:\n # Add the collected blocks to the Graph.\n self._add_block(block)\n\n # Decrease the in degree of the output nodes.\n for output_node in block.outputs:\n output_node_id = self._node_to_id[output_node]\n in_degree[output_node_id] -= 1\n\n def _search_network(\n self, input_node, outputs, in_stack_nodes, visited_nodes\n ):\n visited_nodes.add(input_node)\n in_stack_nodes.add(input_node)\n\n outputs_reached = False\n if input_node in outputs:\n outputs_reached = True\n\n for block in input_node.out_blocks:\n for output_node in block.outputs:\n if output_node in in_stack_nodes:\n raise ValueError(\"The network has a cycle.\")\n if output_node not in visited_nodes:\n self._search_network(\n output_node, outputs, in_stack_nodes, visited_nodes\n )\n if output_node in self._node_to_id.keys():\n outputs_reached = True\n\n if outputs_reached:\n self._add_node(input_node)\n\n in_stack_nodes.remove(input_node)\n\n def _add_block(self, block):\n if block not in self.blocks:\n block_id = len(self.blocks)\n self._block_to_id[block] = block_id\n self.blocks.append(block)\n\n def _add_node(self, input_node):\n if input_node not in self._node_to_id:\n self._node_to_id[input_node] = len(self._node_to_id)\n\n def get_config(self):\n blocks = [blocks_module.serialize(block) for block in self.blocks]\n nodes = {\n str(self._node_to_id[node]): nodes_module.serialize(node)\n for node in self.inputs\n }\n block_inputs = {\n str(block_id): [self._node_to_id[node] for node in block.inputs]\n for block_id, block in enumerate(self.blocks)\n }\n block_outputs = {\n str(block_id): [self._node_to_id[node] for node in block.outputs]\n for block_id, block in enumerate(self.blocks)\n }\n\n outputs = [self._node_to_id[node] for node in self.outputs]\n\n return {\n \"blocks\": blocks, # Dict {id: serialized}.\n \"nodes\": nodes, # Dict {id: serialized}.\n \"outputs\": outputs, # List of node_ids.\n \"block_inputs\": block_inputs, # Dict {id: List of node_ids}.\n \"block_outputs\": block_outputs, # Dict {id: List of node_ids}.\n }\n\n @classmethod\n def from_config(cls, config):\n blocks = [\n blocks_module.deserialize(block) for block in config[\"blocks\"]\n ]\n nodes = {\n int(node_id): nodes_module.deserialize(node)\n for node_id, node in config[\"nodes\"].items()\n }\n\n inputs = [nodes[node_id] for node_id in nodes]\n for block_id, block in enumerate(blocks):\n input_nodes = [\n nodes[node_id]\n for node_id in config[\"block_inputs\"][str(block_id)]\n ]\n output_nodes = tree.flatten(block(input_nodes))\n for output_node, node_id in zip(\n output_nodes, config[\"block_outputs\"][str(block_id)]\n ):\n nodes[node_id] = output_node\n\n outputs = [nodes[node_id] for node_id in config[\"outputs\"]]\n return cls(inputs=inputs, outputs=outputs)\n\n def compile(self):\n \"\"\"Share the information between blocks.\"\"\"\n for block in self.blocks:\n for func in COMPILE_FUNCTIONS.get(block.__class__, []):\n func(block)\n\n def build(self, hp):\n \"\"\"Build the HyperModel into a Keras Model.\"\"\"\n self.compile()\n keras_nodes = {}\n keras_input_nodes = []\n for node in self.inputs:\n node_id = self._node_to_id[node]\n input_node = node.build_node(hp)\n output_node = node.build(hp, input_node)\n keras_input_nodes.append(input_node)\n keras_nodes[node_id] = output_node\n for block in self.blocks:\n temp_inputs = [\n keras_nodes[self._node_to_id[input_node]]\n for input_node in block.inputs\n ]\n outputs = block.build(hp, inputs=temp_inputs)\n outputs = tree.flatten(outputs)\n for output_node, real_output_node in zip(block.outputs, outputs):\n keras_nodes[self._node_to_id[output_node]] = real_output_node\n model = keras.Model(\n keras_input_nodes,\n [\n keras_nodes[self._node_to_id[output_node]]\n for output_node in self.outputs\n ],\n )\n\n return self._compile_keras_model(hp, model)\n\n def _get_metrics(self):\n metrics = {}\n for output_node in self.outputs:\n block = output_node.in_blocks[0]\n if isinstance(block, head_module.Head):\n metrics[block.name] = block.metrics\n return metrics\n\n def _get_loss(self):\n loss = {}\n for output_node in self.outputs:\n block = output_node.in_blocks[0]\n if isinstance(block, head_module.Head):\n loss[block.name] = block.loss\n return loss\n\n def _compile_keras_model(self, hp, model):\n # Specify hyperparameters from compile(...)\n optimizer_name = hp.Choice(\n \"optimizer\",\n [\"adam\", \"sgd\", \"adam_weight_decay\"],\n default=\"adam\",\n )\n # TODO: add adadelta optimizer when it can optimize embedding layer on\n # GPU.\n learning_rate = hp.Choice(\n \"learning_rate\", [1e-1, 1e-2, 1e-3, 1e-4, 2e-5, 1e-5], default=1e-3\n )\n\n if optimizer_name == \"adam\":\n optimizer = keras.optimizers.Adam(learning_rate=learning_rate)\n elif optimizer_name == \"sgd\":\n optimizer = keras.optimizers.SGD(learning_rate=learning_rate)\n elif optimizer_name == \"adam_weight_decay\":\n steps_per_epoch = max(\n 1, int(self.num_samples / (self.batch_size or 32))\n )\n num_train_steps = steps_per_epoch * self.epochs\n\n lr_schedule = keras.optimizers.schedules.PolynomialDecay(\n initial_learning_rate=learning_rate,\n decay_steps=num_train_steps,\n end_learning_rate=0.0,\n )\n\n optimizer = keras.optimizers.AdamW(\n learning_rate=lr_schedule,\n weight_decay=0.01,\n beta_1=0.9,\n beta_2=0.999,\n epsilon=1e-6,\n )\n\n model.compile(\n optimizer=optimizer,\n metrics=self._get_metrics(),\n loss=self._get_loss(),\n )\n\n return model\n\n def save(self, filepath):\n io_utils.save_json(filepath, self.get_config())\n\n def set_io_shapes(self, shapes):\n for node, shape in zip(self.inputs, tree.flatten(shapes[0])):\n node.shape = tuple(shape[1:])\n for node, shape in zip(self.outputs, tree.flatten(shapes[1])):\n node.in_blocks[0].shape = tuple(shape[1:])\n\n def set_fit_args(self, validation_split, epochs=None):\n self.epochs = epochs\n # Epochs not specified by the user\n if self.epochs is None:\n self.epochs = 1\n validation_split = validation_split or 0\n # num_samples from analysers are before split\n self.num_samples = self.inputs[0].num_samples * (1 - validation_split)\n\n @property\n def batch_size(self):\n return self.inputs[0].batch_size\n" }, { "path": "autokeras/graph_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\n\nimport keras_tuner\nimport pytest\n\nimport autokeras as ak\nfrom autokeras import graph as graph_module\n\n\ndef test_input_output_disconnect():\n input_node1 = ak.Input()\n output_node = input_node1\n _ = ak.DenseBlock()(output_node)\n\n input_node = ak.Input()\n output_node = input_node\n output_node = ak.DenseBlock()(output_node)\n output_node = ak.RegressionHead()(output_node)\n\n with pytest.raises(ValueError) as info:\n graph_module.Graph(inputs=input_node1, outputs=output_node)\n assert \"Inputs and outputs not connected.\" in str(info.value)\n\n\ndef test_hyper_graph_cycle():\n input_node1 = ak.Input()\n input_node2 = ak.Input()\n output_node1 = ak.DenseBlock()(input_node1)\n output_node2 = ak.DenseBlock()(input_node2)\n output_node = ak.Merge()([output_node1, output_node2])\n head = ak.RegressionHead()\n output_node = head(output_node)\n head.outputs = output_node1\n\n with pytest.raises(ValueError) as info:\n graph_module.Graph(\n inputs=[input_node1, input_node2], outputs=output_node\n )\n assert \"The network has a cycle.\" in str(info.value)\n\n\ndef test_input_missing():\n input_node1 = ak.Input()\n input_node2 = ak.Input()\n output_node1 = ak.DenseBlock()(input_node1)\n output_node2 = ak.DenseBlock()(input_node2)\n output_node = ak.Merge()([output_node1, output_node2])\n output_node = ak.RegressionHead()(output_node)\n\n with pytest.raises(ValueError) as info:\n graph_module.Graph(inputs=input_node1, outputs=output_node)\n assert \"A required input is missing for HyperModel\" in str(info.value)\n\n\ndef test_graph_basics():\n input_node = ak.Input(shape=(30,))\n output_node = input_node\n output_node = ak.DenseBlock()(output_node)\n output_node = ak.RegressionHead(shape=(1,))(output_node)\n\n model = graph_module.Graph(inputs=input_node, outputs=output_node).build(\n keras_tuner.HyperParameters()\n )\n assert model.input_shape == (None, 30)\n assert model.output_shape == (None, 1)\n\n\ndef test_adamw_optimizer():\n input_node = ak.Input(shape=(30,))\n output_node = input_node\n output_node = ak.DenseBlock()(output_node)\n output_node = ak.RegressionHead(shape=(1,))(output_node)\n\n hp = keras_tuner.HyperParameters()\n hp.Choice(\"optimizer\", [\"adam\", \"sgd\", \"adam_weight_decay\"], default=\"adam\")\n hp.values[\"optimizer\"] = \"adam_weight_decay\"\n graph = graph_module.Graph(inputs=input_node, outputs=output_node)\n graph.inputs[0].num_samples = 100\n graph.inputs[0].batch_size = 32\n graph.epochs = 10\n graph.set_fit_args(0, epochs=10)\n model = graph.build(hp)\n assert model.input_shape == (None, 30)\n assert model.output_shape == (None, 1)\n\n\ndef test_graph_save_load(tmp_path):\n input1 = ak.Input()\n input2 = ak.Input()\n output1 = ak.DenseBlock()(input1)\n output2 = ak.ConvBlock()(input2)\n output = ak.Merge()([output1, output2])\n output1 = ak.RegressionHead()(output)\n output2 = ak.ClassificationHead()(output)\n\n graph = graph_module.Graph(\n inputs=[input1, input2],\n outputs=[output1, output2],\n )\n path = os.path.join(tmp_path, \"graph\")\n graph.save(path)\n graph = graph_module.load_graph(path)\n\n assert len(graph.inputs) == 2\n assert len(graph.outputs) == 2\n assert isinstance(graph.inputs[0].out_blocks[0], ak.DenseBlock)\n assert isinstance(graph.inputs[1].out_blocks[0], ak.ConvBlock)\n\n\ndef test_merge():\n input_node1 = ak.Input(shape=(30,))\n input_node2 = ak.Input(shape=(40,))\n output_node1 = ak.DenseBlock()(input_node1)\n output_node2 = ak.DenseBlock()(input_node2)\n output_node = ak.Merge()([output_node1, output_node2])\n output_node = ak.RegressionHead(shape=(1,))(output_node)\n\n model = graph_module.Graph(\n inputs=[input_node1, input_node2], outputs=output_node\n ).build(keras_tuner.HyperParameters())\n assert model.input_shape == [(None, 30), (None, 40)]\n assert model.output_shape == (None, 1)\n\n\ndef test_save_custom_metrics_loss(tmp_path):\n def custom_metric(y_pred, y_true):\n return 1\n\n def custom_loss(y_pred, y_true):\n return y_pred - y_true\n\n head = ak.ClassificationHead(\n loss=custom_loss, metrics=[\"accuracy\", custom_metric]\n )\n input_node = ak.Input()\n output_node = head(input_node)\n graph = graph_module.Graph(input_node, output_node)\n path = os.path.join(tmp_path, \"graph\")\n graph.save(path)\n new_graph = graph_module.load_graph(\n path,\n custom_objects={\n \"custom_metric\": custom_metric,\n \"custom_loss\": custom_loss,\n },\n )\n assert new_graph.blocks[0].metrics[1](0, 0) == 1\n assert new_graph.blocks[0].loss(3, 2) == 1\n\n\ndef test_graph_can_init_with_one_missing_output():\n input_node = ak.ImageInput()\n output_node = ak.ConvBlock()(input_node)\n output_node = ak.RegressionHead()(output_node)\n ak.ClassificationHead()(output_node)\n\n graph_module.Graph(input_node, output_node)\n\n\ndef test_set_fit_args_with_none_validation_split():\n input_node = ak.Input(shape=(30,))\n output_node = input_node\n output_node = ak.DenseBlock()(output_node)\n output_node = ak.RegressionHead(shape=(1,))(output_node)\n\n graph = graph_module.Graph(inputs=input_node, outputs=output_node)\n graph.inputs[0].num_samples = 100\n graph.inputs[0].batch_size = 32\n graph.set_fit_args(None, epochs=1)\n assert graph.num_samples == 100 # Should handle None as 0\n" }, { "path": "autokeras/hyper_preprocessors.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport keras\n\nfrom autokeras import preprocessors\nfrom autokeras.engine import hyper_preprocessor\nfrom autokeras.utils import utils\n\n\ndef serialize(encoder):\n return utils.serialize_keras_object(encoder)\n\n\ndef deserialize(config, custom_objects=None):\n return utils.deserialize_keras_object(\n config,\n module_objects=globals(),\n custom_objects=custom_objects,\n )\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass DefaultHyperPreprocessor(hyper_preprocessor.HyperPreprocessor):\n \"\"\"HyperPreprocessor without Hyperparameters to tune.\n\n It would always return the same preprocessor. No hyperparameters to be\n tuned.\n\n # Arguments\n preprocessor: The Preprocessor to return when calling build.\n \"\"\"\n\n def __init__(self, preprocessor, *args, **kwargs):\n super().__init__(*args, **kwargs)\n self.preprocessor = preprocessor\n\n def build(self, hp, dataset):\n return self.preprocessor\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\"preprocessor\": preprocessors.serialize(self.preprocessor)}\n )\n return config\n\n @classmethod\n def from_config(cls, config):\n config[\"preprocessor\"] = preprocessors.deserialize(\n config[\"preprocessor\"]\n )\n return super().from_config(config)\n" }, { "path": "autokeras/hyper_preprocessors_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras import hyper_preprocessors\nfrom autokeras import preprocessors\n\n\ndef test_serialize_and_deserialize_default_hpps():\n preprocessor = preprocessors.AddOneDimension()\n hyper_preprocessor = hyper_preprocessors.DefaultHyperPreprocessor(\n preprocessor\n )\n hyper_preprocessor = hyper_preprocessors.deserialize(\n hyper_preprocessors.serialize(hyper_preprocessor)\n )\n assert isinstance(\n hyper_preprocessor.preprocessor, preprocessors.AddOneDimension\n )\n\n\ndef test_serialize_and_deserialize_default_hpps_categorical():\n x_train = np.array([[\"a\", \"ab\", 2.1], [\"b\", \"bc\", 1.0], [\"a\", \"bc\", \"nan\"]])\n preprocessor = preprocessors.CategoricalToNumerical(\n column_names=[\"column_a\", \"column_b\", \"column_c\"],\n column_types={\n \"column_a\": \"categorical\",\n \"column_b\": \"categorical\",\n \"column_c\": \"numerical\",\n },\n )\n\n hyper_preprocessor = hyper_preprocessors.DefaultHyperPreprocessor(\n preprocessor\n )\n hyper_preprocessor.preprocessor.fit(x_train)\n hyper_preprocessor = hyper_preprocessors.deserialize(\n hyper_preprocessors.serialize(hyper_preprocessor)\n )\n assert isinstance(\n hyper_preprocessor.preprocessor,\n preprocessors.CategoricalToNumerical,\n )\n\n results = hyper_preprocessor.preprocessor.transform(x_train)\n\n assert results[0][0] == results[2][0]\n assert results[0][0] != results[1][0]\n assert results[0][1] != results[1][1]\n assert results[0][1] != results[2][1]\n assert results[2][2] == 0\n assert results.dtype == \"float32\"\n" }, { "path": "autokeras/integration_tests/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/integration_tests/functional_api_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport pandas as pd\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\ndef test_text_and_structured_data(tmp_path):\n # Prepare the data.\n num_instances = 3\n x_text = test_utils.generate_text_data(num_instances)\n\n x_structured_data = pd.read_csv(test_utils.TRAIN_CSV_PATH)\n\n x_structured_data = x_structured_data.values\n\n x_structured_data = x_structured_data[:num_instances]\n y_classification = test_utils.generate_one_hot_labels(\n num_instances=num_instances, num_classes=3\n )\n y_regression = test_utils.generate_data(\n num_instances=num_instances, shape=(1,)\n )\n\n # Build model and train.\n structured_data_input = ak.StructuredDataInput()\n structured_data_output = ak.DenseBlock()(structured_data_input)\n\n text_input = ak.TextInput()\n text_output = ak.TextBlock()(text_input)\n merged_outputs = ak.Merge()((structured_data_output, text_output))\n\n regression_outputs = ak.RegressionHead()(merged_outputs)\n classification_outputs = ak.ClassificationHead()(merged_outputs)\n automodel = ak.AutoModel(\n inputs=[text_input, structured_data_input],\n directory=tmp_path,\n outputs=[regression_outputs, classification_outputs],\n max_trials=2,\n tuner=ak.Hyperband,\n seed=test_utils.SEED,\n )\n\n automodel.fit(\n (x_text, x_structured_data),\n (y_regression, y_classification),\n validation_split=0.2,\n epochs=1,\n batch_size=2,\n )\n\n\ndef test_image_blocks(tmp_path):\n num_instances = 3\n x_train = test_utils.generate_data(\n num_instances=num_instances, shape=(28, 28)\n )\n y_train = np.random.randint(0, 10, num_instances)\n\n input_node = ak.ImageInput()\n output = ak.Normalization()(input_node)\n output = ak.ImageAugmentation()(output)\n outputs1 = ak.ResNetBlock(version=\"v2\")(output)\n outputs2 = ak.XceptionBlock()(output)\n output_node = ak.Merge()((outputs1, outputs2))\n output_node = ak.ClassificationHead()(output_node)\n\n automodel = ak.AutoModel(\n inputs=input_node,\n outputs=output_node,\n directory=tmp_path,\n max_trials=1,\n seed=test_utils.SEED,\n )\n\n automodel.fit(\n x_train,\n y_train,\n validation_data=(x_train, y_train),\n epochs=1,\n batch_size=2,\n )\n" }, { "path": "autokeras/integration_tests/io_api_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport pandas as pd\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\ndef test_io_api(tmp_path):\n num_instances = 3\n image_x = test_utils.generate_data(\n num_instances=num_instances, shape=(28, 28)\n )\n text_x = test_utils.generate_text_data(num_instances=num_instances)\n\n image_x = image_x[:num_instances]\n structured_data_x = (\n pd.read_csv(test_utils.TRAIN_CSV_PATH)\n .to_numpy()\n .astype(str)[:num_instances]\n )\n classification_y = test_utils.generate_one_hot_labels(\n num_instances=num_instances, num_classes=3\n )\n regression_y = test_utils.generate_data(\n num_instances=num_instances, shape=(1,)\n )\n\n # Build model and train.\n automodel = ak.AutoModel(\n inputs=[ak.ImageInput(), ak.TextInput(), ak.StructuredDataInput()],\n outputs=[\n ak.RegressionHead(metrics=[\"mae\"]),\n ak.ClassificationHead(\n loss=\"categorical_crossentropy\", metrics=[\"accuracy\"]\n ),\n ],\n directory=tmp_path,\n max_trials=2,\n tuner=ak.RandomSearch,\n seed=test_utils.SEED,\n )\n automodel.fit(\n [image_x, text_x, structured_data_x],\n [regression_y, classification_y],\n epochs=1,\n validation_split=0.2,\n batch_size=2,\n )\n automodel.predict([image_x, text_x, structured_data_x])\n" }, { "path": "autokeras/integration_tests/task_api_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport numpy as np\nimport pandas as pd\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\nNUM_INSTANCES = 3\nBATCH_SIZE = 2\n\n\ndef test_image_classifier(tmp_path):\n train_x = test_utils.generate_data(\n num_instances=NUM_INSTANCES, shape=(32, 32)\n )\n train_y = test_utils.generate_one_hot_labels(\n num_instances=NUM_INSTANCES, num_classes=10\n )\n clf = ak.ImageClassifier(\n directory=tmp_path,\n max_trials=2,\n seed=test_utils.SEED,\n )\n clf.fit(\n train_x, train_y, epochs=1, validation_split=0.2, batch_size=BATCH_SIZE\n )\n keras_model = clf.export_model()\n clf.evaluate(train_x, train_y)\n assert clf.predict(train_x).shape == (len(train_x), 10)\n assert isinstance(keras_model, keras.Model)\n\n\ndef test_image_regressor(tmp_path):\n train_x = test_utils.generate_data(\n num_instances=NUM_INSTANCES, shape=(32, 32, 3)\n )\n train_y = test_utils.generate_data(num_instances=NUM_INSTANCES, shape=(1,))\n clf = ak.ImageRegressor(\n directory=tmp_path, max_trials=2, seed=test_utils.SEED\n )\n clf.fit(\n train_x, train_y, epochs=1, validation_split=0.2, batch_size=BATCH_SIZE\n )\n clf.export_model()\n assert clf.predict(train_x).shape == (len(train_x), 1)\n\n\ndef test_text_classifier(tmp_path):\n train_x = test_utils.generate_text_data(num_instances=NUM_INSTANCES)\n train_y = np.array([0, 1] * ((NUM_INSTANCES + 1) // 2))[:NUM_INSTANCES]\n test_x = train_x\n test_y = train_y\n clf = ak.TextClassifier(\n directory=tmp_path,\n max_trials=2,\n seed=test_utils.SEED,\n metrics=[\"accuracy\"],\n objective=\"accuracy\",\n )\n clf.fit(\n train_x,\n train_y,\n epochs=1,\n validation_data=(test_x, test_y),\n batch_size=BATCH_SIZE,\n )\n clf.export_model()\n assert clf.predict(test_x).shape == (len(test_x), 1)\n assert clf.tuner._get_best_trial_epochs() <= 2\n\n\ndef test_text_regressor(tmp_path):\n train_x = test_utils.generate_text_data(num_instances=NUM_INSTANCES)\n test_x = train_x\n train_y = test_utils.generate_data(num_instances=NUM_INSTANCES, shape=(1,))\n test_y = train_y\n clf = ak.TextRegressor(\n directory=tmp_path, max_trials=2, seed=test_utils.SEED\n )\n clf.fit(\n train_x,\n train_y,\n epochs=1,\n validation_data=(test_x, test_y),\n batch_size=BATCH_SIZE,\n )\n clf.predict(test_x)\n clf.export_model()\n assert clf.predict(test_x).shape == (len(test_x), 1)\n\n\ndef test_structured_data_regressor(tmp_path):\n num_data = NUM_INSTANCES * 2\n num_train = NUM_INSTANCES\n data = (\n pd.read_csv(test_utils.TRAIN_CSV_PATH).to_numpy().astype(str)[:num_data]\n )\n x_train, x_test = data[:num_train], data[num_train:]\n y = test_utils.generate_data(num_instances=num_data, shape=tuple())\n y_train, y_test = y[:num_train], y[num_train:]\n clf = ak.StructuredDataRegressor(\n directory=tmp_path, max_trials=2, seed=test_utils.SEED\n )\n clf.fit(\n x_train,\n y_train,\n epochs=11,\n validation_data=(x_train, y_train),\n batch_size=BATCH_SIZE,\n )\n clf.export_model()\n assert clf.predict(x_test).shape == (len(y_test), 1)\n\n\ndef test_structured_data_classifier(tmp_path):\n num_data = NUM_INSTANCES * 2\n num_train = NUM_INSTANCES\n data = (\n pd.read_csv(test_utils.TRAIN_CSV_PATH).to_numpy().astype(str)[:num_data]\n )\n x_train, x_test = data[:num_train], data[num_train:]\n y = test_utils.generate_one_hot_labels(\n num_instances=num_data, num_classes=3\n )\n y_train, y_test = y[:num_train], y[num_train:]\n clf = ak.StructuredDataClassifier(\n directory=tmp_path, max_trials=1, seed=test_utils.SEED\n )\n clf.fit(\n x_train,\n y_train,\n epochs=2,\n validation_data=(x_train, y_train),\n batch_size=BATCH_SIZE,\n )\n clf.export_model()\n assert clf.predict(x_test).shape == (len(y_test), 3)\n" }, { "path": "autokeras/keras_layers.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport keras\nfrom keras import layers\nfrom keras import ops\n\nfrom autokeras.utils import data_utils\n\nINT = \"int\"\nNONE = \"none\"\nONE_HOT = \"one-hot\"\n\n\nclass PreprocessingLayer(layers.Layer):\n pass\n\n\n@keras.utils.register_keras_serializable()\nclass CastToFloat32(PreprocessingLayer):\n def get_config(self):\n return super().get_config()\n\n def call(self, inputs):\n # Does not and needs not handle strings.\n return data_utils.cast_to_float32(inputs)\n\n def adapt(self, data):\n return\n\n\n@keras.utils.register_keras_serializable()\nclass ExpandLastDim(PreprocessingLayer):\n def get_config(self):\n return super().get_config()\n\n def call(self, inputs):\n return ops.expand_dims(inputs, axis=-1)\n\n def adapt(self, data):\n return\n" }, { "path": "autokeras/keras_layers_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport numpy as np\n\nfrom autokeras import keras_layers as layer_module\n\n\ndef get_text_data():\n train = np.array(\n [\n [\"This is a test example\"],\n [\"This is another text example\"],\n [\"Is this another example?\"],\n [\"\"],\n [\"Is this a long long long long long long example?\"],\n ],\n dtype=str,\n )\n test = np.array(\n [\n [\"This is a test example\"],\n [\"This is another text example\"],\n [\"Is this another example?\"],\n ],\n dtype=str,\n )\n y = np.random.rand(3, 1)\n return train, test, y\n\n\ndef test_cast_to_float32_return_float32_tensor(tmp_path):\n layer = layer_module.CastToFloat32()\n\n tensor = layer(np.array([3], dtype=\"uint8\"))\n\n assert \"float32\" == tensor.numpy().dtype\n\n\ndef test_expand_last_dim_return_tensor_with_more_dims(tmp_path):\n layer = layer_module.ExpandLastDim()\n\n tensor = layer(np.array([0.1, 0.2], dtype=\"float32\"))\n\n assert 2 == len(tensor.shape)\n" }, { "path": "autokeras/nodes.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Dict\nfrom typing import List\nfrom typing import Optional\n\nimport keras\nimport tree\n\nfrom autokeras import adapters\nfrom autokeras import analysers\nfrom autokeras import blocks\nfrom autokeras import hyper_preprocessors\nfrom autokeras import keras_layers\nfrom autokeras import preprocessors\nfrom autokeras.engine import io_hypermodel\nfrom autokeras.engine import node as node_module\nfrom autokeras.utils import utils\n\n\ndef serialize(obj):\n return utils.serialize_keras_object(obj)\n\n\ndef deserialize(config, custom_objects=None):\n return utils.deserialize_keras_object(\n config,\n module_objects=globals(),\n custom_objects=custom_objects,\n )\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Input(node_module.Node, io_hypermodel.IOHyperModel):\n \"\"\"Input node for tensor data.\n\n The data should be numpy.ndarray.\n\n # Arguments\n name: String. The name of the input node. If unspecified, it will be set\n automatically with the class name.\n \"\"\"\n\n def __init__(self, name: Optional[str] = None, **kwargs):\n super().__init__(name=name, **kwargs)\n\n def build_node(self, hp):\n return keras.Input(shape=self.shape, dtype=self.dtype)\n\n def build(self, hp, inputs=None):\n input_node = tree.flatten(inputs)[0]\n return keras_layers.CastToFloat32()(input_node)\n\n def get_adapter(self):\n return adapters.InputAdapter()\n\n def get_analyser(self):\n return analysers.InputAnalyser()\n\n def get_block(self):\n return blocks.GeneralBlock()\n\n def get_hyper_preprocessors(self):\n return []\n\n\nclass ImageInput(Input):\n \"\"\"Input node for image data.\n\n The input data should be numpy.ndarray. The shape of the\n data should be should be (samples, width, height) or (samples, width,\n height, channels).\n\n # Arguments\n name: String. The name of the input node. If unspecified, it will be set\n automatically with the class name.\n \"\"\"\n\n def __init__(self, name: Optional[str] = None, **kwargs):\n super().__init__(name=name, **kwargs)\n\n def build(self, hp, inputs=None):\n inputs = super().build(hp, inputs)\n output_node = tree.flatten(inputs)[0]\n if len(output_node.shape) == 3:\n output_node = keras_layers.ExpandLastDim()(output_node)\n return output_node\n\n def get_adapter(self):\n return adapters.ImageAdapter()\n\n def get_analyser(self):\n return analysers.ImageAnalyser()\n\n def get_block(self):\n return blocks.ImageBlock()\n\n\nclass TextInput(Input):\n \"\"\"Input node for text data.\n\n The input data should be numpy.ndarray. The data should be\n one-dimensional. Each element in the data should be a string which is a\n full sentence.\n\n # Arguments\n name: String. The name of the input node. If unspecified, it will be set\n automatically with the class name.\n \"\"\"\n\n def __init__(self, name: Optional[str] = None, **kwargs):\n super().__init__(name=name, **kwargs)\n\n def build_node(self, hp):\n return keras.Input(shape=self.shape, dtype=\"int32\")\n\n def build(self, hp, inputs=None):\n output_node = tree.flatten(inputs)[0]\n if len(output_node.shape) == 1:\n output_node = keras_layers.ExpandLastDim()( # pragma: no cover\n output_node\n )\n return output_node\n\n def get_adapter(self):\n return adapters.TextAdapter()\n\n def get_analyser(self):\n return analysers.TextAnalyser()\n\n def get_block(self):\n return blocks.TextBlock()\n\n def get_hyper_preprocessors(self):\n return [\n hyper_preprocessors.DefaultHyperPreprocessor(\n preprocessors.CastToString()\n ),\n hyper_preprocessors.DefaultHyperPreprocessor(\n preprocessors.TextTokenizer()\n ),\n ]\n\n\nclass StructuredDataInput(Input):\n \"\"\"Input node for structured data.\n\n The input data should be numpy.ndarray. The data should be two-dimensional\n with numerical or categorical values.\n\n # Arguments\n column_names: A list of strings specifying the names of the columns. The\n length of the list should be equal to the number of columns of the\n data. Defaults to None.\n column_types: Dict. The keys are the column names. The values should\n either be 'numerical' or 'categorical', indicating the type of that\n column. Defaults to None. If not None, the column_names need to be\n specified. If None, it will be inferred from the data. A column will\n be judged as categorical if the number of different values is less\n than 5% of the number of instances.\n name: String. The name of the input node. If unspecified, it will be set\n automatically with the class name.\n \"\"\"\n\n def __init__(\n self,\n column_names: Optional[List[str]] = None,\n column_types: Optional[Dict[str, str]] = None,\n name: Optional[str] = None,\n **kwargs\n ):\n super().__init__(name=name, **kwargs)\n self.column_names = column_names\n self.column_types = column_types\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"column_names\": self.column_names,\n \"column_types\": self.column_types,\n }\n )\n return config\n\n def get_adapter(self):\n return adapters.StructuredDataAdapter()\n\n def get_analyser(self):\n return analysers.StructuredDataAnalyser(\n self.column_names, self.column_types\n )\n\n def get_block(self):\n return blocks.StructuredDataBlock()\n\n def config_from_analyser(self, analyser):\n super().config_from_analyser(analyser)\n self.column_names = analyser.column_names\n # Analyser keeps the specified ones and infer the missing ones.\n self.column_types = analyser.column_types\n\n def build(self, hp, inputs=None):\n return inputs\n\n def get_hyper_preprocessors(self):\n return [\n hyper_preprocessors.DefaultHyperPreprocessor(\n preprocessors.CategoricalToNumerical(\n column_names=self.column_names,\n column_types=self.column_types,\n )\n )\n ]\n" }, { "path": "autokeras/nodes_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras import blocks\nfrom autokeras import nodes\n\n\ndef test_input_get_block_return_general_block():\n input_node = nodes.Input()\n assert isinstance(input_node.get_block(), blocks.GeneralBlock)\n" }, { "path": "autokeras/pipeline.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport tree\n\nfrom autokeras import preprocessors as preprocessors_module\nfrom autokeras.engine import hyper_preprocessor as hpps_module\nfrom autokeras.engine import preprocessor as pps_module\nfrom autokeras.utils import io_utils\n\n\nclass HyperPipeline(hpps_module.HyperPreprocessor):\n \"\"\"A search space consists of HyperPreprocessors.\n\n # Arguments\n inputs: a list of lists of HyperPreprocessors.\n outputs: a list of lists of HyperPreprocessors.\n \"\"\"\n\n def __init__(self, inputs, outputs, **kwargs):\n super().__init__(**kwargs)\n self.inputs = inputs\n self.outputs = outputs\n\n @staticmethod\n def _build_preprocessors(hp, hpps_lists, dataset):\n sources = tree.flatten(dataset)\n preprocessors_list = []\n for source, hpps_list in zip(sources, hpps_lists):\n data = source\n preprocessors = []\n for hyper_preprocessor in hpps_list:\n preprocessor = hyper_preprocessor.build(hp, data)\n preprocessor.fit(data)\n data = preprocessor.transform(data)\n preprocessors.append(preprocessor)\n preprocessors_list.append(preprocessors)\n return preprocessors_list\n\n def build(self, hp, dataset):\n \"\"\"Build a Pipeline by Hyperparameters.\n\n # Arguments\n hp: Hyperparameters.\n dataset: nested numpy arrays. The input dataset for the model.\n\n # Returns\n An instance of Pipeline.\n \"\"\"\n x, y = dataset\n return Pipeline(\n inputs=self._build_preprocessors(hp, self.inputs, x),\n outputs=self._build_preprocessors(hp, self.outputs, y),\n )\n\n\ndef load_pipeline(filepath, custom_objects=None):\n \"\"\"Load a Pipeline instance from disk.\"\"\"\n if custom_objects is None:\n custom_objects = {}\n with keras.utils.custom_object_scope(custom_objects):\n return Pipeline.from_config(io_utils.load_json(filepath))\n\n\nclass Pipeline(pps_module.Preprocessor):\n \"\"\"A data pipeline for transform the entire dataset.\n\n # Arguments\n inputs: A list of lists of Preprocessors. For the input datasets for\n the model.\n outputs: A list of lists of Preprocessors. For the target datasets for\n the model.\n \"\"\"\n\n def __init__(self, inputs, outputs, **kwargs):\n super().__init__(**kwargs)\n self.inputs = inputs\n self.outputs = outputs\n\n def fit(self, dataset):\n \"\"\"Fit the Preprocessors.\"\"\"\n x, y = dataset\n sources_x = tree.flatten(x)\n for pps_list, source in zip(self.inputs, sources_x):\n for preprocessor in pps_list:\n preprocessor.fit(source) # pragma: no cover\n source = preprocessor.transform(source) # pragma: no cover\n sources_y = tree.flatten(y)\n for pps_list, source in zip(self.outputs, sources_y):\n for preprocessor in pps_list:\n preprocessor.fit(source)\n source = preprocessor.transform(source)\n return\n\n def transform(self, dataset):\n \"\"\"Transform the dataset to be ready for the model.\n\n # Arguments\n dataset: nested numpy arrays. The input dataset for the model.\n\n # Returns\n dataset: nested numpy arrays. The input dataset for the model.\n \"\"\"\n x, y = dataset\n x = self.transform_x(x)\n y = self.transform_y(y)\n return (x, y)\n\n def transform_x(self, dataset):\n \"\"\"Transform the input dataset for the model.\n\n # Arguments\n dataset: nested numpy arrays. The input dataset for the model.\n\n # Returns\n dataset: nested numpy arrays. The input dataset for the model.\n \"\"\"\n return self._transform_data(dataset, self.inputs)\n\n def transform_y(self, dataset):\n \"\"\"Transform the target dataset for the model.\n\n # Arguments\n dataset: nested numpy arrays. The target dataset for the model.\n\n # Returns\n dataset: nested numpy arrays. The input dataset for the model.\n \"\"\"\n return self._transform_data(dataset, self.outputs)\n\n def _transform_data(self, dataset, pps_lists):\n sources = tree.flatten(dataset)\n transformed = []\n for pps_list, y in zip(pps_lists, sources):\n for preprocessor in pps_list:\n y = preprocessor.transform(y)\n transformed.append(y)\n if len(transformed) == 1:\n return transformed[0]\n return tuple(transformed)\n\n def save(self, filepath):\n io_utils.save_json(filepath, self.get_config())\n\n def get_config(self):\n return {\n \"inputs\": [\n [\n preprocessors_module.serialize(preprocessor)\n for preprocessor in preprocessors\n ]\n for preprocessors in self.inputs\n ],\n \"outputs\": [\n [\n preprocessors_module.serialize(preprocessor)\n for preprocessor in preprocessors\n ]\n for preprocessors in self.outputs\n ],\n }\n\n @classmethod\n def from_config(cls, config):\n return cls(\n inputs=[\n [\n preprocessors_module.deserialize(preprocessor)\n for preprocessor in preprocessors\n ]\n for preprocessors in config[\"inputs\"]\n ],\n outputs=[\n [\n preprocessors_module.deserialize(preprocessor)\n for preprocessor in preprocessors\n ]\n for preprocessors in config[\"outputs\"]\n ],\n )\n\n def postprocess(self, y):\n \"\"\"Postprocess the outputs of the model.\n\n # Arguments\n y: numpy.ndarray or a list of numpy.ndarrays. The output of the\n Keras model.\n\n # Returns\n A list or an instance of numpy.ndarray. The postprocessed data for\n the heads.\n \"\"\"\n outputs = []\n for source, preprocessors in zip(tree.flatten(y), self.outputs):\n for preprocessor in preprocessors[::-1]:\n if isinstance(preprocessor, pps_module.TargetPreprocessor):\n source = preprocessor.postprocess(source)\n outputs.append(source)\n if len(outputs) == 1:\n return outputs[0]\n return outputs\n" }, { "path": "autokeras/pipeline_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras import pipeline as pipeline_module\nfrom autokeras import preprocessors\n\n\ndef test_pipeline_postprocess_one_hot_to_labels():\n pipeline = pipeline_module.Pipeline(\n inputs=[[]], outputs=[[preprocessors.OneHotEncoder([\"a\", \"b\", \"c\"])]]\n )\n assert np.array_equal(\n pipeline.postprocess(np.eye(3)), [[\"a\"], [\"b\"], [\"c\"]]\n )\n\n\ndef test_pipeline_postprocess_multiple_one_hot_to_labels():\n pipeline = pipeline_module.Pipeline(\n inputs=[[]],\n outputs=[\n [preprocessors.OneHotEncoder([\"a\", \"b\", \"c\"])],\n [preprocessors.OneHotEncoder([\"a\", \"b\", \"c\"])],\n ],\n )\n result = pipeline.postprocess([np.eye(3), np.eye(3)])\n assert np.array_equal(result[0], [[\"a\"], [\"b\"], [\"c\"]])\n assert np.array_equal(result[1], [[\"a\"], [\"b\"], [\"c\"]])\n" }, { "path": "autokeras/preprocessors/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport keras\n\nfrom autokeras.preprocessors.common import AddOneDimension\nfrom autokeras.preprocessors.common import CastToInt32\nfrom autokeras.preprocessors.common import CastToString\nfrom autokeras.preprocessors.common import TextTokenizer\nfrom autokeras.preprocessors.encoders import CategoricalToNumerical\nfrom autokeras.preprocessors.encoders import LabelEncoder\nfrom autokeras.preprocessors.encoders import OneHotEncoder\nfrom autokeras.preprocessors.postprocessors import SigmoidPostprocessor\nfrom autokeras.preprocessors.postprocessors import SoftmaxPostprocessor\nfrom autokeras.utils import utils\n\n\ndef serialize(preprocessor):\n return utils.serialize_keras_object(preprocessor)\n\n\ndef deserialize(config, custom_objects=None):\n return utils.deserialize_keras_object(\n config,\n module_objects=globals(),\n custom_objects=custom_objects,\n )\n" }, { "path": "autokeras/preprocessors/common.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom collections import Counter\n\nimport keras\nimport numpy as np\n\nfrom autokeras.engine import preprocessor\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass AddOneDimension(preprocessor.Preprocessor):\n \"\"\"Append one dimension of size one to the dataset shape.\"\"\"\n\n def transform(self, dataset):\n return np.expand_dims(dataset, axis=-1)\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass CastToInt32(preprocessor.Preprocessor):\n \"\"\"Cast the dataset shape to int32.\"\"\"\n\n def transform(self, dataset):\n return dataset.astype(\"int32\")\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass CastToString(preprocessor.Preprocessor):\n \"\"\"Cast the dataset shape to string.\"\"\"\n\n def transform(self, dataset):\n if np.issubdtype(dataset.dtype, np.bytes_):\n return np.array(\n [x.decode(\"utf-8\", errors=\"ignore\") for x in dataset]\n )\n else:\n return dataset.astype(\"str\")\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass TextTokenizer(preprocessor.Preprocessor):\n \"\"\"Simple text tokenizer that converts strings to integer sequences.\"\"\"\n\n def __init__(self, max_len=100, vocab=None, max_vocab=500, **kwargs):\n super().__init__(**kwargs)\n self.max_len = max_len\n self.vocab = vocab\n self.max_vocab = max_vocab\n\n def fit(self, dataset):\n # Build vocab from unique words in the dataset\n unique_words = []\n for text in dataset:\n words = text.split()\n unique_words.extend(words)\n word_counts = Counter(unique_words)\n sorted_words = sorted(word_counts, key=word_counts.get, reverse=True)\n self.vocab = {\n word: idx + 1\n for idx, word in enumerate(sorted_words[: self.max_vocab])\n } # Start from 1, 0 for padding\n\n def transform(self, dataset):\n # dataset is np.array of strings\n sequences = []\n for text in dataset:\n words = text.split()[: self.max_len]\n seq = [self.vocab.get(word, 0) for word in words] # 0 for unknown\n # Pad with 0s\n seq += [0] * (self.max_len - len(seq))\n sequences.append(seq)\n return np.array(sequences, dtype=np.int32)\n\n def get_config(self):\n config = super().get_config()\n config.update(\n {\n \"max_len\": self.max_len,\n \"vocab\": self.vocab,\n \"max_vocab\": self.max_vocab,\n }\n )\n return config\n" }, { "path": "autokeras/preprocessors/common_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras import test_utils\nfrom autokeras.preprocessors import common\n\n\ndef test_cast_to_int32_return_int32():\n x = test_utils.generate_one_hot_labels(100, 10)\n x = x.astype(\"uint8\")\n x = common.CastToInt32().transform(x)\n assert x.dtype == \"int32\"\n\n\ndef test_cast_to_string_with_bytes():\n x = np.array([b\"hello\", b\"world\"])\n result = common.CastToString().transform(x)\n assert result.dtype.kind in [\"U\", \"S\"] # Unicode or byte string\n assert result[0] == \"hello\"\n assert result[1] == \"world\"\n\n\ndef test_cast_to_string_with_strings():\n x = np.array([\"hello\", \"world\"])\n result = common.CastToString().transform(x)\n assert result.dtype.kind in [\"U\", \"S\"]\n assert result[0] == \"hello\"\n assert result[1] == \"world\"\n\n\ndef test_text_tokenizer_vocab_limit():\n x = np.array([\"word1 word2 word3\", \"word1 word4 word5\"])\n tokenizer = common.TextTokenizer(max_vocab=2)\n tokenizer.fit(x)\n assert len(tokenizer.vocab) <= 3 # 2 words + 1 for unknown (0 is padding)\n # word1 should be most frequent\n assert \"word1\" in tokenizer.vocab\n assert tokenizer.vocab[\"word1\"] == 1\n\n\ndef test_text_tokenizer_transform():\n x = np.array([\"hello world\", \"hello\"])\n tokenizer = common.TextTokenizer(max_vocab=10)\n tokenizer.fit(x)\n result = tokenizer.transform(x)\n assert result.shape == (2, 100) # max_len=100\n assert result.dtype == np.int32\n assert result[0][0] == tokenizer.vocab.get(\"hello\", 0)\n" }, { "path": "autokeras/preprocessors/encoders.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nimport keras\nimport numpy as np\n\nfrom autokeras import analysers\nfrom autokeras.engine import preprocessor\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass Encoder(preprocessor.TargetPreprocessor):\n \"\"\"Transform labels to encodings.\n\n # Arguments\n labels: A list of labels of any type. The labels to be encoded.\n \"\"\"\n\n def __init__(self, labels, **kwargs):\n super().__init__(**kwargs)\n self.labels = [\n label.decode(\"utf-8\") if isinstance(label, bytes) else str(label)\n for label in labels\n ]\n\n def get_config(self):\n return {\"labels\": self.labels}\n\n def fit(self, dataset):\n return\n\n def transform(self, dataset):\n \"\"\"Transform labels to integer encodings.\n\n # Arguments\n dataset: numpy.ndarray. The dataset to be transformed.\n\n # Returns\n numpy.ndarray. The transformed dataset.\n \"\"\"\n label_to_idx = {label: idx for idx, label in enumerate(self.labels)}\n return np.array(\n [\n label_to_idx.get(str(label), len(self.labels))\n for label in dataset.flatten()\n ]\n ).reshape(dataset.shape)\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass OneHotEncoder(Encoder):\n def transform(self, dataset):\n \"\"\"Transform labels to one-hot encodings.\n\n # Arguments\n dataset: numpy.ndarray. The dataset to be transformed.\n\n # Returns\n numpy.ndarray. The transformed dataset.\n \"\"\"\n dataset = super().transform(dataset)\n eye = np.eye(len(self.labels))\n encoded_int = dataset.squeeze(axis=-1)\n result = np.zeros((len(encoded_int), len(self.labels)))\n for i, idx in enumerate(encoded_int):\n if idx < len(self.labels):\n result[i] = eye[idx]\n return result\n\n def postprocess(self, data):\n \"\"\"Transform probabilities back to labels.\n\n # Arguments\n data: numpy.ndarray. The output probabilities of the classification\n head.\n\n # Returns\n numpy.ndarray. The original labels.\n \"\"\"\n return np.array(\n list(\n map(\n lambda x: (\n self.labels[x] if x < len(self.labels) else \"unknown\"\n ),\n np.argmax(np.array(data), axis=1),\n )\n )\n ).reshape(-1, 1)\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass LabelEncoder(Encoder):\n \"\"\"Transform the labels to integer encodings.\"\"\"\n\n def transform(self, dataset):\n \"\"\"Transform labels to integer encodings.\n\n # Arguments\n dataset: numpy.ndarray. The dataset to be transformed.\n\n # Returns\n numpy.ndarray. The transformed dataset.\n \"\"\"\n dataset = super().transform(dataset)\n return dataset\n\n def postprocess(self, data):\n \"\"\"Transform probabilities back to labels.\n\n # Arguments\n data: numpy.ndarray. The output probabilities of the classification\n head.\n\n # Returns\n numpy.ndarray. The original labels.\n \"\"\"\n return np.array(\n list(\n map(\n lambda x: (\n self.labels[int(round(x[0]))]\n if int(round(x[0])) < len(self.labels)\n else \"unknown\"\n ),\n np.array(data),\n )\n )\n ).reshape(-1, 1)\n\n\n@keras.utils.register_keras_serializable()\nclass CategoricalToNumerical(preprocessor.Preprocessor):\n \"\"\"Encode the categorical features to numerical features.\n\n # Arguments\n column_names: A list of strings specifying the names of the columns. The\n length of the list should be equal to the number of columns of the\n data.\n column_types: Dict. The keys are the column names. The values should\n either be 'numerical' or 'categorical', indicating the type of that\n column. Defaults to None. If not None, the column_names need to be\n specified. If None, it will be inferred from the data.\n \"\"\"\n\n # TODO: Support one-hot encoding.\n # TODO: Support frequency encoding.\n\n def __init__(self, column_names, column_types, **kwargs):\n super().__init__(**kwargs)\n self.column_names = column_names\n self.column_types = column_types\n self.encoding = []\n for column_name in self.column_names:\n column_type = self.column_types[column_name]\n if column_type == analysers.CATEGORICAL:\n # TODO: Search to use one-hot or int.\n self.encoding.append(\"int\")\n else:\n self.encoding.append(\"none\")\n self.encoders = [None] * len(self.encoding)\n\n def fit(self, dataset):\n for i, enc_type in enumerate(self.encoding):\n if enc_type == \"none\":\n continue\n column_data = dataset[:, i]\n unique_labels = np.unique(column_data)\n if enc_type == \"int\":\n self.encoders[i] = LabelEncoder(unique_labels)\n elif enc_type == \"one_hot\": # pragma: no cover\n self.encoders[i] = OneHotEncoder( # pragma: no cover\n unique_labels\n )\n else:\n raise ValueError( # pragma: no cover\n f\"Unsupported encoding: {enc_type}\"\n )\n\n def transform(self, dataset):\n outputs = []\n for i, enc_type in enumerate(self.encoding):\n column_data = dataset[:, i : i + 1]\n if enc_type == \"none\":\n column_data = column_data.astype(\"float32\")\n # Replace NaN with 0.\n imputed = np.where(np.isnan(column_data), 0, column_data)\n outputs.append(imputed)\n else:\n encoded = self.encoders[i].transform(column_data)\n outputs.append(encoded)\n return np.concatenate(outputs, axis=1).astype(\"float32\")\n\n def get_config(self):\n encoders_config = []\n for enc in self.encoders:\n if enc is None:\n encoders_config.append(\n {\"encoder_type\": None, \"encoder_config\": None}\n )\n else:\n encoder_type = (\n \"label\" if isinstance(enc, LabelEncoder) else \"one_hot\"\n )\n encoders_config.append(\n {\n \"encoder_type\": encoder_type,\n \"encoder_config\": enc.get_config(),\n }\n )\n return {\n \"column_types\": self.column_types,\n \"column_names\": self.column_names,\n \"encoding\": self.encoding,\n \"encoders\": encoders_config,\n }\n\n @classmethod\n def from_config(cls, config):\n obj = cls(config[\"column_names\"], config[\"column_types\"])\n obj.encoding = config[\"encoding\"]\n obj.encoders = []\n for item in config[\"encoders\"]:\n if item[\"encoder_type\"] is None:\n obj.encoders.append(None)\n elif item[\"encoder_type\"] == \"label\":\n obj.encoders.append(LabelEncoder(**item[\"encoder_config\"]))\n elif item[\"encoder_type\"] == \"one_hot\": # pragma: no cover\n obj.encoders.append( # pragma: no cover\n OneHotEncoder(**item[\"encoder_config\"])\n )\n return obj\n" }, { "path": "autokeras/preprocessors/encoders_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras import preprocessors\nfrom autokeras.preprocessors import encoders\nfrom autokeras.utils import data_utils\n\n\ndef test_one_hot_encoder_deserialize_transforms_to_np():\n encoder = encoders.OneHotEncoder([\"a\", \"b\", \"c\"])\n encoder.fit(np.array([\"a\", \"b\", \"a\"]))\n\n encoder = preprocessors.deserialize(preprocessors.serialize(encoder))\n one_hot = encoder.transform(np.array([[\"a\"], [\"c\"], [\"b\"]]))\n\n assert one_hot.shape[1:] == (3,)\n\n\ndef test_one_hot_encoder_decode_to_same_string():\n encoder = encoders.OneHotEncoder([\"a\", \"b\", \"c\"])\n\n result = encoder.postprocess(np.eye(3))\n\n assert np.array_equal(result, np.array([[\"a\"], [\"b\"], [\"c\"]]))\n\n\ndef test_label_encoder_decode_to_same_string():\n encoder = encoders.LabelEncoder([\"a\", \"b\"])\n\n result = encoder.postprocess([[0], [1]])\n\n assert np.array_equal(result, np.array([[\"a\"], [\"b\"]]))\n\n\ndef test_label_encoder_encode_to_correct_shape():\n encoder = encoders.LabelEncoder([\"a\", \"b\"])\n dataset = np.array([[\"a\"], [\"b\"]])\n\n result = encoder.transform(dataset)\n print(data_utils.dataset_shape(result))\n\n assert np.array_equal(data_utils.dataset_shape(result), [2, 1])\n" }, { "path": "autokeras/preprocessors/postprocessors.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport numpy as np\n\nfrom autokeras.engine import preprocessor\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass PostProcessor(preprocessor.TargetPreprocessor):\n def transform(self, dataset):\n return dataset\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass SigmoidPostprocessor(PostProcessor):\n \"\"\"Postprocessor for sigmoid outputs.\"\"\"\n\n def postprocess(self, data):\n \"\"\"Transform probabilities to zeros and ones.\n\n # Arguments\n data: numpy.ndarray. The output probabilities of the classification\n head.\n\n # Returns\n numpy.ndarray. The zeros and ones predictions.\n \"\"\"\n data[data < 0.5] = 0\n data[data > 0.5] = 1\n return data\n\n\n@keras.utils.register_keras_serializable(package=\"autokeras\")\nclass SoftmaxPostprocessor(PostProcessor):\n \"\"\"Postprocessor for softmax outputs.\"\"\"\n\n def postprocess(self, data):\n \"\"\"Transform probabilities to zeros and ones.\n\n # Arguments\n data: numpy.ndarray. The output probabilities of the classification\n head.\n\n # Returns\n numpy.ndarray. The zeros and ones predictions.\n \"\"\"\n idx = np.argmax(data, axis=-1)\n data = np.zeros(data.shape)\n data[np.arange(data.shape[0]), idx] = 1\n return data\n" }, { "path": "autokeras/preprocessors/postprocessors_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\n\nfrom autokeras import preprocessors\nfrom autokeras.preprocessors import postprocessors\n\n\ndef test_sigmoid_postprocess_to_zero_one():\n postprocessor = postprocessors.SigmoidPostprocessor()\n\n y = postprocessor.postprocess(np.random.rand(10, 3))\n\n assert set(y.flatten().tolist()) == set([1, 0])\n\n\ndef test_sigmoid_deserialize_without_error():\n postprocessor = postprocessors.SigmoidPostprocessor()\n dataset = np.array([1, 2])\n\n postprocessor = preprocessors.deserialize(\n preprocessors.serialize(postprocessor)\n )\n\n assert isinstance(postprocessor.transform(dataset), np.ndarray)\n\n\ndef test_softmax_postprocess_to_zero_one():\n postprocessor = postprocessors.SoftmaxPostprocessor()\n\n y = postprocessor.postprocess(np.random.rand(10, 3))\n\n assert set(y.flatten().tolist()) == set([1, 0])\n\n\ndef test_softmax_transform_dataset_doesnt_change():\n postprocessor = postprocessors.SoftmaxPostprocessor()\n dataset = np.array([1, 2])\n\n assert isinstance(postprocessor.transform(dataset), np.ndarray)\n\n\ndef test_softmax_deserialize_without_error():\n postprocessor = postprocessors.SoftmaxPostprocessor()\n dataset = np.array([1, 2])\n\n postprocessor = preprocessors.deserialize(\n preprocessors.serialize(postprocessor)\n )\n\n assert isinstance(postprocessor.transform(dataset), np.ndarray)\n" }, { "path": "autokeras/tasks/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.tasks.image import ImageClassifier\nfrom autokeras.tasks.image import ImageRegressor\nfrom autokeras.tasks.structured_data import StructuredDataClassifier\nfrom autokeras.tasks.structured_data import StructuredDataRegressor\nfrom autokeras.tasks.text import TextClassifier\nfrom autokeras.tasks.text import TextRegressor\n" }, { "path": "autokeras/tasks/image.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom pathlib import Path\nfrom typing import List\nfrom typing import Optional\nfrom typing import Tuple\nfrom typing import Type\nfrom typing import Union\n\nimport keras\n\nfrom autokeras import auto_model\nfrom autokeras import blocks\nfrom autokeras import nodes as input_module\nfrom autokeras.engine import tuner\nfrom autokeras.tuners import greedy\nfrom autokeras.tuners import task_specific\nfrom autokeras.utils import types\n\n\nclass SupervisedImagePipeline(auto_model.AutoModel):\n def __init__(self, outputs, **kwargs):\n super().__init__(\n inputs=input_module.ImageInput(), outputs=outputs, **kwargs\n )\n\n\nclass ImageClassifier(SupervisedImagePipeline):\n \"\"\"AutoKeras image classification class.\n\n # Arguments\n num_classes: Int. Defaults to None. If None, it will be inferred from\n the data.\n multi_label: Boolean. Defaults to False.\n loss: A Keras loss function. Defaults to use 'binary_crossentropy' or\n 'categorical_crossentropy' based on the number of classes.\n metrics: A list of Keras metrics. Defaults to use 'accuracy'.\n project_name: String. The name of the AutoModel.\n Defaults to 'image_classifier'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n num_classes: Optional[int] = None,\n multi_label: bool = False,\n loss: types.LossType = None,\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"image_classifier\",\n max_trials: int = 100,\n directory: Union[str, Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = task_specific.ImageClassifierTuner\n super().__init__(\n outputs=blocks.ClassificationHead(\n num_classes=num_classes,\n multi_label=multi_label,\n loss=loss,\n metrics=metrics,\n ),\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n\n def fit(\n self,\n x: Optional[types.DatasetType] = None,\n y: Optional[types.DatasetType] = None,\n epochs: Optional[int] = None,\n callbacks: Optional[List[keras.callbacks.Callback]] = None,\n validation_split: Optional[float] = 0.2,\n validation_data: Union[\n Tuple[types.DatasetType, types.DatasetType], None\n ] = None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n It will search for the best model based on the performances on\n validation data.\n\n # Arguments\n x: numpy.ndarray. Training data x. The shape\n of the data should be (samples, width, height) or (samples,\n width, height, channels).\n y: numpy.ndarray. Training data y. It can be\n raw labels, one-hot encoded if more than two classes, or binary\n encoded for binary classification.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, by default we train for a maximum of\n 1000 epochs, but we stop training if the validation loss stops\n improving for 10 epochs (unless you specified an EarlyStopping\n callback as part of the callbacks argument, in which case the\n EarlyStopping callback you specified will determine early\n stopping).\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset. The best model found would be fit on the entire dataset\n including the validation data.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training loss\n values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n\n\nclass ImageRegressor(SupervisedImagePipeline):\n \"\"\"AutoKeras image regression class.\n\n # Arguments\n output_dim: Int. The number of output dimensions. Defaults to None.\n If None, it will be inferred from the data.\n loss: A Keras loss function. Defaults to use 'mean_squared_error'.\n metrics: A list of Keras metrics. Defaults to use 'mean_squared_error'.\n project_name: String. The name of the AutoModel.\n Defaults to 'image_regressor'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n output_dim: Optional[int] = None,\n loss: types.LossType = \"mean_squared_error\",\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"image_regressor\",\n max_trials: int = 100,\n directory: Union[str, Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = greedy.Greedy\n super().__init__(\n outputs=blocks.RegressionHead(\n output_dim=output_dim, loss=loss, metrics=metrics\n ),\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n\n def fit(\n self,\n x: Optional[types.DatasetType] = None,\n y: Optional[types.DatasetType] = None,\n epochs: Optional[int] = None,\n callbacks: Optional[List[keras.callbacks.Callback]] = None,\n validation_split: Optional[float] = 0.2,\n validation_data: Union[\n types.DatasetType, Tuple[types.DatasetType], None\n ] = None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n It will search for the best model based on the performances on\n validation data.\n\n # Arguments\n x: numpy.ndarray. Training data x. The shape\n of the data should be (samples, width, height) or (samples,\n width, height, channels).\n y: numpy.ndarray. Training data y. The targets\n passing to the head would have to be np.ndarray. It can be\n single-column or multi-column. The values should all be\n numerical.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, by default we train for a maximum of\n 1000 epochs, but we stop training if the validation loss stops\n improving for 10 epochs (unless you specified an EarlyStopping\n callback as part of the callbacks argument, in which case the\n EarlyStopping callback you specified will determine early\n stopping).\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset. The best model found would be fit on the entire\n dataset including the validation data.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n" }, { "path": "autokeras/tasks/image_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_img_clf_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.ImageClassifier(directory=tmp_path, seed=test_utils.SEED)\n\n auto_model.fit(\n x=test_utils.generate_data(num_instances=100, shape=(32, 32, 3)),\n y=test_utils.generate_one_hot_labels(num_instances=100, num_classes=10),\n )\n\n assert fit.is_called\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_img_reg_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.ImageRegressor(directory=tmp_path, seed=test_utils.SEED)\n\n auto_model.fit(\n x=test_utils.generate_data(num_instances=100, shape=(32, 32, 3)),\n y=test_utils.generate_data(num_instances=100, shape=(1,)),\n )\n\n assert fit.is_called\n" }, { "path": "autokeras/tasks/structured_data.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport pathlib\nfrom typing import Dict\nfrom typing import List\nfrom typing import Optional\nfrom typing import Type\nfrom typing import Union\n\nimport tree\n\nfrom autokeras import auto_model\nfrom autokeras import blocks\nfrom autokeras import nodes as input_module\nfrom autokeras.engine import tuner\nfrom autokeras.tuners import task_specific\nfrom autokeras.utils import types\n\n\nclass BaseStructuredDataPipeline(auto_model.AutoModel):\n def __init__(self, inputs, outputs, **kwargs):\n self.check(inputs.column_names, inputs.column_types)\n super().__init__(inputs=inputs, outputs=outputs, **kwargs)\n self._target_col_name = None\n\n def check(self, column_names, column_types):\n if column_types:\n for column_type in column_types.values():\n if column_type not in [\"categorical\", \"numerical\"]:\n raise ValueError(\n 'column_types should be either \"categorical\" '\n 'or \"numerical\", but got {name}'.format(\n name=column_type\n )\n )\n\n def check_in_fit(self, x):\n input_node = tree.flatten(self.inputs)[0]\n if input_node.column_names and input_node.column_types:\n for column_name in input_node.column_types:\n if column_name not in input_node.column_names:\n raise ValueError(\n \"column_names and column_types are \"\n \"mismatched. Cannot find column name \"\n \"{name} in the data.\".format(name=column_name)\n )\n\n def fit(\n self,\n x=None,\n y=None,\n epochs=None,\n callbacks=None,\n validation_split=0.2,\n validation_data=None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n # Arguments\n x: numpy.ndarray.\n Training data x. It can be string or numerical data.\n y: numpy.ndarray. Training data y.\n It can be raw labels, one-hot encoded if more than two classes,\n or binary encoded for binary classification.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, we would use epochs equal to 1000 and\n early stopping with patience equal to 10.\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n self.check_in_fit(x)\n\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n\n def predict(self, x, **kwargs):\n \"\"\"Predict the output for a given testing data.\n\n # Arguments\n x: numpy.ndarray.\n Testing data x. It can be string or numerical data.\n **kwargs: Any arguments supported by keras.Model.predict.\n\n # Returns\n A list of numpy.ndarray objects or a single numpy.ndarray.\n The predicted results.\n \"\"\"\n return super().predict(x=x, **kwargs)\n\n def evaluate(self, x, y=None, **kwargs):\n \"\"\"Evaluate the best model for the given data.\n\n # Arguments\n x: numpy.ndarray.\n Testing data x. It can be string or numerical data.\n y: numpy.ndarray. Testing data y.\n It can be string labels or numerical values.\n **kwargs: Any arguments supported by keras.Model.evaluate.\n\n # Returns\n Scalar test loss (if the model has a single output and no metrics)\n or list of scalars (if the model has multiple outputs and/or\n metrics). The attribute model.metrics_names will give you the\n display labels for the scalar outputs.\n \"\"\"\n return super().evaluate(x=x, y=y, **kwargs)\n\n\nclass SupervisedStructuredDataPipeline(BaseStructuredDataPipeline):\n def __init__(self, outputs, column_names, column_types, **kwargs):\n inputs = input_module.StructuredDataInput(\n column_names=column_names, column_types=column_types\n )\n super().__init__(inputs=inputs, outputs=outputs, **kwargs)\n\n\nclass StructuredDataClassifier(SupervisedStructuredDataPipeline):\n \"\"\"AutoKeras structured data classification class.\n\n # Arguments\n column_names: A list of strings specifying the names of the columns. The\n length of the list should be equal to the number of columns of the\n data excluding the target column. Defaults to None.\n column_types: Dict. The keys are the column names. The values should\n either be 'numerical' or 'categorical', indicating the type of that\n column. Defaults to None. If not None, the column_names need to be\n specified. If None, it will be inferred from the data.\n num_classes: Int. Defaults to None. If None, it will be inferred from\n the data.\n multi_label: Boolean. Defaults to False.\n loss: A Keras loss function. Defaults to use 'binary_crossentropy' or\n 'categorical_crossentropy' based on the number of classes.\n metrics: A list of Keras metrics. Defaults to use 'accuracy'.\n project_name: String. The name of the AutoModel. Defaults to\n 'structured_data_classifier'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize. Defaults to 'val_accuracy'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n column_names: Optional[List[str]] = None,\n column_types: Optional[Dict] = None,\n num_classes: Optional[int] = None,\n multi_label: bool = False,\n loss: Optional[types.LossType] = None,\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"structured_data_classifier\",\n max_trials: int = 100,\n directory: Optional[Union[str, pathlib.Path]] = None,\n objective: str = \"val_accuracy\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = task_specific.StructuredDataClassifierTuner\n super().__init__(\n outputs=blocks.ClassificationHead(\n num_classes=num_classes,\n multi_label=multi_label,\n loss=loss,\n metrics=metrics,\n ),\n column_names=column_names,\n column_types=column_types,\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n\n def fit(\n self,\n x=None,\n y=None,\n epochs=None,\n callbacks=None,\n validation_split=0.2,\n validation_data=None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n # Arguments\n x: numpy.ndarray. Training data x.\n y: numpy.ndarray. Training data y.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, we would use epochs equal to 1000 and\n early stopping with patience equal to 10.\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n\n\nclass StructuredDataRegressor(SupervisedStructuredDataPipeline):\n \"\"\"AutoKeras structured data regression class.\n\n # Arguments\n column_names: A list of strings specifying the names of the columns. The\n length of the list should be equal to the number of columns of the\n data excluding the target column. Defaults to None.\n column_types: Dict. The keys are the column names. The values should\n either be 'numerical' or 'categorical', indicating the type of that\n column. Defaults to None. If not None, the column_names need to be\n specified. If None, it will be inferred from the data.\n output_dim: Int. The number of output dimensions. Defaults to None.\n If None, it will be inferred from the data.\n loss: A Keras loss function. Defaults to use 'mean_squared_error'.\n metrics: A list of Keras metrics. Defaults to use 'mean_squared_error'.\n project_name: String. The name of the AutoModel. Defaults to\n 'structured_data_regressor'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n column_names: Optional[List[str]] = None,\n column_types: Optional[Dict[str, str]] = None,\n output_dim: Optional[int] = None,\n loss: types.LossType = \"mean_squared_error\",\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"structured_data_regressor\",\n max_trials: int = 100,\n directory: Union[str, pathlib.Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = task_specific.StructuredDataRegressorTuner\n super().__init__(\n outputs=blocks.RegressionHead(\n output_dim=output_dim, loss=loss, metrics=metrics\n ),\n column_names=column_names,\n column_types=column_types,\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n" }, { "path": "autokeras/tasks/structured_data_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport numpy as np\nimport pandas as pd\nimport pytest\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\ndef test_raise_error_unknown_str_in_col_type(tmp_path):\n with pytest.raises(ValueError) as info:\n ak.StructuredDataClassifier(\n column_types={\"age\": \"num\", \"parch\": \"categorical\"},\n directory=tmp_path,\n seed=test_utils.SEED,\n )\n\n assert 'column_types should be either \"categorical\"' in str(info.value)\n\n\ndef test_structured_data_input_name_type_mismatch_error(tmp_path):\n with pytest.raises(ValueError) as info:\n clf = ak.StructuredDataClassifier(\n column_types={\"_age\": \"numerical\", \"parch\": \"categorical\"},\n column_names=[\"age\", \"fare\"],\n directory=tmp_path,\n seed=test_utils.SEED,\n )\n clf.fit(x=test_utils.TRAIN_CSV_PATH, y=\"survived\")\n\n assert \"column_names and column_types are mismatched.\" in str(info.value)\n\n\ndef test_structured_data_col_type_no_name_error(tmp_path):\n with pytest.raises(ValueError) as info:\n clf = ak.StructuredDataClassifier(\n column_types={\"age\": \"numerical\", \"parch\": \"categorical\"},\n directory=tmp_path,\n seed=test_utils.SEED,\n )\n clf.fit(x=np.random.rand(100, 30), y=np.random.rand(100, 1))\n\n assert \"column_names must be specified\" in str(info.value)\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\n@mock.patch(\"autokeras.AutoModel.evaluate\")\ndef test_structured_clf_evaluate_call_automodel_evaluate(\n evaluate, fit, tmp_path\n):\n auto_model = ak.StructuredDataClassifier(\n directory=tmp_path, seed=test_utils.SEED\n )\n\n auto_model.fit(x=test_utils.TRAIN_CSV_PATH, y=\"survived\")\n auto_model.evaluate(x=test_utils.TRAIN_CSV_PATH, y=\"survived\")\n\n assert evaluate.is_called\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\n@mock.patch(\"autokeras.AutoModel.predict\")\ndef test_structured_clf_predict_csv_call_automodel_predict(\n predict, fit, tmp_path\n):\n auto_model = ak.StructuredDataClassifier(\n directory=tmp_path, seed=test_utils.SEED\n )\n\n auto_model.fit(x=test_utils.TRAIN_CSV_PATH, y=\"survived\")\n auto_model.predict(x=test_utils.TEST_CSV_PATH)\n\n assert predict.is_called\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_structured_clf_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.StructuredDataClassifier(\n directory=tmp_path, seed=test_utils.SEED\n )\n\n auto_model.fit(\n x=pd.read_csv(test_utils.TRAIN_CSV_PATH).to_numpy().astype(str)[:100],\n y=test_utils.generate_one_hot_labels(num_instances=100, num_classes=3),\n )\n\n assert fit.is_called\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_structured_reg_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.StructuredDataRegressor(\n directory=tmp_path, seed=test_utils.SEED\n )\n\n auto_model.fit(\n x=pd.read_csv(test_utils.TRAIN_CSV_PATH).to_numpy().astype(str)[:100],\n y=test_utils.generate_data(num_instances=100, shape=(1,)),\n )\n\n assert fit.is_called\n" }, { "path": "autokeras/tasks/text.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom pathlib import Path\nfrom typing import Optional\nfrom typing import Type\nfrom typing import Union\n\nfrom autokeras import auto_model\nfrom autokeras import blocks\nfrom autokeras import nodes as input_module\nfrom autokeras.engine import tuner\nfrom autokeras.tuners import greedy\nfrom autokeras.tuners import task_specific\nfrom autokeras.utils import types\n\n\nclass SupervisedTextPipeline(auto_model.AutoModel):\n def __init__(self, outputs, **kwargs):\n super().__init__(\n inputs=input_module.TextInput(), outputs=outputs, **kwargs\n )\n\n\nclass TextClassifier(SupervisedTextPipeline):\n \"\"\"AutoKeras text classification class.\n\n # Arguments\n num_classes: Int. Defaults to None. If None, it will be inferred from\n the data.\n multi_label: Boolean. Defaults to False.\n loss: A Keras loss function. Defaults to use 'binary_crossentropy' or\n 'categorical_crossentropy' based on the number of classes.\n metrics: A list of Keras metrics. Defaults to use 'accuracy'.\n project_name: String. The name of the AutoModel.\n Defaults to 'text_classifier'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n num_classes: Optional[int] = None,\n multi_label: bool = False,\n loss: types.LossType = None,\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"text_classifier\",\n max_trials: int = 100,\n directory: Union[str, Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = task_specific.TextClassifierTuner\n super().__init__(\n outputs=blocks.ClassificationHead(\n num_classes=num_classes,\n multi_label=multi_label,\n loss=loss,\n metrics=metrics,\n ),\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n\n def fit(\n self,\n x=None,\n y=None,\n epochs=None,\n callbacks=None,\n validation_split=0.2,\n validation_data=None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n It will search for the best model based on the performances on\n validation data.\n\n # Arguments\n x: numpy.ndarray. Training data x. The input\n data should be numpy.ndarray. The data should\n be one dimensional. Each element in the data should be a string\n which is a full sentence.\n y: numpy.ndarray. Training data y. It can be\n raw labels, one-hot encoded if more than two classes, or binary\n encoded for binary classification.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, by default we train for a maximum of\n 1000 epochs, but we stop training if the validation loss stops\n improving for 10 epochs (unless you specified an EarlyStopping\n callback as part of the callbacks argument, in which case the\n EarlyStopping callback you specified will determine early\n stopping).\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset. The best model found would be fit on the entire\n dataset including the validation data.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n\n\nclass TextRegressor(SupervisedTextPipeline):\n \"\"\"AutoKeras text regression class.\n\n # Arguments\n output_dim: Int. The number of output dimensions. Defaults to None.\n If None, it will be inferred from the data.\n loss: A Keras loss function. Defaults to use 'mean_squared_error'.\n metrics: A list of Keras metrics. Defaults to use 'mean_squared_error'.\n project_name: String. The name of the AutoModel.\n Defaults to 'text_regressor'.\n max_trials: Int. The maximum number of different Keras Models to try.\n The search may finish before reaching the max_trials. Defaults to\n 100.\n directory: String. The path to a directory for storing the search\n outputs. Defaults to None, which would create a folder with the\n name of the AutoModel in the current directory.\n objective: String. Name of model metric to minimize\n or maximize, e.g. 'val_accuracy'. Defaults to 'val_loss'.\n tuner: String or subclass of AutoTuner. If string, it should be one of\n 'greedy', 'bayesian', 'hyperband' or 'random'. It can also be a\n subclass of AutoTuner. If left unspecified, it uses a task specific\n tuner, which first evaluates the most commonly used models for the\n task before exploring other models.\n overwrite: Boolean. Defaults to `False`. If `False`, reloads an existing\n project of the same name if one is found. Otherwise, overwrites the\n project.\n seed: Int. Random seed.\n max_model_size: Int. Maximum number of scalars in the parameters of a\n model. Models larger than this are rejected.\n **kwargs: Any arguments supported by AutoModel.\n \"\"\"\n\n def __init__(\n self,\n output_dim: Optional[int] = None,\n loss: types.LossType = \"mean_squared_error\",\n metrics: Optional[types.MetricsType] = None,\n project_name: str = \"text_regressor\",\n max_trials: int = 100,\n directory: Union[str, Path, None] = None,\n objective: str = \"val_loss\",\n tuner: Union[str, Type[tuner.AutoTuner]] = None,\n overwrite: bool = False,\n seed: Optional[int] = None,\n max_model_size: Optional[int] = None,\n **kwargs\n ):\n if tuner is None:\n tuner = greedy.Greedy\n super().__init__(\n outputs=blocks.RegressionHead(\n output_dim=output_dim, loss=loss, metrics=metrics\n ),\n max_trials=max_trials,\n directory=directory,\n project_name=project_name,\n objective=objective,\n tuner=tuner,\n overwrite=overwrite,\n seed=seed,\n max_model_size=max_model_size,\n **kwargs\n )\n\n def fit(\n self,\n x=None,\n y=None,\n epochs=None,\n callbacks=None,\n validation_split=0.2,\n validation_data=None,\n **kwargs\n ):\n \"\"\"Search for the best model and hyperparameters for the AutoModel.\n\n It will search for the best model based on the performances on\n validation data.\n\n # Arguments\n x: numpy.ndarray. Training data x. The input\n data should be numpy.ndarray. The data should\n be one dimensional. Each element in the data should be a string\n which is a full sentence.\n y: numpy.ndarray. Training data y. The targets\n passing to the head would have to be np.ndarray,. It can be\n single-column or multi-column. The values should all be\n numerical.\n epochs: Int. The number of epochs to train each model during the\n search. If unspecified, by default we train for a maximum of\n 1000 epochs, but we stop training if the validation loss stops\n improving for 10 epochs (unless you specified an EarlyStopping\n callback as part of the callbacks argument, in which case the\n EarlyStopping callback you specified will determine early\n stopping).\n callbacks: List of Keras callbacks to apply during training and\n validation.\n validation_split: Float between 0 and 1. Defaults to 0.2. Fraction\n of the training data to be used as validation data. The model\n will set apart this fraction of the training data, will not\n train on it, and will evaluate the loss and any model metrics on\n this data at the end of each epoch. The validation data is\n selected from the last samples in the `x` and `y` data provided,\n before shuffling. This argument is not supported when `x` is a\n dataset. The best model found would be fit on the entire\n dataset including the validation data.\n validation_data: Data on which to evaluate the loss and any model\n metrics at the end of each epoch. The model will not be trained\n on this data. `validation_data` will override\n `validation_split`. The type of the validation data should be\n the same as the training data. The best model found would be\n fit on the training dataset without the validation data.\n **kwargs: Any arguments supported by\n [keras.Model.fit](https://keras.io/api/models/model_training_apis/#fit-method).\n # Returns\n history: A Keras History object corresponding to the best model.\n Its History.history attribute is a record of training\n loss values and metrics values at successive epochs, as well as\n validation loss values and validation metrics values (if\n applicable).\n \"\"\"\n history = super().fit(\n x=x,\n y=y,\n epochs=epochs,\n callbacks=callbacks,\n validation_split=validation_split,\n validation_data=validation_data,\n **kwargs\n )\n return history\n" }, { "path": "autokeras/tasks/text_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport numpy as np\n\nimport autokeras as ak\nfrom autokeras import test_utils\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_txt_clf_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.TextClassifier(directory=tmp_path, seed=test_utils.SEED)\n\n auto_model.fit(x=np.array([\"a b c\", \"b b c\"]), y=np.array([1, 2]))\n\n assert fit.is_called\n\n\n@mock.patch(\"autokeras.AutoModel.fit\")\ndef test_txt_reg_fit_call_auto_model_fit(fit, tmp_path):\n auto_model = ak.TextRegressor(directory=tmp_path, seed=test_utils.SEED)\n\n auto_model.fit(x=np.array([\"a b c\", \"b b c\"]), y=np.array([1.0, 2.0]))\n\n assert fit.is_called\n" }, { "path": "autokeras/test_utils.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport inspect\nimport os\n\nimport keras\nimport numpy as np\n\nimport autokeras as ak\n\nSEED = 5\n\n\nCOLUMN_NAMES = [\n \"sex\",\n \"age\",\n \"n_siblings_spouses\",\n \"parch\",\n \"fare\",\n \"class\",\n \"deck\",\n \"embark_town\",\n \"alone\",\n]\nCOLUMN_TYPES = {\n \"sex\": \"categorical\",\n \"age\": \"numerical\",\n \"n_siblings_spouses\": \"categorical\",\n \"parch\": \"categorical\",\n \"fare\": \"numerical\",\n \"class\": \"categorical\",\n \"deck\": \"categorical\",\n \"embark_town\": \"categorical\",\n \"alone\": \"categorical\",\n}\n\nROOT_DIR = os.path.abspath(os.path.join(os.path.dirname(__file__), \"..\"))\n# Use pre-split CSVs stored under benchmark/datasets to avoid downloads and\n# processing at import time.\nTRAIN_CSV_PATH = os.path.join(\n ROOT_DIR, \"benchmark\", \"datasets\", \"titanic_train.csv\"\n)\nTEST_CSV_PATH = os.path.join(\n ROOT_DIR, \"benchmark\", \"datasets\", \"titanic_test.csv\"\n)\n\n\ndef generate_data(num_instances=100, shape=(32, 32, 3)):\n np.random.seed(SEED)\n result = np.random.rand(*((num_instances,) + shape))\n if result.dtype == np.float64:\n result = result.astype(np.float32)\n return result\n\n\ndef generate_one_hot_labels(num_instances=100, num_classes=10):\n np.random.seed(SEED)\n labels = np.random.randint(num_classes, size=num_instances)\n result = keras.utils.to_categorical(labels, num_classes=num_classes)\n return result\n\n\ndef generate_text_data(num_instances=100):\n vocab = np.array(\n [\n [\"adorable\", \"clueless\", \"dirty\", \"odd\", \"stupid\"],\n [\"puppy\", \"car\", \"rabbit\", \"girl\", \"monkey\"],\n [\"runs\", \"hits\", \"jumps\", \"drives\", \"barfs\"],\n [\n \"crazily.\",\n \"dutifully.\",\n \"foolishly.\",\n \"merrily.\",\n \"occasionally.\",\n ],\n ]\n )\n return np.array(\n [\n \" \".join([vocab[j][np.random.randint(0, 5)] for j in range(4)])\n for i in range(num_instances)\n ]\n )\n\n\ndef build_graph():\n keras.backend.clear_session()\n image_input = ak.ImageInput(shape=(32, 32, 3))\n image_input.batch_size = 32\n image_input.num_samples = 1000\n merged_outputs = ak.SpatialReduction()(image_input)\n head = ak.ClassificationHead(num_classes=10, shape=(10,))\n classification_outputs = head(merged_outputs)\n return ak.graph.Graph(inputs=image_input, outputs=classification_outputs)\n\n\ndef get_func_args(func):\n params = inspect.signature(func).parameters.keys()\n return set(params) - set([\"self\", \"args\", \"kwargs\"])\n\n\ndef get_object_detection_data():\n images = generate_data(num_instances=2, shape=(32, 32, 3))\n\n bbox_0 = np.random.rand(3, 4)\n class_id_0 = np.random.rand(\n 3,\n )\n\n bbox_1 = np.random.rand(5, 4)\n class_id_1 = np.random.rand(\n 5,\n )\n\n labels = np.array(\n [(bbox_0, class_id_0), (bbox_1, class_id_1)], dtype=object\n )\n\n return images, labels\n\n\ndef generate_data_with_categorical(\n num_instances=100,\n num_numerical=10,\n num_categorical=3,\n num_classes=5,\n):\n categorical_data = np.random.randint(\n num_classes, size=(num_instances, num_categorical)\n )\n numerical_data = np.random.rand(num_instances, num_numerical)\n data = np.concatenate((numerical_data, categorical_data), axis=1)\n if data.dtype == np.float64:\n data = data.astype(np.float32)\n return data\n" }, { "path": "autokeras/tuners/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.tuners.bayesian_optimization import BayesianOptimization\nfrom autokeras.tuners.greedy import Greedy\nfrom autokeras.tuners.hyperband import Hyperband\nfrom autokeras.tuners.random_search import RandomSearch\nfrom autokeras.tuners.task_specific import ImageClassifierTuner\n" }, { "path": "autokeras/tuners/bayesian_optimization.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras_tuner\n\nfrom autokeras.engine import tuner as tuner_module\n\n\nclass BayesianOptimization(\n keras_tuner.BayesianOptimization, tuner_module.AutoTuner\n):\n \"\"\"KerasTuner BayesianOptimization with preprocessing layer tuning.\"\"\"\n\n pass\n" }, { "path": "autokeras/tuners/greedy.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport copy\nfrom typing import Any\nfrom typing import Dict\nfrom typing import List\nfrom typing import Optional\n\nimport keras_tuner\nimport numpy as np\n\nfrom autokeras.engine import tuner as tuner_module\n\n\nclass TrieNode(object):\n def __init__(self):\n super().__init__()\n self.num_leaves = 0\n self.children = {}\n self.hp_name = None\n\n def is_leaf(self):\n return len(self.children) == 0\n\n\nclass Trie(object):\n def __init__(self):\n super().__init__()\n self.root = TrieNode()\n\n def insert(self, hp_name):\n names = hp_name.split(\"/\")\n\n new_word = False\n current_node = self.root\n nodes_on_path = [current_node]\n for name in names:\n if name not in current_node.children:\n current_node.children[name] = TrieNode()\n new_word = True\n current_node = current_node.children[name]\n nodes_on_path.append(current_node)\n current_node.hp_name = hp_name\n\n if new_word:\n for node in nodes_on_path:\n node.num_leaves += 1\n\n @property\n def nodes(self):\n return self._get_all_nodes(self.root)\n\n def _get_all_nodes(self, node):\n ret = [node]\n for key, value in node.children.items():\n ret += self._get_all_nodes(value)\n return ret\n\n def get_hp_names(self, node):\n if node.is_leaf():\n return [node.hp_name]\n ret = []\n for key, value in node.children.items():\n ret += self.get_hp_names(value)\n return ret\n\n\nclass GreedyOracle(keras_tuner.Oracle):\n \"\"\"An oracle combining random search and greedy algorithm.\n\n It groups the HyperParameters into several categories, namely, HyperGraph,\n Preprocessor, Architecture, and Optimization. The oracle tunes each group\n separately using random search. In each trial, it use a greedy strategy to\n generate new values for one of the categories of HyperParameters and use the\n best trial so far for the rest of the HyperParameters values.\n\n # Arguments\n initial_hps: A list of dictionaries in the form of\n {HyperParameter name (String): HyperParameter value}.\n Each dictionary is one set of HyperParameters, which are used as the\n initial trials for the search. Defaults to None.\n seed: Int. Random seed.\n \"\"\"\n\n def __init__(self, initial_hps=None, seed=None, **kwargs):\n super().__init__(seed=seed, **kwargs)\n self.initial_hps = copy.deepcopy(initial_hps) or []\n self._tried_initial_hps = [False] * len(self.initial_hps)\n\n def get_state(self):\n state = super().get_state()\n state.update(\n {\n \"initial_hps\": self.initial_hps,\n \"tried_initial_hps\": self._tried_initial_hps,\n }\n )\n return state\n\n def set_state(self, state):\n super().set_state(state)\n self.initial_hps = state[\"initial_hps\"]\n self._tried_initial_hps = state[\"tried_initial_hps\"]\n\n def _select_hps(self):\n trie = Trie()\n best_hps = self._get_best_hps()\n for hp in best_hps.space:\n # Not picking the fixed hps for generating new values.\n if best_hps.is_active(hp) and not isinstance(\n hp, keras_tuner.engine.hyperparameters.Fixed\n ):\n trie.insert(hp.name)\n all_nodes = trie.nodes\n\n if len(all_nodes) <= 1:\n return []\n\n probabilities = np.array([1 / node.num_leaves for node in all_nodes])\n sum_p = np.sum(probabilities)\n probabilities = probabilities / sum_p\n node = np.random.choice(all_nodes, p=probabilities)\n\n return trie.get_hp_names(node)\n\n def _next_initial_hps(self):\n for index, hps in enumerate(self.initial_hps):\n if not self._tried_initial_hps[index]:\n self._tried_initial_hps[index] = True\n return hps\n\n def populate_space(self, trial_id):\n if not all(self._tried_initial_hps):\n values = self._next_initial_hps()\n return {\n \"status\": keras_tuner.engine.trial.TrialStatus.RUNNING,\n \"values\": values,\n }\n\n for _ in range(self._max_collisions):\n hp_names = self._select_hps()\n values = self._generate_hp_values(hp_names)\n # Reached max collisions.\n if values is None:\n continue\n # Values found.\n return {\n \"status\": keras_tuner.engine.trial.TrialStatus.RUNNING,\n \"values\": values,\n }\n # All stages reached max collisions.\n return {\n \"status\": keras_tuner.engine.trial.TrialStatus.STOPPED,\n \"values\": None,\n }\n\n def _get_best_hps(self):\n best_trials = self.get_best_trials()\n if best_trials:\n return best_trials[0].hyperparameters.copy()\n else:\n return self.hyperparameters.copy()\n\n def _generate_hp_values(self, hp_names):\n best_hps = self._get_best_hps()\n\n collisions = 0\n while True:\n hps = keras_tuner.HyperParameters()\n # Generate a set of random values.\n for hp in self.hyperparameters.space:\n hps.merge([hp])\n # if not active, do nothing.\n # if active, check if selected to be changed.\n if hps.is_active(hp):\n # if was active and not selected, do nothing.\n if best_hps.is_active(hp.name) and hp.name not in hp_names:\n hps.values[hp.name] = best_hps.values[hp.name]\n continue\n # if was not active or selected, sample.\n hps.values[hp.name] = hp.random_sample(self._seed_state)\n self._seed_state += 1\n values = hps.values\n # Keep trying until the set of values is unique,\n # or until we exit due to too many collisions.\n values_hash = self._compute_values_hash(values)\n if values_hash in self._tried_so_far:\n collisions += 1\n if collisions <= self._max_collisions:\n continue\n return None\n self._tried_so_far.add(values_hash)\n break\n return values\n\n\nclass Greedy(tuner_module.AutoTuner):\n def __init__(\n self,\n hypermodel: keras_tuner.HyperModel,\n objective: str = \"val_loss\",\n max_trials: int = 10,\n initial_hps: Optional[List[Dict[str, Any]]] = None,\n seed: Optional[int] = None,\n hyperparameters: Optional[keras_tuner.HyperParameters] = None,\n tune_new_entries: bool = True,\n allow_new_entries: bool = True,\n **kwargs\n ):\n self.seed = seed\n oracle = GreedyOracle(\n objective=objective,\n max_trials=max_trials,\n initial_hps=initial_hps,\n seed=seed,\n hyperparameters=hyperparameters,\n tune_new_entries=tune_new_entries,\n allow_new_entries=allow_new_entries,\n )\n super().__init__(oracle=oracle, hypermodel=hypermodel, **kwargs)\n" }, { "path": "autokeras/tuners/greedy_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom unittest import mock\n\nimport keras\nimport keras_tuner\n\nimport autokeras as ak\nfrom autokeras import test_utils\nfrom autokeras.tuners import greedy\nfrom autokeras.tuners import task_specific\n\n\ndef test_greedy_oracle_get_state_update_space_can_run():\n oracle = greedy.GreedyOracle(objective=\"val_loss\")\n oracle.set_state(oracle.get_state())\n hp = keras_tuner.HyperParameters()\n hp.Boolean(\"test\")\n oracle.update_space(hp)\n\n\n@mock.patch(\"autokeras.tuners.greedy.GreedyOracle.get_best_trials\")\ndef test_greedy_oracle_populate_different_values(get_best_trials):\n hp = keras_tuner.HyperParameters()\n test_utils.build_graph().build(hp)\n\n oracle = greedy.GreedyOracle(objective=\"val_loss\", seed=test_utils.SEED)\n trial = mock.Mock()\n trial.hyperparameters = hp\n get_best_trials.return_value = [trial]\n\n oracle.update_space(hp)\n values_a = oracle.populate_space(\"a\")[\"values\"]\n values_b = oracle.populate_space(\"b\")[\"values\"]\n\n assert not all([values_a[key] == values_b[key] for key in values_a])\n\n\n@mock.patch(\"autokeras.tuners.greedy.GreedyOracle.get_best_trials\")\ndef test_greedy_oracle_populate_doesnt_crash_with_init_hps(get_best_trials):\n hp = keras_tuner.HyperParameters()\n keras.backend.clear_session()\n input_node = ak.ImageInput(shape=(32, 32, 3))\n input_node.batch_size = 32\n input_node.num_samples = 1000\n output_node = ak.ImageBlock()(input_node)\n head = ak.ClassificationHead(num_classes=10)\n head.shape = (10,)\n output_node = head(output_node)\n graph = ak.graph.Graph(inputs=input_node, outputs=output_node)\n graph.build(hp)\n\n oracle = greedy.GreedyOracle(\n initial_hps=task_specific.IMAGE_CLASSIFIER,\n objective=\"val_loss\",\n seed=test_utils.SEED,\n )\n trial = mock.Mock()\n trial.hyperparameters = hp\n get_best_trials.return_value = [trial]\n\n for i in range(10):\n keras.backend.clear_session()\n values = oracle.populate_space(\"a\")[\"values\"]\n hp = oracle.hyperparameters.copy()\n hp.values = values\n graph.build(hp)\n oracle.update_space(hp)\n\n\n@mock.patch(\"autokeras.tuners.greedy.GreedyOracle._compute_values_hash\")\n@mock.patch(\"autokeras.tuners.greedy.GreedyOracle.get_best_trials\")\ndef test_greedy_oracle_stop_reach_max_collision(\n get_best_trials, compute_values_hash\n):\n hp = keras_tuner.HyperParameters()\n test_utils.build_graph().build(hp)\n\n oracle = greedy.GreedyOracle(objective=\"val_loss\", seed=test_utils.SEED)\n trial = mock.Mock()\n trial.hyperparameters = hp\n get_best_trials.return_value = [trial]\n compute_values_hash.return_value = 1\n\n oracle.update_space(hp)\n oracle.populate_space(\"a\")[\"values\"]\n assert (\n oracle.populate_space(\"b\")[\"status\"]\n == keras_tuner.engine.trial.TrialStatus.STOPPED\n )\n\n\n@mock.patch(\"autokeras.tuners.greedy.GreedyOracle.get_best_trials\")\ndef test_greedy_oracle_populate_space_with_no_hp(get_best_trials):\n hp = keras_tuner.HyperParameters()\n\n oracle = greedy.GreedyOracle(objective=\"val_loss\", seed=test_utils.SEED)\n trial = mock.Mock()\n trial.hyperparameters = hp\n get_best_trials.return_value = [trial]\n\n oracle.update_space(hp)\n values_a = oracle.populate_space(\"a\")[\"values\"]\n\n assert len(values_a) == 0\n" }, { "path": "autokeras/tuners/hyperband.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras_tuner\n\nfrom autokeras.engine import tuner as tuner_module\n\n\nclass Hyperband(keras_tuner.Hyperband, tuner_module.AutoTuner):\n \"\"\"KerasTuner Hyperband with preprocessing layer tuning.\"\"\"\n\n def __init__(\n self, max_epochs: int = 1000, max_trials: int = 100, *args, **kwargs\n ):\n super().__init__(max_epochs=max_epochs, *args, **kwargs)\n self.oracle.max_trials = max_trials\n" }, { "path": "autokeras/tuners/hyperband_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/tuners/random_search.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras_tuner\n\nfrom autokeras.engine import tuner as tuner_module\n\n\nclass RandomSearch(keras_tuner.RandomSearch, tuner_module.AutoTuner):\n \"\"\"KerasTuner RandomSearch with preprocessing layer tuning.\"\"\"\n\n pass\n" }, { "path": "autokeras/tuners/random_search_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/tuners/task_specific.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom autokeras.tuners import greedy\n\nIMAGE_CLASSIFIER = [\n {\n \"image_block_1/block_type\": \"vanilla\",\n \"image_block_1/normalize\": True,\n \"image_block_1/augment\": False,\n \"image_block_1/conv_block_1/kernel_size\": 3,\n \"image_block_1/conv_block_1/num_blocks\": 1,\n \"image_block_1/conv_block_1/num_layers\": 2,\n \"image_block_1/conv_block_1/max_pooling\": True,\n \"image_block_1/conv_block_1/separable\": False,\n \"image_block_1/conv_block_1/dropout\": 0.25,\n \"image_block_1/conv_block_1/filters_0_0\": 32,\n \"image_block_1/conv_block_1/filters_0_1\": 64,\n \"classification_head_1/spatial_reduction_1/reduction_type\": \"flatten\",\n \"classification_head_1/dropout\": 0.5,\n \"optimizer\": \"adam\",\n \"learning_rate\": 1e-3,\n },\n {\n \"image_block_1/block_type\": \"resnet\",\n \"image_block_1/normalize\": True,\n \"image_block_1/augment\": True,\n \"image_block_1/image_augmentation_1/horizontal_flip\": True,\n \"image_block_1/image_augmentation_1/vertical_flip\": True,\n \"image_block_1/image_augmentation_1/contrast_factor\": 0.0,\n \"image_block_1/image_augmentation_1/rotation_factor\": 0.0,\n \"image_block_1/image_augmentation_1/translation_factor\": 0.1,\n \"image_block_1/image_augmentation_1/zoom_factor\": 0.0,\n \"image_block_1/res_net_block_1/pretrained\": False,\n \"image_block_1/res_net_block_1/version\": \"resnet50\",\n \"image_block_1/res_net_block_1/imagenet_size\": True,\n \"classification_head_1/spatial_reduction_1/reduction_type\": \"global_avg\", # noqa: E501\n \"classification_head_1/dropout\": 0,\n \"optimizer\": \"adam\",\n \"learning_rate\": 1e-3,\n },\n {\n \"image_block_1/block_type\": \"efficient\",\n \"image_block_1/normalize\": True,\n \"image_block_1/augment\": True,\n \"image_block_1/image_augmentation_1/horizontal_flip\": True,\n \"image_block_1/image_augmentation_1/vertical_flip\": False,\n \"image_block_1/image_augmentation_1/contrast_factor\": 0.0,\n \"image_block_1/image_augmentation_1/rotation_factor\": 0.0,\n \"image_block_1/image_augmentation_1/translation_factor\": 0.1,\n \"image_block_1/image_augmentation_1/zoom_factor\": 0.0,\n \"image_block_1/efficient_net_block_1/pretrained\": True,\n \"image_block_1/efficient_net_block_1/version\": \"b7\",\n \"image_block_1/efficient_net_block_1/trainable\": True,\n \"image_block_1/efficient_net_block_1/imagenet_size\": True,\n \"classification_head_1/spatial_reduction_1/reduction_type\": \"global_avg\", # noqa: E501\n \"classification_head_1/dropout\": 0,\n \"optimizer\": \"adam\",\n \"learning_rate\": 2e-5,\n },\n]\n\nTEXT_CLASSIFIER = [\n {\n \"text_block_1/max_tokens\": 500,\n \"text_block_1/embedding_1/embedding_dim\": 32,\n \"text_block_1/embedding_1/dropout\": 0.25,\n \"text_block_1/conv_block_1/kernel_size\": 3,\n \"text_block_1/conv_block_1/separable\": False,\n \"text_block_1/conv_block_1/max_pooling\": True,\n \"text_block_1/conv_block_1/num_blocks\": 2,\n \"text_block_1/conv_block_1/num_layers\": 2,\n \"text_block_1/conv_block_1/filters_0_0\": 32,\n \"text_block_1/conv_block_1/filters_0_1\": 32,\n \"text_block_1/conv_block_1/dropout\": 0.0,\n \"text_block_1/conv_block_1/filters_1_0\": 32,\n \"text_block_1/conv_block_1/filters_1_1\": 32,\n \"text_block_1/spatial_reduction_1/reduction_type\": \"flatten\",\n \"text_block_1/dense_block_1/use_batchnorm\": False,\n \"text_block_1/dense_block_1/num_layers\": 2,\n \"text_block_1/dense_block_1/units_0\": 32,\n \"text_block_1/dense_block_1/dropout\": 0.0,\n \"text_block_1/dense_block_1/units_1\": 32,\n \"classification_head_1/dropout\": 0,\n \"optimizer\": \"adam_weight_decay\",\n \"learning_rate\": 2e-5,\n },\n]\n\nSTRUCTURED_DATA_CLASSIFIER = [\n {\n \"structured_data_block_1/normalize\": True,\n \"structured_data_block_1/dense_block_1/num_layers\": 2,\n \"structured_data_block_1/dense_block_1/use_batchnorm\": False,\n \"structured_data_block_1/dense_block_1/dropout\": 0,\n \"structured_data_block_1/dense_block_1/units_0\": 32,\n \"structured_data_block_1/dense_block_1/units_1\": 32,\n \"classification_head_1/dropout\": 0.0,\n \"optimizer\": \"adam\",\n \"learning_rate\": 0.001,\n }\n]\n\nSTRUCTURED_DATA_REGRESSOR = [\n {\n \"structured_data_block_1/normalize\": True,\n \"structured_data_block_1/dense_block_1/num_layers\": 2,\n \"structured_data_block_1/dense_block_1/use_batchnorm\": False,\n \"structured_data_block_1/dense_block_1/dropout\": 0,\n \"structured_data_block_1/dense_block_1/units_0\": 32,\n \"structured_data_block_1/dense_block_1/units_1\": 32,\n \"regression_head_1/dropout\": 0.0,\n \"optimizer\": \"adam\",\n \"learning_rate\": 0.001,\n }\n]\n\n\nclass ImageClassifierTuner(greedy.Greedy):\n def __init__(self, **kwargs):\n super().__init__(initial_hps=IMAGE_CLASSIFIER, **kwargs)\n\n\nclass TextClassifierTuner(greedy.Greedy):\n def __init__(self, **kwargs):\n super().__init__(initial_hps=TEXT_CLASSIFIER, **kwargs)\n\n\nclass StructuredDataClassifierTuner(greedy.Greedy):\n def __init__(self, **kwargs):\n super().__init__(initial_hps=STRUCTURED_DATA_CLASSIFIER, **kwargs)\n\n\nclass StructuredDataRegressorTuner(greedy.Greedy):\n def __init__(self, **kwargs):\n super().__init__(initial_hps=STRUCTURED_DATA_REGRESSOR, **kwargs)\n" }, { "path": "autokeras/tuners/task_specific_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport copy\n\nimport keras_tuner\n\nimport autokeras as ak\nfrom autokeras.tuners import task_specific\n\n\ndef test_img_clf_init_hp0_equals_hp_of_a_model(tmp_path):\n clf = ak.ImageClassifier(directory=tmp_path)\n clf.inputs[0].shape = (32, 32, 3)\n clf.outputs[0].in_blocks[0].shape = (10,)\n init_hp = task_specific.IMAGE_CLASSIFIER[0]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n\n\ndef test_img_clf_init_hp1_equals_hp_of_a_model(tmp_path):\n clf = ak.ImageClassifier(directory=tmp_path)\n clf.inputs[0].shape = (32, 32, 3)\n clf.outputs[0].in_blocks[0].shape = (10,)\n init_hp = task_specific.IMAGE_CLASSIFIER[1]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n\n\ndef test_img_clf_init_hp2_equals_hp_of_a_model(tmp_path):\n clf = ak.ImageClassifier(directory=tmp_path)\n clf.inputs[0].shape = (32, 32, 3)\n clf.outputs[0].in_blocks[0].shape = (10,)\n init_hp = task_specific.IMAGE_CLASSIFIER[2]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n\n\ndef test_txt_clf_init_hp0_equals_hp_of_a_model(tmp_path):\n clf = ak.TextClassifier(directory=tmp_path)\n clf.inputs[0].shape = (1,)\n clf.inputs[0].batch_size = 6\n clf.inputs[0].num_samples = 1000\n clf.outputs[0].in_blocks[0].shape = (10,)\n clf.tuner.hypermodel.epochs = 1000\n clf.tuner.hypermodel.num_samples = 20000\n init_hp = task_specific.TEXT_CLASSIFIER[0]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n\n\ndef test_sd_clf_init_hp0_equals_hp_of_a_model(tmp_path):\n clf = ak.StructuredDataClassifier(\n directory=tmp_path,\n column_names=[\"a\", \"b\"],\n column_types={\"a\": \"numerical\", \"b\": \"numerical\"},\n )\n clf.inputs[0].shape = (2,)\n clf.outputs[0].in_blocks[0].shape = (10,)\n init_hp = task_specific.STRUCTURED_DATA_CLASSIFIER[0]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n\n\ndef test_sd_reg_init_hp0_equals_hp_of_a_model(tmp_path):\n clf = ak.StructuredDataRegressor(\n directory=tmp_path,\n column_names=[\"a\", \"b\"],\n column_types={\"a\": \"numerical\", \"b\": \"numerical\"},\n )\n clf.inputs[0].shape = (2,)\n clf.outputs[0].in_blocks[0].shape = (10,)\n init_hp = task_specific.STRUCTURED_DATA_REGRESSOR[0]\n hp = keras_tuner.HyperParameters()\n hp.values = copy.copy(init_hp)\n\n clf.tuner.hypermodel.build(hp)\n\n assert set(init_hp.keys()) == set(hp._hps.keys())\n" }, { "path": "autokeras/utils/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "autokeras/utils/data_utils.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport numpy as np\nimport tree\nfrom keras import ops\n\n\ndef split_dataset(dataset, validation_split):\n \"\"\"Split nested numpy arrays into training and validation.\n\n # Arguments\n dataset: A nested structure of numpy arrays.\n validation_split: Float. The split ratio for the validation set.\n\n # Raises\n ValueError: If the dataset provided is too small to be split.\n\n # Returns\n A tuple of two nested structures.\n The training set and the validation set.\n \"\"\"\n # Get the number of instances from the first array's shape\n first_shape = tree.flatten(\n tree.map_structure(lambda x: np.array(x.shape), dataset)\n )[0]\n num_instances = first_shape[0]\n if num_instances < 2:\n raise ValueError(\n \"The dataset should at least contain 2 instances to be split.\"\n )\n validation_set_size = min(\n max(int(num_instances * validation_split), 1), num_instances - 1\n )\n train_set_size = num_instances - validation_set_size\n # Split each array in the nested structure\n train_dataset = tree.map_structure(lambda x: x[:train_set_size], dataset)\n validation_dataset = tree.map_structure(\n lambda x: x[train_set_size:], dataset\n )\n return train_dataset, validation_dataset\n\n\ndef cast_to_float32(tensor):\n if keras.backend.standardize_dtype(tensor.dtype) == \"float32\":\n return tensor\n return ops.cast(tensor, \"float32\") # pragma: no cover\n\n\ndef dataset_shape(dataset):\n \"\"\"Recursively get shapes from a nested structure of numpy arrays.\n\n # Arguments\n nested_arrays: A nested structure (dict, list, tuple) containing numpy\n arrays.\n\n # Returns\n The same nested structure with shapes instead of arrays.\n \"\"\"\n return tree.map_structure(lambda x: np.array(x.shape), dataset)\n" }, { "path": "autokeras/utils/data_utils_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport numpy as np\nimport pytest\n\nfrom autokeras.utils import data_utils\n\n\ndef test_split_data_has_one_batch_error():\n with pytest.raises(ValueError) as info:\n data_utils.split_dataset(np.array([1]), 0.2)\n assert \"The dataset should at least contain 2\" in str(info.value)\n" }, { "path": "autokeras/utils/io_utils.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport json\n\nfrom keras_tuner.engine import hyperparameters\n\nWHITELIST_FORMATS = (\".bmp\", \".gif\", \".jpeg\", \".jpg\", \".png\")\n\n\ndef save_json(path, obj):\n with open(path, \"w\") as f:\n json.dump(obj, f)\n\n\ndef load_json(path):\n with open(path, \"r\") as f:\n return json.load(f)\n\n\ndef deserialize_block_arg(arg):\n if isinstance(arg, dict):\n return hyperparameters.deserialize(arg)\n return arg\n\n\ndef serialize_block_arg(arg):\n if isinstance(arg, hyperparameters.HyperParameter):\n return hyperparameters.serialize(arg)\n return arg\n" }, { "path": "autokeras/utils/layer_utils.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom keras import layers\n\n\ndef get_global_average_pooling(shape):\n return [\n layers.GlobalAveragePooling1D,\n layers.GlobalAveragePooling2D,\n layers.GlobalAveragePooling3D,\n ][len(shape) - 3]\n\n\ndef get_global_max_pooling(shape):\n return [\n layers.GlobalMaxPool1D,\n layers.GlobalMaxPool2D,\n layers.GlobalMaxPool3D,\n ][len(shape) - 3]\n\n\ndef get_max_pooling(shape):\n return [\n layers.MaxPool1D,\n layers.MaxPool2D,\n layers.MaxPool3D,\n ][len(shape) - 3]\n\n\ndef get_conv(shape):\n return [layers.Conv1D, layers.Conv2D, layers.Conv3D][len(shape) - 3]\n\n\ndef get_sep_conv(shape):\n return [ # pragma: no cover\n layers.SeparableConv1D,\n layers.SeparableConv2D,\n layers.Conv3D,\n ][len(shape) - 3]\n" }, { "path": "autokeras/utils/types.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom typing import Callable\nfrom typing import Dict\nfrom typing import List\nfrom typing import Union\n\nimport numpy as np\nfrom keras.losses import Loss\nfrom keras.metrics import Metric\n\nDatasetType = Union[np.ndarray]\nLossType = Union[str, Callable, Loss]\nAcceptableMetric = Union[str, Callable, Metric]\nMetricsType = Union[\n List[AcceptableMetric],\n List[List[AcceptableMetric]],\n Dict[str, AcceptableMetric],\n]\n" }, { "path": "autokeras/utils/utils.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport re\n\nimport keras\nimport keras_tuner\nimport tree\n\n# Collect OOM exceptions from available libraries\noom_exceptions = []\ntry:\n import tensorflow as tf\n\n oom_exceptions.append(tf.errors.ResourceExhaustedError) # pragma: no cover\nexcept ImportError: # pragma: no cover\n pass # pragma: no cover\n\ntry:\n import torch\n\n oom_exceptions.append(torch.cuda.OutOfMemoryError) # pragma: no cover\nexcept ImportError: # pragma: no cover\n pass # pragma: no cover\n\ntry:\n import jax\n\n oom_exceptions.append(jax.errors.ResourceExhaustedError) # pragma: no cover\nexcept (ImportError, AttributeError): # pragma: no cover\n pass # pragma: no cover\n\noom_exceptions = tuple(oom_exceptions)\n\n\ndef validate_num_inputs(inputs, num):\n inputs = tree.flatten(inputs)\n if not len(inputs) == num:\n raise ValueError(\n \"Expected {num} elements in the inputs list \"\n \"but received {len} inputs.\".format(num=num, len=len(inputs))\n )\n\n\ndef to_snake_case(name):\n intermediate = re.sub(\"(.)([A-Z][a-z0-9]+)\", r\"\\1_\\2\", name)\n insecure = re.sub(\"([a-z])([A-Z])\", r\"\\1_\\2\", intermediate).lower()\n return insecure\n\n\ndef contain_instance(instance_list, instance_type):\n return any(\n [isinstance(instance, instance_type) for instance in instance_list]\n )\n\n\ndef evaluate_with_adaptive_batch_size(\n model, batch_size, verbose=1, **fit_kwargs\n):\n return run_with_adaptive_batch_size(\n batch_size,\n lambda x, validation_data, **kwargs: model.evaluate(\n x, verbose=verbose, **kwargs\n ),\n **fit_kwargs,\n )\n\n\ndef predict_with_adaptive_batch_size(\n model, batch_size, verbose=1, **fit_kwargs\n):\n return run_with_adaptive_batch_size(\n batch_size,\n lambda x, validation_data, **kwargs: model.predict(\n x, verbose=verbose, **kwargs\n ),\n **fit_kwargs,\n )\n\n\ndef fit_with_adaptive_batch_size(model, batch_size, **fit_kwargs):\n history = run_with_adaptive_batch_size(\n batch_size, lambda **kwargs: model.fit(**kwargs), **fit_kwargs\n )\n return model, history\n\n\ndef run_with_adaptive_batch_size(batch_size, func, **fit_kwargs):\n validation_data = None\n if \"validation_data\" in fit_kwargs:\n validation_data = fit_kwargs.pop(\"validation_data\")\n while batch_size > 0:\n try:\n history = func(\n validation_data=validation_data,\n batch_size=batch_size,\n **fit_kwargs,\n )\n break\n except oom_exceptions as e: # pragma: no cover\n if batch_size == 1: # pragma: no cover\n raise e # pragma: no cover\n batch_size //= 2 # pragma: no cover\n print( # pragma: no cover\n \"Not enough memory, reduce batch size to {batch_size}.\".format(\n batch_size=batch_size\n )\n )\n return history\n\n\ndef get_hyperparameter(value, hp, dtype):\n if value is None:\n return hp\n return value\n\n\ndef add_to_hp(hp, hps, name=None):\n \"\"\"Add the HyperParameter (self) to the HyperParameters.\n\n # Arguments\n hp: keras_tuner.HyperParameters.\n name: String. If left unspecified, the hp name is used.\n \"\"\"\n if not isinstance(hp, keras_tuner.engine.hyperparameters.HyperParameter):\n return hp\n kwargs = hp.get_config()\n if name is None:\n name = hp.name\n kwargs.pop(\"conditions\")\n kwargs.pop(\"name\")\n class_name = hp.__class__.__name__\n func = getattr(hps, class_name)\n return func(name=name, **kwargs)\n\n\ndef serialize_keras_object(obj):\n return keras.utils.serialize_keras_object(obj) # pragma: no cover\n\n\ndef deserialize_keras_object(config, module_objects=None, custom_objects=None):\n return keras.utils.deserialize_keras_object(\n config, custom_objects, module_objects\n )\n" }, { "path": "autokeras/utils/utils_test.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport pytest\nfrom keras_tuner.engine import hyperparameters\n\nfrom autokeras.utils import utils\n\n\ndef test_validate_num_inputs_error():\n with pytest.raises(ValueError) as info:\n utils.validate_num_inputs([1, 2, 3], 2)\n\n assert \"Expected 2 elements in the inputs list\" in str(info.value)\n\n\ndef test_get_hyperparameter_with_none_return_hp():\n hp = utils.get_hyperparameter(\n None, hyperparameters.Choice(\"hp\", [10, 20]), int\n )\n assert isinstance(hp, hyperparameters.Choice)\n\n\ndef test_get_hyperparameter_with_int_return_int():\n value = utils.get_hyperparameter(\n 10, hyperparameters.Choice(\"hp\", [10, 20]), int\n )\n assert isinstance(value, int)\n assert value == 10\n\n\ndef test_get_hyperparameter_with_hp_return_same():\n hp = utils.get_hyperparameter(\n hyperparameters.Choice(\"hp\", [10, 30]),\n hyperparameters.Choice(\"hp\", [10, 20]),\n int,\n )\n assert isinstance(hp, hyperparameters.Choice)\n" }, { "path": "benchmark/README.md", "content": "Usage: python benchmark/run.py structured_data_classification report.csv\n" }, { "path": "benchmark/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "benchmark/datasets/titanic_test.csv", "content": "sex,age,n_siblings_spouses,parch,fare,class,deck,embark_town,alone,survived\nfemale,15,0,0,7.225,3,?,C,True,1\nfemale,1,0,2,15.7417,3,?,C,False,1\nmale,20,1,1,15.7417,3,?,C,False,1\nfemale,19,1,1,15.7417,3,?,C,False,1\nmale,33,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nfemale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,12,1,0,11.2417,3,?,C,False,1\nfemale,14,1,0,11.2417,3,?,C,False,1\nfemale,29,0,0,7.925,3,?,S,True,0\nmale,28,0,0,8.05,3,?,S,True,0\nfemale,18,0,0,7.775,3,?,S,True,1\nfemale,26,0,0,7.8542,3,?,S,True,1\nmale,21,0,0,7.8542,3,?,S,True,0\nmale,41,0,0,7.125,3,?,S,True,0\nmale,39,0,0,7.925,3,?,S,True,1\nmale,21,0,0,7.8,3,?,S,True,0\nmale,28.5,0,0,7.2292,3,?,C,True,0\nfemale,22,0,0,7.75,3,?,S,True,1\nmale,61,0,0,6.2375,3,?,S,True,0\nmale,,1,0,15.5,3,?,Q,False,0\nmale,,0,0,7.8292,3,?,Q,True,0\nfemale,,1,0,15.5,3,?,Q,False,1\nmale,,0,0,7.7333,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,23,0,0,9.225,3,?,S,True,0\nfemale,,0,0,7.75,3,?,Q,True,0\nfemale,,0,0,7.75,3,?,Q,True,1\nfemale,,0,0,7.8792,3,?,Q,True,1\nfemale,22,0,0,7.775,3,?,S,True,1\nmale,,0,0,7.75,3,?,Q,True,1\nfemale,,0,0,7.8292,3,?,Q,True,1\nmale,9,0,1,3.1708,3,?,S,False,1\nmale,28,0,0,22.525,3,?,S,True,0\nmale,42,0,1,8.4042,3,?,S,False,0\nmale,,0,0,7.3125,3,?,S,True,0\nfemale,31,0,0,7.8542,3,?,S,True,0\nmale,28,0,0,7.8542,3,?,S,True,0\nmale,32,0,0,7.775,3,?,S,True,1\nmale,20,0,0,9.225,3,?,S,True,0\nfemale,23,0,0,8.6625,3,?,S,True,0\nfemale,20,0,0,8.6625,3,?,S,True,0\nmale,20,0,0,8.6625,3,?,S,True,0\nmale,16,0,0,9.2167,3,?,S,True,0\nfemale,31,0,0,8.6833,3,?,S,True,1\nfemale,,0,0,7.6292,3,?,Q,True,0\nmale,2,3,1,21.075,3,?,S,False,0\nmale,6,3,1,21.075,3,?,S,False,0\nfemale,3,3,1,21.075,3,?,S,False,0\nfemale,8,3,1,21.075,3,?,S,False,0\nfemale,29,0,4,21.075,3,?,S,False,0\nmale,1,4,1,39.6875,3,?,S,False,0\nmale,7,4,1,39.6875,3,?,S,False,0\nmale,2,4,1,39.6875,3,?,S,False,0\nmale,16,4,1,39.6875,3,?,S,False,0\nmale,14,4,1,39.6875,3,?,S,False,0\nfemale,41,0,5,39.6875,3,?,S,False,0\nmale,21,0,0,8.6625,3,?,S,True,0\nmale,19,0,0,14.5,3,?,S,True,0\nmale,,0,0,8.7125,3,?,C,True,0\nmale,32,0,0,7.8958,3,?,S,True,0\nmale,0.75,1,1,13.775,3,?,S,False,0\nfemale,3,1,1,13.775,3,?,S,False,0\nfemale,26,0,2,13.775,3,?,S,False,0\nmale,,0,0,7,3,?,S,True,0\nmale,,0,0,7.775,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,21,0,0,7.925,3,?,S,True,0\nmale,25,0,0,7.925,3,?,S,True,0\nmale,22,0,0,7.25,3,?,S,True,0\nmale,25,1,0,7.775,3,?,S,False,1\nmale,,1,1,22.3583,3,?,C,False,1\nfemale,,1,1,22.3583,3,F E69,C,False,1\nfemale,,0,2,22.3583,3,?,C,False,1\nfemale,,0,0,8.1375,3,?,Q,True,0\nmale,24,0,0,8.05,3,?,S,True,0\nfemale,28,0,0,7.8958,3,?,S,True,0\nmale,19,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,25,1,0,7.775,3,?,S,False,0\nfemale,18,0,0,7.775,3,?,S,True,0\nmale,32,0,0,8.05,3,E10,S,True,1\nmale,,0,0,7.8958,3,?,S,True,0\nmale,17,0,0,8.6625,3,?,S,True,0\nmale,24,0,0,8.6625,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nfemale,,0,0,8.1125,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nmale,,0,0,7.25,3,?,S,True,0\nmale,38,0,0,7.8958,3,?,S,True,0\nmale,21,0,0,8.05,3,?,S,True,0\nmale,10,4,1,29.125,3,?,Q,False,0\nmale,4,4,1,29.125,3,?,Q,False,0\nmale,7,4,1,29.125,3,?,Q,False,0\nmale,2,4,1,29.125,3,?,Q,False,0\nmale,8,4,1,29.125,3,?,Q,False,0\nfemale,39,0,5,29.125,3,?,Q,False,0\nfemale,22,0,0,39.6875,3,?,S,True,0\nmale,35,0,0,7.125,3,?,S,True,0\nfemale,,0,0,7.7208,3,?,Q,True,1\nmale,,0,0,14.5,3,?,S,True,0\nfemale,,0,0,14.5,3,?,S,True,0\nmale,50,1,0,14.5,3,?,S,False,0\nfemale,47,1,0,14.5,3,?,S,False,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.775,3,?,S,True,0\nfemale,2,1,1,20.2125,3,?,S,False,0\nmale,18,1,1,20.2125,3,?,S,False,0\nfemale,41,0,2,20.2125,3,?,S,False,0\nfemale,,0,0,8.05,3,?,S,True,1\nmale,50,0,0,8.05,3,?,S,True,0\nmale,16,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,1\nmale,,0,0,24.15,3,?,Q,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nmale,25,0,0,7.225,3,?,C,True,0\nmale,,0,0,7.225,3,?,C,True,0\nmale,,0,0,7.7292,3,?,Q,True,0\nmale,,0,0,7.575,3,?,S,True,0\nmale,38.5,0,0,7.25,3,?,S,True,0\nmale,,8,2,69.55,3,?,S,False,0\nmale,14.5,8,2,69.55,3,?,S,False,0\nfemale,,8,2,69.55,3,?,S,False,0\nfemale,,8,2,69.55,3,?,S,False,0\nfemale,,8,2,69.55,3,?,S,False,0\nfemale,,8,2,69.55,3,?,S,False,0\nmale,,8,2,69.55,3,?,S,False,0\nmale,,8,2,69.55,3,?,S,False,0\nmale,,8,2,69.55,3,?,S,False,0\nmale,,1,9,69.55,3,?,S,False,0\nfemale,,1,9,69.55,3,?,S,False,0\nmale,24,0,0,9.325,3,?,S,True,0\nfemale,21,0,0,7.65,3,?,S,True,1\nmale,39,0,0,7.925,3,?,S,True,0\nmale,,2,0,21.6792,3,?,C,False,0\nmale,,2,0,21.6792,3,?,C,False,0\nmale,,2,0,21.6792,3,?,C,False,0\nfemale,1,1,1,16.7,3,G6,S,False,1\nfemale,24,0,2,16.7,3,G6,S,False,1\nfemale,4,1,1,16.7,3,G6,S,False,1\nmale,25,0,0,9.5,3,?,S,True,1\nmale,20,0,0,8.05,3,?,S,True,0\nmale,24.5,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.725,3,?,Q,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,29,0,0,9.5,3,?,S,True,1\nmale,,0,0,15.1,3,?,S,True,0\nfemale,,0,0,7.7792,3,?,Q,True,1\nmale,,0,0,8.05,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,22,0,0,7.2292,3,?,C,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,40,0,0,7.8958,3,?,S,True,0\nmale,21,0,0,7.925,3,?,S,True,0\nfemale,18,0,0,7.4958,3,?,S,True,1\nmale,4,3,2,27.9,3,?,S,False,0\nmale,10,3,2,27.9,3,?,S,False,0\nfemale,9,3,2,27.9,3,?,S,False,0\nfemale,2,3,2,27.9,3,?,S,False,0\nmale,40,1,4,27.9,3,?,S,False,0\nfemale,45,1,4,27.9,3,?,S,False,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,,0,0,8.6625,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nfemale,,0,0,7.7333,3,?,Q,True,1\nmale,19,0,0,7.65,3,F G73,S,True,0\nmale,30,0,0,8.05,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,32,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,33,0,0,8.6625,3,?,C,True,0\nfemale,23,0,0,7.55,3,?,S,True,1\nmale,21,0,0,8.05,3,?,S,True,0\nmale,60.5,0,0,?,3,?,S,True,0\nmale,19,0,0,7.8958,3,?,S,True,0\nfemale,22,0,0,9.8375,3,?,S,True,0\nmale,31,0,0,7.925,3,?,S,True,1\nmale,27,0,0,8.6625,3,?,S,True,0\nfemale,2,0,1,10.4625,3,G6,S,False,0\nfemale,29,1,1,10.4625,3,G6,S,False,0\nmale,16,0,0,8.05,3,?,S,True,1\nmale,44,0,0,7.925,3,?,S,True,1\nmale,25,0,0,7.05,3,?,S,True,0\nmale,74,0,0,7.775,3,?,S,True,0\nmale,14,0,0,9.225,3,?,S,True,1\nmale,24,0,0,7.7958,3,?,S,True,0\nmale,25,0,0,7.7958,3,?,S,True,1\nmale,34,0,0,8.05,3,?,S,True,0\nmale,0.4167,0,1,8.5167,3,?,C,False,1\nmale,,1,0,6.4375,3,?,C,False,0\nmale,,0,0,6.4375,3,?,C,True,0\nmale,,0,0,7.225,3,?,C,True,0\nfemale,16,1,1,8.5167,3,?,C,False,1\nmale,,0,0,8.05,3,?,S,True,0\nmale,,1,0,16.1,3,?,S,False,0\nfemale,,1,0,16.1,3,?,S,False,1\nmale,32,0,0,7.925,3,?,S,True,0\nmale,,0,0,7.75,3,F38,Q,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,30.5,0,0,8.05,3,?,S,True,0\nmale,44,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nmale,25,0,0,0,3,?,S,True,1\nmale,,0,0,7.2292,3,?,C,True,0\nmale,7,1,1,15.2458,3,?,C,False,1\nfemale,9,1,1,15.2458,3,?,C,False,1\nfemale,29,0,2,15.2458,3,?,C,False,1\nmale,36,0,0,7.8958,3,?,S,True,0\nfemale,18,0,0,9.8417,3,?,S,True,1\nfemale,63,0,0,9.5875,3,?,S,True,1\nmale,,1,1,14.5,3,?,S,False,0\nmale,11.5,1,1,14.5,3,?,S,False,0\nmale,40.5,0,2,14.5,3,?,S,False,0\nfemale,10,0,2,24.15,3,?,S,False,0\nmale,36,1,1,24.15,3,?,S,False,0\nfemale,30,1,1,24.15,3,?,S,False,0\nmale,,0,0,9.5,3,?,S,True,0\nmale,33,0,0,9.5,3,?,S,True,0\nmale,28,0,0,9.5,3,?,S,True,0\nmale,28,0,0,9.5,3,?,S,True,0\nmale,47,0,0,9,3,?,S,True,0\nfemale,18,2,0,18,3,?,S,False,0\nmale,31,3,0,18,3,?,S,False,0\nmale,16,2,0,18,3,?,S,False,0\nfemale,31,1,0,18,3,?,S,False,0\nmale,22,0,0,7.225,3,?,C,True,1\nmale,20,0,0,7.8542,3,?,S,True,0\nfemale,14,0,0,7.8542,3,?,S,True,0\nmale,22,0,0,7.8958,3,?,S,True,0\nmale,22,0,0,9,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.55,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,32.5,0,0,9.5,3,?,S,True,0\nfemale,38,0,0,7.2292,3,?,C,True,1\nmale,51,0,0,7.75,3,?,S,True,0\nmale,18,1,0,6.4958,3,?,S,False,0\nmale,21,1,0,6.4958,3,?,S,False,0\nfemale,47,1,0,7,3,?,S,False,1\nmale,,0,0,8.7125,3,?,S,True,0\nmale,,0,0,7.55,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,28.5,0,0,16.1,3,?,S,True,0\nmale,21,0,0,7.25,3,?,S,True,0\nmale,27,0,0,8.6625,3,?,S,True,0\nmale,,0,0,7.25,3,?,S,True,0\nmale,36,0,0,9.5,3,?,S,True,0\nmale,27,1,0,14.4542,3,?,C,False,0\nfemale,15,1,0,14.4542,3,?,C,False,1\nmale,45.5,0,0,7.225,3,?,C,True,0\nmale,,0,0,7.225,3,?,C,True,0\nmale,,0,0,14.4583,3,?,C,True,0\nfemale,14.5,1,0,14.4542,3,?,C,False,0\nfemale,,1,0,14.4542,3,?,C,False,0\nmale,26.5,0,0,7.225,3,?,C,True,0\nmale,27,0,0,7.225,3,?,C,True,0\nmale,29,0,0,7.875,3,?,S,True,0\n" }, { "path": "benchmark/datasets/titanic_train.csv", "content": "sex,age,n_siblings_spouses,parch,fare,class,deck,embark_town,alone,survived\nfemale,29,0,0,211.3375,1,B5,S,True,1\nmale,0.9167,1,2,151.55,1,C22 C26,S,False,1\nfemale,2,1,2,151.55,1,C22 C26,S,False,0\nmale,30,1,2,151.55,1,C22 C26,S,False,0\nfemale,25,1,2,151.55,1,C22 C26,S,False,0\nmale,48,0,0,26.55,1,E12,S,True,1\nfemale,63,1,0,77.9583,1,D7,S,False,1\nmale,39,0,0,0.0,1,A36,S,True,0\nfemale,53,2,0,51.4792,1,C101,S,False,1\nmale,71,0,0,49.5042,1,?,C,True,0\nmale,47,1,0,227.525,1,C62 C64,C,False,0\nfemale,18,1,0,227.525,1,C62 C64,C,False,1\nfemale,24,0,0,69.3,1,B35,C,True,1\nfemale,26,0,0,78.85,1,?,S,True,1\nmale,80,0,0,30.0,1,A23,S,True,1\nmale,,0,0,25.925,1,?,S,True,0\nmale,24,0,1,247.5208,1,B58 B60,C,False,0\nfemale,50,0,1,247.5208,1,B58 B60,C,False,1\nfemale,32,0,0,76.2917,1,D15,C,True,1\nmale,36,0,0,75.2417,1,C6,C,True,0\nmale,37,1,1,52.5542,1,D35,S,False,1\nfemale,47,1,1,52.5542,1,D35,S,False,1\nmale,26,0,0,30.0,1,C148,C,True,1\nfemale,42,0,0,227.525,1,?,C,True,1\nfemale,29,0,0,221.7792,1,C97,S,True,1\nmale,25,0,0,26.0,1,?,C,True,0\nmale,25,1,0,91.0792,1,B49,C,False,1\nfemale,19,1,0,91.0792,1,B49,C,False,1\nfemale,35,0,0,135.6333,1,C99,S,True,1\nmale,28,0,0,26.55,1,C52,S,True,1\nmale,45,0,0,35.5,1,T,S,True,0\nmale,40,0,0,31.0,1,A31,C,True,1\nfemale,30,0,0,164.8667,1,C7,S,True,1\nfemale,58,0,0,26.55,1,C103,S,True,1\nmale,42,0,0,26.55,1,D22,S,True,0\nfemale,45,0,0,262.375,1,?,C,True,1\nfemale,22,0,1,55.0,1,E33,S,False,1\nmale,,0,0,26.55,1,?,S,True,1\nmale,41,0,0,30.5,1,A21,S,True,0\nmale,48,0,0,50.4958,1,B10,C,True,0\nmale,,0,0,39.6,1,?,C,True,0\nfemale,44,0,0,27.7208,1,B4,C,True,1\nfemale,59,2,0,51.4792,1,C101,S,False,1\nfemale,60,0,0,76.2917,1,D15,C,True,1\nfemale,41,0,0,134.5,1,E40,C,True,1\nmale,45,0,0,26.55,1,B38,S,True,0\nmale,,0,0,31.0,1,?,S,True,0\nmale,42,0,0,26.2875,1,E24,S,True,1\nfemale,53,0,0,27.4458,1,?,C,True,1\nmale,36,0,1,512.3292,1,B51 B53 B55,C,False,1\nfemale,58,0,1,512.3292,1,B51 B53 B55,C,False,1\nmale,33,0,0,5.0,1,B51 B53 B55,S,True,0\nmale,28,0,0,47.1,1,?,S,True,0\nmale,17,0,0,47.1,1,?,S,True,0\nmale,11,1,2,120.0,1,B96 B98,S,False,1\nfemale,14,1,2,120.0,1,B96 B98,S,False,1\nmale,36,1,2,120.0,1,B96 B98,S,False,1\nfemale,36,1,2,120.0,1,B96 B98,S,False,1\nmale,49,0,0,26.0,1,?,S,True,0\nfemale,,0,0,27.7208,1,?,C,True,1\nmale,36,1,0,78.85,1,C46,S,False,0\nfemale,76,1,0,78.85,1,C46,S,False,1\nmale,46,1,0,61.175,1,E31,S,False,0\nfemale,47,1,0,61.175,1,E31,S,False,1\nmale,27,1,0,53.1,1,E8,S,False,1\nfemale,33,1,0,53.1,1,E8,S,False,1\nfemale,36,0,0,262.375,1,B61,C,True,1\nfemale,30,0,0,86.5,1,B77,S,True,1\nmale,45,0,0,29.7,1,A9,C,True,1\nfemale,,0,1,55.0,1,E33,S,False,1\nmale,,0,0,0.0,1,?,S,True,0\nmale,27,1,0,136.7792,1,C89,C,False,0\nfemale,26,1,0,136.7792,1,C89,C,False,1\nfemale,22,0,0,151.55,1,?,S,True,1\nmale,,0,0,52.0,1,A14,S,True,0\nmale,47,0,0,25.5875,1,E58,S,True,0\nfemale,39,1,1,83.1583,1,E49,C,False,1\nmale,37,1,1,83.1583,1,E52,C,False,0\nfemale,64,0,2,83.1583,1,E45,C,False,1\nfemale,55,2,0,25.7,1,C101,S,False,1\nmale,,0,0,26.55,1,?,S,True,0\nmale,70,1,1,71.0,1,B22,S,False,0\nfemale,36,0,2,71.0,1,B22,S,False,1\nfemale,64,1,1,26.55,1,B26,S,False,1\nmale,39,1,0,71.2833,1,C85,C,False,0\nfemale,38,1,0,71.2833,1,C85,C,False,1\nmale,51,0,0,26.55,1,E17,S,True,1\nmale,27,0,0,30.5,1,?,S,True,1\nfemale,33,0,0,151.55,1,?,S,True,1\nmale,31,1,0,52.0,1,B71,S,False,0\nfemale,27,1,2,52.0,1,B71,S,False,1\nmale,31,1,0,57.0,1,B20,S,False,1\nfemale,17,1,0,57.0,1,B20,S,False,1\nmale,53,1,1,81.8583,1,A34,S,False,1\nmale,4,0,2,81.8583,1,A34,S,False,1\nfemale,54,1,1,81.8583,1,A34,S,False,1\nmale,50,1,0,106.425,1,C86,C,False,0\nfemale,27,1,1,247.5208,1,B58 B60,C,False,1\nfemale,48,1,0,106.425,1,C86,C,False,1\nfemale,48,1,0,39.6,1,A16,C,False,1\nmale,49,1,0,56.9292,1,A20,C,False,1\nmale,39,0,0,29.7,1,A18,C,True,0\nfemale,23,0,1,83.1583,1,C54,C,False,1\nfemale,38,0,0,227.525,1,C45,C,True,1\nfemale,54,1,0,78.2667,1,D20,C,False,1\nfemale,36,0,0,31.6792,1,A29,C,True,0\nmale,,0,0,221.7792,1,C95,S,True,0\nfemale,,0,0,31.6833,1,?,S,True,1\nfemale,,0,0,110.8833,1,?,C,True,1\nmale,36,0,0,26.3875,1,E25,S,True,1\nmale,30,0,0,27.75,1,C111,C,True,0\nfemale,24,3,2,263.0,1,C23 C25 C27,S,False,1\nfemale,28,3,2,263.0,1,C23 C25 C27,S,False,1\nfemale,23,3,2,263.0,1,C23 C25 C27,S,False,1\nmale,19,3,2,263.0,1,C23 C25 C27,S,False,0\nmale,64,1,4,263.0,1,C23 C25 C27,S,False,0\nfemale,60,1,4,263.0,1,C23 C25 C27,S,False,1\nfemale,30,0,0,56.9292,1,E36,C,True,1\nmale,,0,0,26.55,1,D34,S,True,0\nmale,50,2,0,133.65,1,?,S,False,1\nmale,43,1,0,27.7208,1,D40,C,False,1\nfemale,,1,0,133.65,1,?,S,False,1\nfemale,22,0,2,49.5,1,B39,C,False,1\nmale,60,1,1,79.2,1,B41,C,False,1\nfemale,48,1,1,79.2,1,B41,C,False,1\nmale,,0,0,0.0,1,B102,S,True,0\nmale,37,1,0,53.1,1,C123,S,False,0\nfemale,35,1,0,53.1,1,C123,S,False,1\nmale,47,0,0,38.5,1,E63,S,True,0\nfemale,35,0,0,211.5,1,C130,C,True,1\nfemale,22,0,1,59.4,1,?,C,False,1\nfemale,45,0,1,59.4,1,?,C,False,1\nmale,24,0,0,79.2,1,B86,C,True,0\nmale,49,1,0,89.1042,1,C92,C,False,1\nfemale,,1,0,89.1042,1,C92,C,False,1\nmale,71,0,0,34.6542,1,A5,C,True,0\nmale,53,0,0,28.5,1,C51,C,True,1\nfemale,19,0,0,30.0,1,B42,S,True,1\nmale,38,0,1,153.4625,1,C91,S,False,0\nfemale,58,0,1,153.4625,1,C125,S,False,1\nmale,23,0,1,63.3583,1,D10 D12,C,False,1\nfemale,45,0,1,63.3583,1,D10 D12,C,False,1\nmale,46,0,0,79.2,1,B82 B84,C,True,0\nmale,25,1,0,55.4417,1,E50,C,False,1\nfemale,25,1,0,55.4417,1,E50,C,False,1\nmale,48,1,0,76.7292,1,D33,C,False,1\nfemale,49,1,0,76.7292,1,D33,C,False,1\nmale,,0,0,42.4,1,?,S,True,0\nmale,45,1,0,83.475,1,C83,S,False,0\nfemale,35,1,0,83.475,1,C83,S,False,1\nmale,40,0,0,0.0,1,B94,S,True,0\nmale,27,0,0,76.7292,1,D49,C,True,1\nmale,,0,0,30.0,1,D45,S,True,1\nfemale,24,0,0,83.1583,1,C54,C,True,1\nmale,55,1,1,93.5,1,B69,S,False,0\nfemale,52,1,1,93.5,1,B69,S,False,1\nmale,42,0,0,42.5,1,B11,S,True,0\nmale,,0,0,51.8625,1,E46,S,True,0\nmale,55,0,0,50.0,1,C39,S,True,0\nfemale,16,0,1,57.9792,1,B18,C,False,1\nfemale,44,0,1,57.9792,1,B18,C,False,1\nfemale,51,1,0,77.9583,1,D11,S,False,1\nmale,42,1,0,52.0,1,?,S,False,0\nfemale,35,1,0,52.0,1,?,S,False,1\nmale,35,0,0,26.55,1,?,C,True,1\nmale,38,1,0,90.0,1,C93,S,False,1\nmale,,0,0,30.6958,1,?,C,True,0\nfemale,35,1,0,90.0,1,C93,S,False,1\nfemale,38,0,0,80.0,1,B28,?,True,1\nfemale,50,0,0,28.7125,1,C49,C,True,0\nmale,49,0,0,0.0,1,B52 B54 B56,S,True,1\nmale,46,0,0,26.0,1,?,S,True,0\nmale,50,0,0,26.0,1,E60,S,True,0\nmale,32.5,0,0,211.5,1,C132,C,True,0\nmale,58,0,0,29.7,1,B37,C,True,0\nmale,41,1,0,51.8625,1,D21,S,False,0\nfemale,,1,0,51.8625,1,D21,S,False,1\nmale,42,1,0,52.5542,1,D19,S,False,1\nfemale,45,1,0,52.5542,1,D19,S,False,1\nmale,,0,0,26.55,1,C124,S,True,0\nfemale,39,0,0,211.3375,1,?,S,True,1\nfemale,49,0,0,25.9292,1,D17,S,True,1\nfemale,30,0,0,106.425,1,?,C,True,1\nmale,35,0,0,512.3292,1,B101,C,True,1\nmale,,0,0,27.7208,1,?,C,True,0\nmale,42,0,0,26.55,1,?,S,True,0\nfemale,55,0,0,27.7208,1,?,C,True,1\nfemale,16,0,1,39.4,1,D28,S,False,1\nfemale,51,0,1,39.4,1,D28,S,False,1\nmale,29,0,0,30.0,1,D6,S,True,0\nfemale,21,0,0,77.9583,1,D9,S,True,1\nmale,30,0,0,45.5,1,?,S,True,0\nfemale,58,0,0,146.5208,1,B80,C,True,1\nfemale,15,0,1,211.3375,1,B5,S,False,1\nmale,30,0,0,26.0,1,C106,S,True,0\nfemale,16,0,0,86.5,1,B79,S,True,1\nmale,,0,0,29.7,1,C47,C,True,1\nmale,19,1,0,53.1,1,D30,S,False,0\nfemale,18,1,0,53.1,1,D30,S,False,1\nfemale,24,0,0,49.5042,1,C90,C,True,1\nmale,46,0,0,75.2417,1,C6,C,True,0\nmale,54,0,0,51.8625,1,E46,S,True,0\nmale,36,0,0,26.2875,1,E25,S,True,1\nmale,28,1,0,82.1708,1,?,C,False,0\nfemale,,1,0,82.1708,1,?,C,False,1\nmale,65,0,0,26.55,1,E38,S,True,0\nmale,44,2,0,90.0,1,C78,Q,False,0\nfemale,33,1,0,90.0,1,C78,Q,False,1\nfemale,37,1,0,90.0,1,C78,Q,False,1\nmale,30,1,0,57.75,1,C78,C,False,1\nmale,55,0,0,30.5,1,C30,S,True,0\nmale,47,0,0,42.4,1,?,S,True,0\nmale,37,0,1,29.7,1,C118,C,False,0\nfemale,31,1,0,113.275,1,D36,C,False,1\nfemale,23,1,0,113.275,1,D36,C,False,1\nmale,58,0,2,113.275,1,D48,C,False,0\nfemale,19,0,2,26.2833,1,D47,S,False,1\nmale,64,0,0,26.0,1,?,S,True,0\nfemale,39,0,0,108.9,1,C105,C,True,1\nmale,,0,0,25.7417,1,?,C,True,1\nfemale,22,0,1,61.9792,1,B36,C,False,1\nmale,65,0,1,61.9792,1,B30,C,False,0\nmale,28.5,0,0,27.7208,1,D43,C,True,0\nmale,,0,0,0.0,1,?,S,True,0\nmale,45.5,0,0,28.5,1,C124,S,True,0\nmale,23,0,0,93.5,1,B24,S,True,0\nmale,29,1,0,66.6,1,C2,S,False,0\nfemale,22,1,0,66.6,1,C2,S,False,1\nmale,18,1,0,108.9,1,C65,C,False,0\nfemale,17,1,0,108.9,1,C65,C,False,1\nfemale,30,0,0,93.5,1,B73,S,True,1\nmale,52,0,0,30.5,1,C104,S,True,1\nmale,47,0,0,52.0,1,C110,S,True,0\nfemale,56,0,1,83.1583,1,C50,C,False,1\nmale,38,0,0,0.0,1,?,S,True,0\nmale,,0,0,39.6,1,?,S,True,1\nmale,22,0,0,135.6333,1,?,C,True,0\nmale,,0,0,227.525,1,?,C,True,0\nfemale,43,0,1,211.3375,1,B3,S,False,1\nmale,31,0,0,50.4958,1,A24,S,True,0\nmale,45,0,0,26.55,1,?,S,True,1\nmale,,0,0,50.0,1,A32,S,True,0\nfemale,33,0,0,27.7208,1,A11,C,True,1\nmale,46,0,0,79.2,1,?,C,True,0\nmale,36,0,0,40.125,1,A10,C,True,0\nfemale,33,0,0,86.5,1,B77,S,True,1\nmale,55,1,0,59.4,1,?,C,False,0\nfemale,54,1,0,59.4,1,?,C,False,1\nmale,33,0,0,26.55,1,?,S,True,0\nmale,13,2,2,262.375,1,B57 B59 B63 B66,C,False,1\nfemale,18,2,2,262.375,1,B57 B59 B63 B66,C,False,1\nfemale,21,2,2,262.375,1,B57 B59 B63 B66,C,False,1\nmale,61,1,3,262.375,1,B57 B59 B63 B66,C,False,0\nfemale,48,1,3,262.375,1,B57 B59 B63 B66,C,False,1\nmale,,0,0,30.5,1,C106,S,True,1\nfemale,24,0,0,69.3,1,B35,C,True,1\nmale,,0,0,26.0,1,?,S,True,1\nfemale,35,1,0,57.75,1,C28,C,False,1\nfemale,30,0,0,31.0,1,?,C,True,1\nmale,34,0,0,26.55,1,?,S,True,1\nfemale,40,0,0,153.4625,1,C125,S,True,1\nmale,35,0,0,26.2875,1,E24,S,True,1\nmale,50,1,0,55.9,1,E44,S,False,0\nfemale,39,1,0,55.9,1,E44,S,False,1\nmale,56,0,0,35.5,1,A26,C,True,1\nmale,28,0,0,35.5,1,A6,S,True,1\nmale,56,0,0,26.55,1,?,S,True,0\nmale,56,0,0,30.6958,1,A7,C,True,0\nmale,24,1,0,60.0,1,C31,S,False,0\nmale,,0,0,26.0,1,A19,S,True,0\nfemale,18,1,0,60.0,1,C31,S,False,1\nmale,24,1,0,82.2667,1,B45,S,False,1\nfemale,23,1,0,82.2667,1,B45,S,False,1\nmale,6,0,2,134.5,1,E34,C,False,1\nmale,45,1,1,134.5,1,E34,C,False,1\nfemale,40,1,1,134.5,1,E34,C,False,1\nmale,57,1,0,146.5208,1,B78,C,False,0\nfemale,,1,0,146.5208,1,B78,C,False,1\nmale,32,0,0,30.5,1,B50,C,True,1\nmale,62,0,0,26.55,1,C87,S,True,0\nmale,54,1,0,55.4417,1,C116,C,False,1\nfemale,43,1,0,55.4417,1,C116,C,False,1\nfemale,52,1,0,78.2667,1,D20,C,False,1\nmale,,0,0,27.7208,1,?,C,True,0\nfemale,62,0,0,80.0,1,B28,?,True,1\nmale,67,1,0,221.7792,1,C55 C57,S,False,0\nfemale,63,1,0,221.7792,1,C55 C57,S,False,0\nmale,61,0,0,32.3208,1,D50,S,True,0\nfemale,48,0,0,25.9292,1,D17,S,True,1\nfemale,18,0,2,79.65,1,E68,S,False,1\nmale,52,1,1,79.65,1,E67,S,False,0\nfemale,39,1,1,79.65,1,E67,S,False,1\nmale,48,1,0,52.0,1,C126,S,False,1\nfemale,,1,0,52.0,1,C126,S,False,1\nmale,49,1,1,110.8833,1,C68,C,False,0\nmale,17,0,2,110.8833,1,C70,C,False,1\nfemale,39,1,1,110.8833,1,C68,C,False,1\nfemale,,0,0,79.2,1,?,C,True,1\nmale,31,0,0,28.5375,1,C53,C,True,1\nmale,40,0,0,27.7208,1,?,C,True,0\nmale,61,0,0,33.5,1,B19,S,True,0\nmale,47,0,0,34.0208,1,D46,S,True,0\nfemale,35,0,0,512.3292,1,?,C,True,1\nmale,64,1,0,75.25,1,D37,C,False,0\nfemale,60,1,0,75.25,1,D37,C,False,1\nmale,60,0,0,26.55,1,?,S,True,0\nmale,54,0,1,77.2875,1,D26,S,False,0\nmale,21,0,1,77.2875,1,D26,S,False,0\nfemale,55,0,0,135.6333,1,C32,C,True,1\nfemale,31,0,2,164.8667,1,C7,S,False,1\nmale,57,1,1,164.8667,1,?,S,False,0\nfemale,45,1,1,164.8667,1,?,S,False,1\nmale,50,1,1,211.5,1,C80,C,False,0\nmale,27,0,2,211.5,1,C82,C,False,0\nfemale,50,1,1,211.5,1,C80,C,False,1\nfemale,21,0,0,26.55,1,?,S,True,1\nmale,51,0,1,61.3792,1,?,C,False,0\nmale,21,0,1,61.3792,1,?,C,False,1\nmale,,0,0,35.0,1,C128,S,True,0\nfemale,31,0,0,134.5,1,E39 E41,C,True,1\nmale,,0,0,35.5,1,C52,S,True,1\nmale,62,0,0,26.55,1,?,S,True,0\nfemale,36,0,0,135.6333,1,C32,C,True,1\nmale,30,1,0,24.0,2,?,C,False,0\nfemale,28,1,0,24.0,2,?,C,False,1\nmale,30,0,0,13.0,2,?,S,True,0\nmale,18,0,0,11.5,2,?,S,True,0\nmale,25,0,0,10.5,2,?,S,True,0\nmale,34,1,0,26.0,2,?,S,False,0\nfemale,36,1,0,26.0,2,?,S,False,1\nmale,57,0,0,13.0,2,?,S,True,0\nmale,18,0,0,11.5,2,?,S,True,0\nmale,23,0,0,10.5,2,?,S,True,0\nfemale,36,0,0,13.0,2,D,S,True,1\nmale,28,0,0,10.5,2,?,S,True,0\nmale,51,0,0,12.525,2,?,S,True,0\nmale,32,1,0,26.0,2,?,S,False,1\nfemale,19,1,0,26.0,2,?,S,False,1\nmale,28,0,0,26.0,2,?,S,True,0\nmale,1,2,1,39.0,2,F4,S,False,1\nfemale,4,2,1,39.0,2,F4,S,False,1\nfemale,12,2,1,39.0,2,F4,S,False,1\nfemale,36,0,3,39.0,2,F4,S,False,1\nmale,34,0,0,13.0,2,D56,S,True,1\nfemale,19,0,0,13.0,2,?,S,True,1\nmale,23,0,0,13.0,2,?,S,True,0\nmale,26,0,0,13.0,2,?,S,True,0\nmale,42,0,0,13.0,2,?,S,True,0\nmale,27,0,0,13.0,2,?,S,True,0\nfemale,24,0,0,13.0,2,F33,S,True,1\nfemale,15,0,2,39.0,2,?,S,False,1\nmale,60,1,1,39.0,2,?,S,False,0\nfemale,40,1,1,39.0,2,?,S,False,1\nfemale,20,1,0,26.0,2,?,S,False,1\nmale,25,1,0,26.0,2,?,S,False,0\nfemale,36,0,0,13.0,2,?,S,True,1\nmale,25,0,0,13.0,2,?,S,True,0\nmale,42,0,0,13.0,2,?,S,True,0\nfemale,42,0,0,13.0,2,?,S,True,1\nmale,0.8333,0,2,29.0,2,?,S,False,1\nmale,26,1,1,29.0,2,?,S,False,1\nfemale,22,1,1,29.0,2,?,S,False,1\nfemale,35,0,0,21.0,2,?,S,True,1\nmale,,0,0,0.0,2,?,S,True,0\nmale,19,0,0,13.0,2,?,S,True,0\nfemale,44,1,0,26.0,2,?,S,False,0\nmale,54,1,0,26.0,2,?,S,False,0\nmale,52,0,0,13.5,2,?,S,True,0\nmale,37,1,0,26.0,2,?,S,False,0\nfemale,29,1,0,26.0,2,?,S,False,0\nfemale,25,1,1,30.0,2,?,S,False,1\nfemale,45,0,2,30.0,2,?,S,False,1\nmale,29,1,0,26.0,2,?,S,False,0\nfemale,28,1,0,26.0,2,?,S,False,1\nmale,29,0,0,10.5,2,?,S,True,0\nmale,28,0,0,13.0,2,?,S,True,0\nmale,24,0,0,10.5,2,?,S,True,1\nfemale,8,0,2,26.25,2,?,S,False,1\nmale,31,1,1,26.25,2,?,S,False,0\nfemale,31,1,1,26.25,2,?,S,False,1\nfemale,22,0,0,10.5,2,F33,S,True,1\nfemale,30,0,0,13.0,2,?,S,True,0\nfemale,,0,0,21.0,2,?,S,True,0\nmale,21,0,0,11.5,2,?,S,True,0\nmale,,0,0,0.0,2,?,S,True,0\nmale,8,1,1,36.75,2,?,S,False,1\nmale,18,0,0,73.5,2,?,S,True,0\nfemale,48,0,2,36.75,2,?,S,False,1\nfemale,28,0,0,13.0,2,?,S,True,1\nmale,32,0,0,13.0,2,?,S,True,0\nmale,17,0,0,73.5,2,?,S,True,0\nmale,29,1,0,27.7208,2,?,C,False,0\nfemale,24,1,0,27.7208,2,?,C,False,1\nmale,25,0,0,31.5,2,?,S,True,0\nmale,18,0,0,73.5,2,?,S,True,0\nfemale,18,0,1,23.0,2,?,S,False,1\nfemale,34,0,1,23.0,2,?,S,False,1\nmale,54,0,0,26.0,2,?,S,True,0\nmale,8,0,2,32.5,2,?,S,False,1\nmale,42,1,1,32.5,2,?,S,False,0\nfemale,34,1,1,32.5,2,?,S,False,1\nfemale,27,1,0,13.8583,2,?,C,False,1\nfemale,30,1,0,13.8583,2,?,C,False,1\nmale,23,0,0,13.0,2,?,S,True,0\nmale,21,0,0,13.0,2,?,S,True,0\nmale,18,0,0,13.0,2,?,S,True,0\nmale,40,1,0,26.0,2,?,S,False,0\nfemale,29,1,0,26.0,2,?,S,False,1\nmale,18,0,0,10.5,2,?,S,True,0\nmale,36,0,0,13.0,2,?,S,True,0\nmale,,0,0,0.0,2,?,S,True,0\nfemale,38,0,0,13.0,2,?,S,True,0\nmale,35,0,0,26.0,2,?,S,True,0\nmale,38,1,0,21.0,2,?,S,False,0\nmale,34,1,0,21.0,2,?,S,False,0\nfemale,34,0,0,13.0,2,?,S,True,1\nmale,16,0,0,26.0,2,?,S,True,0\nmale,26,0,0,10.5,2,?,S,True,0\nmale,47,0,0,10.5,2,?,S,True,0\nmale,21,1,0,11.5,2,?,S,False,0\nmale,21,1,0,11.5,2,?,S,False,0\nmale,24,0,0,13.5,2,?,S,True,0\nmale,24,0,0,13.0,2,?,S,True,0\nmale,34,0,0,13.0,2,?,S,True,0\nmale,30,0,0,13.0,2,?,S,True,0\nmale,52,0,0,13.0,2,?,S,True,0\nmale,30,0,0,13.0,2,?,S,True,0\nmale,0.6667,1,1,14.5,2,?,S,False,1\nfemale,24,0,2,14.5,2,?,S,False,1\nmale,44,0,0,13.0,2,?,S,True,0\nfemale,6,0,1,33.0,2,?,S,False,1\nmale,28,0,1,33.0,2,?,S,False,0\nmale,62,0,0,10.5,2,?,S,True,1\nmale,30,0,0,10.5,2,?,S,True,0\nfemale,7,0,2,26.25,2,?,S,False,1\nmale,43,1,1,26.25,2,?,S,False,0\nfemale,45,1,1,26.25,2,?,S,False,1\nfemale,24,1,2,65.0,2,?,S,False,1\nfemale,24,1,2,65.0,2,?,S,False,1\nmale,49,1,2,65.0,2,?,S,False,0\nfemale,48,1,2,65.0,2,?,S,False,1\nfemale,55,0,0,16.0,2,?,S,True,1\nmale,24,2,0,73.5,2,?,S,False,0\nmale,32,2,0,73.5,2,?,S,False,0\nmale,21,2,0,73.5,2,?,S,False,0\nfemale,18,1,1,13.0,2,?,S,False,0\nfemale,20,2,1,23.0,2,?,S,False,1\nmale,23,2,1,11.5,2,?,S,False,0\nmale,36,0,0,13.0,2,?,S,True,0\nfemale,54,1,3,23.0,2,?,S,False,1\nmale,50,0,0,13.0,2,?,S,True,0\nmale,44,1,0,26.0,2,?,S,False,0\nfemale,29,1,0,26.0,2,?,S,False,1\nmale,21,0,0,73.5,2,?,S,True,0\nmale,42,0,0,13.0,2,?,S,True,1\nmale,63,1,0,26.0,2,?,S,False,0\nfemale,60,1,0,26.0,2,?,S,False,0\nmale,33,0,0,12.275,2,?,S,True,0\nfemale,17,0,0,10.5,2,?,S,True,1\nmale,42,1,0,27.0,2,?,S,False,0\nfemale,24,2,1,27.0,2,?,S,False,1\nmale,47,0,0,15.0,2,?,S,True,0\nmale,24,2,0,31.5,2,?,S,False,0\nmale,22,2,0,31.5,2,?,S,False,0\nmale,32,0,0,10.5,2,?,S,True,0\nfemale,23,0,0,13.7917,2,D,C,True,1\nmale,34,1,0,26.0,2,?,S,False,0\nfemale,24,1,0,26.0,2,?,S,False,1\nfemale,22,0,0,21.0,2,?,S,True,0\nfemale,,0,0,12.35,2,E101,Q,True,1\nmale,35,0,0,12.35,2,?,Q,True,0\nfemale,45,0,0,13.5,2,?,S,True,1\nmale,57,0,0,12.35,2,?,Q,True,0\nmale,,0,0,0.0,2,?,S,True,0\nmale,31,0,0,10.5,2,?,S,True,0\nfemale,26,1,1,26.0,2,?,S,False,0\nmale,30,1,1,26.0,2,?,S,False,0\nmale,,0,0,10.7083,2,?,Q,True,0\nfemale,1,1,2,41.5792,2,?,C,False,1\nfemale,3,1,2,41.5792,2,?,C,False,1\nmale,25,1,2,41.5792,2,?,C,False,0\nfemale,22,1,2,41.5792,2,?,C,False,1\nfemale,17,0,0,12.0,2,?,C,True,1\nfemale,,0,0,33.0,2,?,S,True,1\nfemale,34,0,0,10.5,2,F33,S,True,1\nmale,36,0,0,12.875,2,D,C,True,0\nmale,24,0,0,10.5,2,?,S,True,0\nmale,61,0,0,12.35,2,?,Q,True,0\nmale,50,1,0,26.0,2,?,S,False,0\nfemale,42,1,0,26.0,2,?,S,False,1\nfemale,57,0,0,10.5,2,E77,S,True,0\nmale,,0,0,15.0458,2,D,C,True,0\nmale,1,0,2,37.0042,2,?,C,False,1\nmale,31,1,1,37.0042,2,?,C,False,0\nfemale,24,1,1,37.0042,2,?,C,False,1\nmale,,0,0,15.5792,2,?,C,True,0\nmale,30,0,0,13.0,2,?,S,True,0\nmale,40,0,0,16.0,2,?,S,True,0\nmale,32,0,0,13.5,2,?,S,True,0\nmale,30,0,0,13.0,2,?,S,True,0\nmale,46,0,0,26.0,2,?,S,True,0\nfemale,13,0,1,19.5,2,?,S,False,1\nfemale,41,0,1,19.5,2,?,S,False,1\nmale,19,0,0,10.5,2,?,S,True,1\nmale,39,0,0,13.0,2,?,S,True,0\nmale,48,0,0,13.0,2,?,S,True,0\nmale,70,0,0,10.5,2,?,S,True,0\nmale,27,0,0,13.0,2,?,S,True,0\nmale,54,0,0,14.0,2,?,S,True,0\nmale,39,0,0,26.0,2,?,S,True,0\nmale,16,0,0,10.5,2,?,S,True,0\nmale,62,0,0,9.6875,2,?,Q,True,0\nmale,32.5,1,0,30.0708,2,?,C,False,0\nfemale,14,1,0,30.0708,2,?,C,False,1\nmale,2,1,1,26.0,2,F2,S,False,1\nmale,3,1,1,26.0,2,F2,S,False,1\nmale,36.5,0,2,26.0,2,F2,S,False,0\nmale,26,0,0,13.0,2,F2,S,True,0\nmale,19,1,1,36.75,2,?,S,False,0\nmale,28,0,0,13.5,2,?,S,True,0\nmale,20,0,0,13.8625,2,D38,C,True,1\nfemale,29,0,0,10.5,2,F33,S,True,1\nmale,39,0,0,13.0,2,?,S,True,0\nmale,22,0,0,10.5,2,?,S,True,1\nmale,,0,0,13.8625,2,?,C,True,1\nmale,23,0,0,10.5,2,?,S,True,0\nmale,29,0,0,13.8583,2,?,C,True,1\nmale,28,0,0,10.5,2,?,S,True,0\nmale,,0,0,0.0,2,?,S,True,0\nfemale,50,0,1,26.0,2,?,S,False,1\nmale,19,0,0,10.5,2,?,S,True,0\nmale,,0,0,15.05,2,?,C,True,0\nmale,41,0,0,13.0,2,?,S,True,0\nfemale,21,0,1,21.0,2,?,S,False,1\nfemale,19,0,0,26.0,2,?,S,True,1\nmale,43,0,1,21.0,2,?,S,False,0\nfemale,32,0,0,13.0,2,?,S,True,1\nmale,34,0,0,13.0,2,?,S,True,0\nmale,30,0,0,12.7375,2,?,C,True,1\nmale,27,0,0,15.0333,2,?,C,True,0\nfemale,2,1,1,26.0,2,?,S,False,1\nfemale,8,1,1,26.0,2,?,S,False,1\nfemale,33,0,2,26.0,2,?,S,False,1\nmale,36,0,0,10.5,2,?,S,True,0\nmale,34,1,0,21.0,2,?,S,False,0\nfemale,30,3,0,21.0,2,?,S,False,1\nfemale,28,0,0,13.0,2,?,S,True,1\nmale,23,0,0,15.0458,2,?,C,True,0\nmale,0.8333,1,1,18.75,2,?,S,False,1\nmale,3,1,1,18.75,2,?,S,False,1\nfemale,24,2,3,18.75,2,?,S,False,1\nfemale,50,0,0,10.5,2,?,S,True,1\nmale,19,0,0,10.5,2,?,S,True,0\nfemale,21,0,0,10.5,2,?,S,True,1\nmale,26,0,0,13.0,2,?,S,True,0\nmale,25,0,0,13.0,2,?,S,True,0\nmale,27,0,0,26.0,2,?,S,True,0\nfemale,25,0,1,26.0,2,?,S,False,1\nfemale,18,0,2,13.0,2,?,S,False,1\nfemale,20,0,0,36.75,2,?,S,True,1\nfemale,30,0,0,13.0,2,?,S,True,1\nmale,59,0,0,13.5,2,?,S,True,0\nfemale,30,0,0,12.35,2,?,Q,True,1\nmale,35,0,0,10.5,2,?,S,True,0\nfemale,40,0,0,13.0,2,?,S,True,1\nmale,25,0,0,13.0,2,?,S,True,0\nmale,41,0,0,15.0458,2,?,C,True,0\nmale,25,0,0,10.5,2,?,S,True,0\nmale,18.5,0,0,13.0,2,F,S,True,0\nmale,14,0,0,65.0,2,?,S,True,0\nfemale,50,0,0,10.5,2,?,S,True,1\nmale,23,0,0,13.0,2,?,S,True,0\nfemale,28,0,0,12.65,2,?,S,True,1\nfemale,27,0,0,10.5,2,E101,S,True,1\nmale,29,1,0,21.0,2,?,S,False,0\nfemale,27,1,0,21.0,2,?,S,False,0\nmale,40,0,0,13.0,2,?,S,True,0\nfemale,31,0,0,21.0,2,?,S,True,1\nmale,30,1,0,21.0,2,?,S,False,0\nmale,23,1,0,10.5,2,?,S,False,0\nfemale,31,0,0,21.0,2,?,S,True,1\nmale,,0,0,0.0,2,?,S,True,0\nfemale,12,0,0,15.75,2,?,S,True,1\nfemale,40,0,0,15.75,2,?,S,True,1\nfemale,32.5,0,0,13.0,2,E101,S,True,1\nmale,27,1,0,26.0,2,?,S,False,0\nfemale,29,1,0,26.0,2,?,S,False,1\nmale,2,1,1,23.0,2,?,S,False,1\nfemale,4,1,1,23.0,2,?,S,False,1\nfemale,29,0,2,23.0,2,?,S,False,1\nfemale,0.9167,1,2,27.75,2,?,S,False,1\nfemale,5,1,2,27.75,2,?,S,False,1\nmale,36,1,2,27.75,2,?,S,False,0\nfemale,33,1,2,27.75,2,?,S,False,1\nmale,66,0,0,10.5,2,?,S,True,0\nmale,,0,0,12.875,2,?,S,True,0\nmale,31,0,0,13.0,2,?,S,True,1\nmale,,0,0,13.0,2,?,S,True,1\nfemale,26,0,0,13.5,2,?,S,True,1\nfemale,24,0,0,13.0,2,?,S,True,0\nmale,42,0,0,7.55,3,?,S,True,0\nmale,13,0,2,20.25,3,?,S,False,0\nmale,16,1,1,20.25,3,?,S,False,0\nfemale,35,1,1,20.25,3,?,S,False,1\nfemale,16,0,0,7.65,3,?,S,True,1\nmale,25,0,0,7.65,3,F G63,S,True,1\nmale,20,0,0,7.925,3,?,S,True,1\nfemale,18,0,0,7.2292,3,?,C,True,1\nmale,30,0,0,7.25,3,?,S,True,0\nmale,26,0,0,8.05,3,?,S,True,0\nfemale,40,1,0,9.475,3,?,S,False,0\nmale,0.8333,0,1,9.35,3,?,S,False,1\nfemale,18,0,1,9.35,3,?,S,False,1\nmale,26,0,0,18.7875,3,?,C,True,1\nmale,26,0,0,7.8875,3,?,S,True,0\nmale,20,0,0,7.925,3,?,S,True,0\nmale,24,0,0,7.05,3,?,S,True,0\nmale,25,0,0,7.05,3,?,S,True,0\nmale,35,0,0,8.05,3,?,S,True,0\nmale,18,0,0,8.3,3,?,S,True,0\nmale,32,0,0,22.525,3,?,S,True,0\nfemale,19,1,0,7.8542,3,?,S,False,1\nmale,4,4,2,31.275,3,?,S,False,0\nfemale,6,4,2,31.275,3,?,S,False,0\nfemale,2,4,2,31.275,3,?,S,False,0\nfemale,17,4,2,7.925,3,?,S,False,1\nfemale,38,4,2,7.775,3,?,S,False,0\nfemale,9,4,2,31.275,3,?,S,False,0\nfemale,11,4,2,31.275,3,?,S,False,0\nmale,39,1,5,31.275,3,?,S,False,0\nmale,27,0,0,7.7958,3,?,S,True,1\nmale,26,0,0,7.775,3,?,S,True,0\nfemale,39,1,5,31.275,3,?,S,False,0\nmale,20,0,0,7.8542,3,?,S,True,0\nmale,26,0,0,7.8958,3,?,S,True,0\nmale,25,1,0,17.8,3,?,S,False,0\nfemale,18,1,0,17.8,3,?,S,False,0\nmale,24,0,0,7.775,3,?,S,True,0\nmale,35,0,0,7.05,3,?,S,True,0\nmale,5,4,2,31.3875,3,?,S,False,0\nmale,9,4,2,31.3875,3,?,S,False,0\nmale,3,4,2,31.3875,3,?,S,False,1\nmale,13,4,2,31.3875,3,?,S,False,0\nfemale,5,4,2,31.3875,3,?,S,False,1\nmale,40,1,5,31.3875,3,?,S,False,0\nmale,23,0,0,7.7958,3,?,S,True,1\nfemale,38,1,5,31.3875,3,?,S,False,1\nfemale,45,0,0,7.225,3,?,C,True,1\nmale,21,0,0,7.225,3,?,C,True,0\nmale,23,0,0,7.05,3,?,S,True,0\nfemale,17,0,0,14.4583,3,?,C,True,0\nmale,30,0,0,7.225,3,?,C,True,0\nmale,23,0,0,7.8542,3,?,S,True,0\nfemale,13,0,0,7.2292,3,?,C,True,1\nmale,20,0,0,7.225,3,?,C,True,0\nmale,32,1,0,15.85,3,?,S,False,0\nfemale,33,3,0,15.85,3,?,S,False,1\nfemale,0.75,2,1,19.2583,3,?,C,False,1\nfemale,0.75,2,1,19.2583,3,?,C,False,1\nfemale,5,2,1,19.2583,3,?,C,False,1\nfemale,24,0,3,19.2583,3,?,C,False,1\nfemale,18,0,0,8.05,3,?,S,True,1\nmale,40,0,0,7.225,3,?,C,True,0\nmale,26,0,0,7.8958,3,?,S,True,0\nmale,20,0,0,7.2292,3,?,C,True,1\nfemale,18,0,1,14.4542,3,?,C,False,0\nfemale,45,0,1,14.4542,3,?,C,False,0\nfemale,27,0,0,7.8792,3,?,Q,True,0\nmale,22,0,0,8.05,3,?,S,True,0\nmale,19,0,0,8.05,3,?,S,True,0\nmale,26,0,0,7.775,3,?,S,True,0\nmale,22,0,0,9.35,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nmale,20,0,0,4.0125,3,?,C,True,0\nmale,32,0,0,56.4958,3,?,S,True,1\nmale,21,0,0,7.775,3,?,S,True,0\nmale,18,0,0,7.75,3,?,S,True,0\nmale,26,0,0,7.8958,3,?,S,True,0\nmale,6,1,1,15.2458,3,?,C,False,0\nfemale,9,1,1,15.2458,3,?,C,False,0\nmale,,0,0,7.225,3,?,C,True,0\nfemale,,0,2,15.2458,3,?,C,False,0\nfemale,,0,2,7.75,3,?,Q,False,0\nmale,40,1,1,15.5,3,?,Q,False,0\nfemale,32,1,1,15.5,3,?,Q,False,0\nmale,21,0,0,16.1,3,?,S,True,0\nfemale,22,0,0,7.725,3,?,Q,True,1\nfemale,20,0,0,7.8542,3,?,S,True,0\nmale,29,1,0,7.0458,3,?,S,False,0\nmale,22,1,0,7.25,3,?,S,False,0\nmale,22,0,0,7.7958,3,?,S,True,0\nmale,35,0,0,8.05,3,?,S,True,0\nfemale,18.5,0,0,7.2833,3,?,Q,True,0\nmale,21,0,0,7.8208,3,?,Q,True,1\nmale,19,0,0,6.75,3,?,Q,True,0\nfemale,18,0,0,7.8792,3,?,Q,True,0\nfemale,21,0,0,8.6625,3,?,S,True,0\nfemale,30,0,0,8.6625,3,?,S,True,0\nmale,18,0,0,8.6625,3,?,S,True,0\nmale,38,0,0,8.6625,3,?,S,True,0\nmale,17,0,0,8.6625,3,?,S,True,0\nmale,17,0,0,8.6625,3,?,S,True,0\nfemale,21,0,0,7.75,3,?,Q,True,0\nmale,21,0,0,7.75,3,?,Q,True,0\nmale,21,0,0,8.05,3,?,S,True,0\nmale,,1,0,14.4583,3,?,C,False,0\nfemale,,1,0,14.4583,3,?,C,False,0\nmale,28,0,0,7.7958,3,?,S,True,0\nmale,24,0,0,7.8542,3,?,S,True,0\nfemale,16,0,0,7.75,3,?,Q,True,1\nfemale,37,0,0,7.75,3,?,Q,True,0\nmale,28,0,0,7.25,3,?,S,True,0\nmale,24,0,0,8.05,3,?,S,True,0\nmale,21,0,0,7.7333,3,?,Q,True,0\nmale,32,0,0,56.4958,3,?,S,True,1\nmale,29,0,0,8.05,3,?,S,True,0\nmale,26,1,0,14.4542,3,?,C,False,0\nmale,18,1,0,14.4542,3,?,C,False,0\nmale,20,0,0,7.05,3,?,S,True,0\nmale,18,0,0,8.05,3,?,S,True,1\nmale,24,0,0,7.25,3,?,Q,True,0\nmale,36,0,0,7.4958,3,?,S,True,0\nmale,24,0,0,7.4958,3,?,S,True,0\nmale,31,0,0,7.7333,3,?,Q,True,0\nmale,31,0,0,7.75,3,?,Q,True,0\nfemale,22,0,0,7.75,3,?,Q,True,1\nfemale,30,0,0,7.6292,3,?,Q,True,0\nmale,70.5,0,0,7.75,3,?,Q,True,0\nmale,43,0,0,8.05,3,?,S,True,0\nmale,35,0,0,7.8958,3,?,S,True,0\nmale,27,0,0,7.8958,3,?,S,True,0\nmale,19,0,0,7.8958,3,?,S,True,0\nmale,30,0,0,8.05,3,?,S,True,0\nmale,9,1,1,15.9,3,?,S,False,1\nmale,3,1,1,15.9,3,?,S,False,1\nfemale,36,0,2,15.9,3,?,S,False,1\nmale,59,0,0,7.25,3,?,S,True,0\nmale,19,0,0,8.1583,3,?,S,True,0\nfemale,17,0,1,16.1,3,?,S,False,1\nmale,44,0,1,16.1,3,?,S,False,0\nmale,17,0,0,8.6625,3,?,S,True,0\nmale,22.5,0,0,7.225,3,?,C,True,0\nmale,45,0,0,8.05,3,?,S,True,1\nfemale,22,0,0,10.5167,3,?,S,True,0\nmale,19,0,0,10.1708,3,?,S,True,0\nfemale,30,0,0,6.95,3,?,Q,True,1\nmale,29,0,0,7.75,3,?,Q,True,1\nmale,0.3333,0,2,14.4,3,?,S,False,0\nmale,34,1,1,14.4,3,?,S,False,0\nfemale,28,1,1,14.4,3,?,S,False,0\nmale,27,0,0,7.8958,3,?,S,True,0\nmale,25,0,0,7.8958,3,?,S,True,0\nmale,24,2,0,24.15,3,?,S,False,0\nmale,22,0,0,8.05,3,?,S,True,0\nmale,21,2,0,24.15,3,?,S,False,0\nmale,17,2,0,8.05,3,?,S,False,0\nmale,,1,0,16.1,3,?,S,False,0\nfemale,,1,0,16.1,3,?,S,False,1\nmale,36.5,1,0,17.4,3,?,S,False,1\nfemale,36,1,0,17.4,3,?,S,False,1\nmale,30,0,0,9.5,3,?,S,True,1\nmale,16,0,0,9.5,3,?,S,True,0\nmale,1,1,2,20.575,3,?,S,False,1\nfemale,0.1667,1,2,20.575,3,?,S,False,1\nmale,26,1,2,20.575,3,?,S,False,0\nfemale,33,1,2,20.575,3,?,S,False,1\nmale,25,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,22,0,0,7.25,3,?,S,True,0\nmale,36,0,0,7.25,3,?,S,True,0\nfemale,19,0,0,7.8792,3,?,Q,True,1\nmale,17,0,0,7.8958,3,?,S,True,0\nmale,42,0,0,8.6625,3,?,S,True,0\nmale,43,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,0\nmale,32,0,0,7.75,3,?,Q,True,0\nmale,19,0,0,8.05,3,?,S,True,1\nfemale,30,0,0,12.475,3,?,S,True,1\nfemale,24,0,0,7.75,3,?,Q,True,0\nfemale,23,0,0,8.05,3,?,S,True,1\nmale,33,0,0,7.8958,3,?,C,True,0\nmale,65,0,0,7.75,3,?,Q,True,0\nmale,24,0,0,7.55,3,?,S,True,1\nmale,23,1,0,13.9,3,?,S,False,0\nfemale,22,1,0,13.9,3,?,S,False,1\nmale,18,0,0,7.775,3,?,S,True,0\nmale,16,0,0,7.775,3,?,S,True,0\nmale,45,0,0,6.975,3,?,S,True,0\nmale,,0,0,7.225,3,?,C,True,0\nmale,39,0,2,7.2292,3,?,C,False,0\nmale,17,1,1,7.2292,3,?,C,False,0\nmale,15,1,1,7.2292,3,?,C,False,0\nmale,47,0,0,7.25,3,?,S,True,0\nfemale,5,0,0,12.475,3,?,S,True,1\nmale,,0,0,7.225,3,?,C,True,0\nmale,40.5,0,0,15.1,3,?,S,True,0\nmale,40.5,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.05,3,?,S,True,1\nmale,18,0,0,7.7958,3,?,S,True,0\nfemale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,6.95,3,?,Q,True,0\nmale,26,0,0,7.8792,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,56.4958,3,?,S,True,1\nfemale,21,2,2,34.375,3,?,S,False,0\nfemale,9,2,2,34.375,3,?,S,False,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,18,2,2,34.375,3,?,S,False,0\nmale,16,1,3,34.375,3,?,S,False,0\nfemale,48,1,3,34.375,3,?,S,False,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.25,3,?,S,True,0\nmale,25,0,0,7.7417,3,?,Q,True,0\nmale,,0,0,14.5,3,?,S,True,0\nmale,,0,0,7.8958,3,?,C,True,0\nmale,22,0,0,8.05,3,?,S,True,0\nfemale,16,0,0,7.7333,3,?,Q,True,1\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,9,0,2,20.525,3,?,S,False,1\nmale,33,1,1,20.525,3,?,S,False,0\nmale,41,0,0,7.85,3,?,S,True,0\nfemale,31,1,1,20.525,3,?,S,False,1\nmale,38,0,0,7.05,3,?,S,True,0\nmale,9,5,2,46.9,3,?,S,False,0\nmale,1,5,2,46.9,3,?,S,False,0\nmale,11,5,2,46.9,3,?,S,False,0\nfemale,10,5,2,46.9,3,?,S,False,0\nfemale,16,5,2,46.9,3,?,S,False,0\nmale,14,5,2,46.9,3,?,S,False,0\nmale,40,1,6,46.9,3,?,S,False,0\nfemale,43,1,6,46.9,3,?,S,False,0\nmale,51,0,0,8.05,3,?,S,True,0\nmale,32,0,0,8.3625,3,?,S,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,20,0,0,9.8458,3,?,S,True,0\nmale,37,2,0,7.925,3,?,S,False,0\nmale,28,2,0,7.925,3,?,S,False,0\nmale,19,0,0,7.775,3,?,S,True,0\nfemale,24,0,0,8.85,3,?,S,True,0\nfemale,17,0,0,7.7333,3,?,Q,True,0\nmale,,1,0,19.9667,3,?,S,False,0\nmale,,1,0,19.9667,3,?,S,False,0\nmale,28,1,0,15.85,3,?,S,False,0\nfemale,24,1,0,15.85,3,?,S,False,1\nmale,20,0,0,9.5,3,?,S,True,0\nmale,23.5,0,0,7.2292,3,?,C,True,0\nmale,41,2,0,14.1083,3,?,S,False,0\nmale,26,1,0,7.8542,3,?,S,False,0\nmale,21,0,0,7.8542,3,?,S,True,0\nfemale,45,1,0,14.1083,3,?,S,False,1\nfemale,,0,0,7.55,3,?,S,True,0\nmale,25,0,0,7.25,3,?,S,True,0\nmale,,0,0,6.8583,3,?,Q,True,0\nmale,11,0,0,18.7875,3,?,C,True,0\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,27,0,0,6.975,3,?,S,True,1\nmale,,0,0,56.4958,3,?,S,True,1\nfemale,18,0,0,6.75,3,?,Q,True,0\nfemale,26,0,0,7.925,3,?,S,True,1\nfemale,23,0,0,7.925,3,?,S,True,0\nfemale,22,0,0,8.9625,3,?,S,True,1\nmale,28,0,0,7.8958,3,?,S,True,0\nfemale,28,0,0,7.775,3,?,S,True,0\nfemale,,0,0,7.75,3,?,Q,True,0\nfemale,2,0,1,12.2875,3,?,S,False,1\nfemale,22,1,1,12.2875,3,?,S,False,1\nmale,43,0,0,6.45,3,?,S,True,0\nmale,28,0,0,22.525,3,?,S,True,0\nfemale,27,0,0,7.925,3,?,S,True,1\nmale,,0,0,7.75,3,?,Q,True,0\nfemale,,0,0,8.05,3,?,S,True,1\nmale,42,0,0,7.65,3,F G63,S,True,0\nmale,,0,0,7.8875,3,?,S,True,1\nmale,30,0,0,7.2292,3,?,C,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nfemale,27,1,0,7.925,3,?,S,False,0\nfemale,25,1,0,7.925,3,?,S,False,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,29,0,0,7.8958,3,?,C,True,1\nmale,21,0,0,7.7958,3,?,S,True,1\nmale,,0,0,7.05,3,?,S,True,0\nmale,20,0,0,7.8542,3,?,S,True,0\nmale,48,0,0,7.8542,3,?,S,True,0\nmale,17,1,0,7.0542,3,?,S,False,0\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,,0,0,8.1125,3,?,S,True,1\nmale,34,0,0,6.4958,3,?,S,True,0\nmale,26,0,0,7.775,3,?,S,True,1\nmale,22,0,0,7.7958,3,?,S,True,0\nmale,33,0,0,8.6542,3,?,S,True,0\nmale,31,0,0,7.775,3,?,S,True,0\nmale,29,0,0,7.8542,3,?,S,True,0\nmale,4,1,1,11.1333,3,?,S,False,1\nfemale,1,1,1,11.1333,3,?,S,False,1\nmale,49,0,0,0.0,3,?,S,True,0\nmale,33,0,0,7.775,3,?,S,True,0\nmale,19,0,0,0.0,3,?,S,True,0\nfemale,27,0,2,11.1333,3,?,S,False,1\nmale,,1,2,23.45,3,?,S,False,0\nfemale,,1,2,23.45,3,?,S,False,0\nmale,,1,2,23.45,3,?,S,False,0\nfemale,,1,2,23.45,3,?,S,False,0\nmale,23,0,0,7.8958,3,?,S,True,0\nmale,32,0,0,7.8542,3,?,S,True,1\nmale,27,0,0,7.8542,3,?,S,True,0\nfemale,20,1,0,9.825,3,?,S,False,0\nfemale,21,1,0,9.825,3,?,S,False,0\nmale,32,0,0,7.925,3,?,S,True,1\nmale,17,0,0,7.125,3,?,S,True,0\nmale,21,0,0,8.4333,3,?,S,True,0\nmale,30,0,0,7.8958,3,?,S,True,0\nmale,21,0,0,7.7958,3,?,S,True,1\nmale,33,0,0,7.8542,3,?,S,True,0\nmale,22,0,0,7.5208,3,?,S,True,0\nfemale,4,0,1,13.4167,3,?,C,False,1\nmale,39,0,1,13.4167,3,?,C,False,1\nmale,,0,0,7.2292,3,?,C,True,0\nmale,18.5,0,0,7.2292,3,?,C,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.25,3,?,S,True,0\nfemale,,0,0,7.75,3,?,Q,True,1\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,34.5,0,0,7.8292,3,?,Q,True,0\nmale,44,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,1\nmale,,1,0,14.4542,3,?,C,False,0\nfemale,,1,0,14.4542,3,?,C,False,0\nmale,,1,0,7.75,3,?,Q,False,0\nmale,,1,0,7.75,3,?,Q,False,0\nmale,,0,0,7.7375,3,?,Q,True,0\nfemale,22,2,0,8.6625,3,?,S,False,0\nmale,26,2,0,8.6625,3,?,S,False,0\nfemale,4,0,2,22.025,3,?,S,False,1\nmale,29,3,1,22.025,3,?,S,False,1\nfemale,26,1,1,22.025,3,?,S,False,1\nfemale,1,1,1,12.1833,3,?,S,False,0\nmale,18,1,1,7.8542,3,?,S,False,0\nfemale,36,0,2,12.1833,3,?,S,False,0\nmale,,0,0,7.8958,3,?,C,True,0\nmale,25,0,0,7.2292,3,F E57,C,True,1\nmale,,0,0,7.225,3,?,C,True,0\nfemale,37,0,0,9.5875,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,56.4958,3,?,S,True,1\nmale,,0,0,56.4958,3,?,S,True,0\nfemale,22,0,0,7.25,3,?,S,True,1\nmale,,0,0,7.75,3,?,Q,True,0\nmale,26,0,0,56.4958,3,?,S,True,1\nmale,29,0,0,9.4833,3,?,S,True,0\nmale,29,0,0,7.775,3,?,S,True,0\nmale,22,0,0,7.775,3,?,S,True,0\nmale,22,0,0,7.225,3,?,C,True,1\nmale,,3,1,25.4667,3,?,S,False,0\nfemale,,3,1,25.4667,3,?,S,False,0\nfemale,,3,1,25.4667,3,?,S,False,0\nfemale,,3,1,25.4667,3,?,S,False,0\nfemale,,0,4,25.4667,3,?,S,False,0\nmale,32,0,0,7.925,3,?,S,True,0\nmale,34.5,0,0,6.4375,3,?,C,True,0\nfemale,,1,0,15.5,3,?,Q,False,0\nmale,,1,0,15.5,3,?,Q,False,0\nmale,36,0,0,0.0,3,?,S,True,0\nmale,39,0,0,24.15,3,?,S,True,0\nmale,24,0,0,9.5,3,?,S,True,0\nfemale,25,0,0,7.775,3,?,S,True,0\nfemale,45,0,0,7.75,3,?,S,True,0\nmale,36,1,0,15.55,3,?,S,False,0\nfemale,30,1,0,15.55,3,?,S,False,0\nmale,20,1,0,7.925,3,?,S,False,1\nmale,,0,0,7.8792,3,?,Q,True,0\nmale,28,0,0,56.4958,3,?,S,True,0\nmale,,0,0,7.55,3,?,S,True,0\nmale,30,1,0,16.1,3,?,S,False,0\nfemale,26,1,0,16.1,3,?,S,False,0\nmale,,0,0,7.8792,3,?,S,True,0\nmale,20.5,0,0,7.25,3,?,S,True,0\nmale,27,0,0,8.6625,3,?,S,True,1\nmale,51,0,0,7.0542,3,?,S,True,0\nfemale,23,0,0,7.8542,3,?,S,True,1\nmale,32,0,0,7.5792,3,?,S,True,1\nmale,,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.55,3,?,S,True,0\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,24,0,0,7.1417,3,?,S,True,1\nmale,22,0,0,7.125,3,?,S,True,0\nfemale,,0,0,7.8792,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,8.05,3,?,S,True,0\nmale,29,0,0,7.925,3,?,S,True,0\nmale,,0,0,7.2292,3,?,C,True,1\nfemale,30.5,0,0,7.75,3,?,Q,True,0\nfemale,,0,0,7.7375,3,?,Q,True,1\nmale,,0,0,7.2292,3,F E46,C,True,0\nmale,35,0,0,7.8958,3,?,C,True,0\nmale,33,0,0,7.8958,3,?,S,True,0\nfemale,,0,0,7.225,3,?,C,True,1\nmale,,0,0,7.8958,3,?,C,True,0\nfemale,,0,0,7.75,3,?,Q,True,1\nmale,,0,0,7.75,3,?,Q,True,1\nfemale,,2,0,23.25,3,?,Q,False,1\nfemale,,2,0,23.25,3,?,Q,False,1\nmale,,2,0,23.25,3,?,Q,False,1\nfemale,,0,0,7.7875,3,?,Q,True,1\nmale,,0,0,15.5,3,?,Q,True,0\nfemale,,0,0,7.8792,3,?,Q,True,1\nfemale,15,0,0,8.0292,3,?,Q,True,1\nfemale,35,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,24,1,0,16.1,3,?,S,False,0\nfemale,19,1,0,16.1,3,?,S,False,0\nfemale,,0,0,7.75,3,?,Q,True,0\nfemale,,0,0,8.05,3,?,S,True,0\nfemale,,0,0,8.05,3,?,S,True,0\nmale,55.5,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,21,0,0,7.775,3,?,S,True,1\nmale,,0,0,8.05,3,?,S,True,0\nmale,24,0,0,7.8958,3,?,S,True,0\nmale,21,0,0,7.8958,3,?,S,True,0\nmale,28,0,0,7.8958,3,?,S,True,0\nmale,,0,0,7.8958,3,?,S,True,0\nfemale,,0,0,7.8792,3,?,Q,True,1\nmale,25,0,0,7.65,3,F G73,S,True,0\nmale,6,0,1,12.475,3,E121,S,False,1\nfemale,27,0,1,12.475,3,E121,S,False,1\nmale,,0,0,8.05,3,?,S,True,0\nfemale,,1,0,24.15,3,?,Q,False,1\nmale,,1,0,24.15,3,?,Q,False,0\nmale,,0,0,8.4583,3,?,Q,True,0\nmale,34,0,0,8.05,3,?,S,True,0\nmale,,0,0,7.75,3,?,Q,True,0\nmale,,0,0,7.775,3,?,S,True,1\nmale,,1,1,15.2458,3,?,C,False,1\nmale,,1,1,15.2458,3,?,C,False,1\nfemale,,0,2,15.2458,3,?,C,False,1\nfemale,,0,0,7.2292,3,?,C,True,1\nmale,,0,0,8.05,3,?,S,True,0\nfemale,,0,0,7.7333,3,?,Q,True,1\nfemale,24,0,0,7.75,3,?,Q,True,1\nmale,,0,0,8.05,3,?,S,True,0\nfemale,,1,0,15.5,3,?,Q,False,1\nfemale,,1,0,15.5,3,?,Q,False,1\nfemale,,0,0,15.5,3,?,Q,True,1\nmale,18,0,0,7.75,3,?,S,True,0\nmale,22,0,0,7.8958,3,?,S,True,0\n" }, { "path": "benchmark/experiments/__init__.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom benchmark.experiments import structured_data\n\n\ndef get_experiments(task_name):\n if task_name == \"structured_data_classification\":\n return [\n structured_data.Titanic(),\n structured_data.Iris(),\n structured_data.Wine(),\n ]\n" }, { "path": "benchmark/experiments/experiment.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport shutil\nimport timeit\n\n\nclass Experiment(object):\n def __init__(self, name, tmp_dir=\"tmp_dir\"):\n self.name = name\n self.tmp_dir = tmp_dir\n\n def get_auto_model(self):\n raise NotImplementedError\n\n @staticmethod\n def load_data():\n raise NotImplementedError\n\n def run_once(self):\n (x_train, y_train), (x_test, y_test) = self.load_data()\n auto_model = self.get_auto_model()\n\n start_time = timeit.default_timer()\n auto_model.fit(x_train, y_train)\n stop_time = timeit.default_timer()\n\n accuracy = auto_model.evaluate(x_test, y_test)[1]\n total_time = stop_time - start_time\n\n return total_time, accuracy\n\n def run(self, repeat_times=1):\n total_times = []\n metric_values = []\n for i in range(repeat_times):\n total_time, metric = self.run_once()\n total_times.append(total_time)\n metric_values.append(metric)\n self.tear_down()\n return total_times, metric_values\n\n def tear_down(self):\n shutil.rmtree(self.tmp_dir)\n" }, { "path": "benchmark/experiments/image.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom keras import datasets\n\nimport autokeras as ak\nfrom benchmark.experiments import experiment\n\n\nclass ImageClassifierExperiment(experiment.Experiment):\n def get_auto_model(self):\n return ak.ImageClassifier(\n max_trials=10, directory=self.tmp_dir, overwrite=True\n )\n\n\nclass MNIST(ImageClassifierExperiment):\n def __init__(self):\n super().__init__(name=\"MNIST\")\n\n @staticmethod\n def load_data():\n return datasets.mnist.load_data()\n\n\nclass CIFAR10(ImageClassifierExperiment):\n def __init__(self):\n super().__init__(name=\"CIFAR10\")\n\n @staticmethod\n def load_data():\n return datasets.cifar10.load_data()\n" }, { "path": "benchmark/experiments/structured_data.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport keras\nimport numpy as np\nimport pandas as pd\nimport sklearn\n\nimport autokeras as ak\nfrom benchmark.experiments import experiment\n\n\nclass StructuredDataClassifierExperiment(experiment.Experiment):\n def get_auto_model(self):\n return ak.StructuredDataClassifier(\n max_trials=10, directory=self.tmp_dir, overwrite=True\n )\n\n\nclass Titanic(StructuredDataClassifierExperiment):\n def __init__(self):\n super().__init__(name=\"Titanic\")\n\n @staticmethod\n def load_data():\n TRAIN_DATA_URL = (\n \"https://storage.googleapis.com/tf-datasets/titanic/train.csv\"\n )\n TEST_DATA_URL = (\n \"https://storage.googleapis.com/tf-datasets/titanic/eval.csv\"\n )\n x_train = keras.utils.get_file(\"titanic_train.csv\", TRAIN_DATA_URL)\n x_test = keras.utils.get_file(\"titanic_eval.csv\", TEST_DATA_URL)\n\n return (x_train, \"survived\"), (x_test, \"survived\")\n\n\nclass Iris(StructuredDataClassifierExperiment):\n def __init__(self):\n super().__init__(name=\"Iris\")\n\n @staticmethod\n def load_data():\n # Prepare the dataset.\n TRAIN_DATA_URL = (\n \"https://storage.googleapis.com/\"\n \"download.tensorflow.org/data/iris_training.csv\"\n )\n x_train = keras.utils.get_file(\"iris_train.csv\", TRAIN_DATA_URL)\n\n TEST_DATA_URL = (\n \"https://storage.googleapis.com/\"\n \"download.tensorflow.org/data/iris_test.csv\"\n )\n x_test = keras.utils.get_file(\"iris_test.csv\", TEST_DATA_URL)\n\n return (x_train, \"virginica\"), (x_test, \"virginica\")\n\n\nclass Wine(StructuredDataClassifierExperiment):\n def __init__(self):\n super().__init__(name=\"Wine\")\n\n @staticmethod\n def load_data():\n DATASET_URL = (\n \"https://archive.ics.uci.edu/ml/\"\n \"machine-learning-databases/wine/wine.data\"\n )\n\n # save data\n dataset = keras.utils.get_file(\"wine.csv\", DATASET_URL)\n\n data = pd.read_csv(dataset, header=None).sample(frac=1, random_state=5)\n split_length = int(data.shape[0] * 0.8) # 141\n\n return (data.iloc[:split_length, 1:], data.iloc[:split_length, 0]), (\n data.iloc[split_length:, 1:],\n data.iloc[split_length:, 0],\n )\n\n\nclass StructuredDataRegressorExperiment(experiment.Experiment):\n def get_auto_model(self):\n return ak.StructuredDataRegressor(\n max_trials=10, directory=self.tmp_dir, overwrite=True\n )\n\n\nclass CaliforniaHousing(StructuredDataRegressorExperiment):\n @staticmethod\n def load_data():\n house_dataset = sklearn.datasets.fetch_california_housing()\n (\n x_train,\n x_test,\n y_train,\n y_test,\n ) = sklearn.model_selection.train_test_split(\n house_dataset.data,\n np.array(house_dataset.target),\n test_size=0.2,\n random_state=42,\n )\n return (x_train, y_train), (x_test, y_test)\n" }, { "path": "benchmark/experiments/text.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\n\nimport keras\nimport numpy as np\nfrom sklearn.datasets import load_files\n\nimport autokeras as ak\nfrom benchmark.experiments import experiment\n\n\nclass IMDB(experiment.Experiment):\n def __init__(self):\n super().__init__(name=\"IMDB\")\n\n def get_auto_model(self):\n return ak.TextClassifier(\n max_trials=10, directory=self.tmp_dir, overwrite=True\n )\n\n @staticmethod\n def load_data():\n dataset = keras.utils.get_file(\n fname=\"aclImdb.tar.gz\",\n origin=\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\", # noqa: E501\n extract=True,\n )\n\n # set path to dataset\n IMDB_DATADIR = os.path.join(os.path.dirname(dataset), \"aclImdb\")\n\n classes = [\"pos\", \"neg\"]\n train_data = load_files(\n os.path.join(IMDB_DATADIR, \"train\"),\n shuffle=True,\n categories=classes,\n )\n test_data = load_files(\n os.path.join(IMDB_DATADIR, \"test\"),\n shuffle=False,\n categories=classes,\n )\n\n x_train = np.array(train_data.data)\n y_train = np.array(train_data.target)\n x_test = np.array(test_data.data)\n y_test = np.array(test_data.target)\n return (x_train, y_train), (x_test, y_test)\n" }, { "path": "benchmark/performance.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\nimport os\n\nimport keras\nimport numpy as np\nfrom keras.datasets import cifar10\nfrom keras.datasets import mnist\nfrom sklearn.datasets import load_files\n\nimport autokeras as ak\n\n\ndef imdb_raw(num_instances=100):\n dataset = keras.utils.get_file(\n fname=\"aclImdb.tar.gz\",\n origin=\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\",\n extract=True,\n )\n\n # set path to dataset\n IMDB_DATADIR = os.path.join(os.path.dirname(dataset), \"aclImdb\")\n\n classes = [\"pos\", \"neg\"]\n train_data = load_files(\n os.path.join(IMDB_DATADIR, \"train\"), shuffle=True, categories=classes\n )\n test_data = load_files(\n os.path.join(IMDB_DATADIR, \"test\"), shuffle=False, categories=classes\n )\n\n x_train = np.array(train_data.data)\n y_train = np.array(train_data.target)\n x_test = np.array(test_data.data)\n y_test = np.array(test_data.target)\n\n if num_instances is not None:\n x_train = x_train[:num_instances]\n y_train = y_train[:num_instances]\n x_test = x_test[:num_instances]\n y_test = y_test[:num_instances]\n return (x_train, y_train), (x_test, y_test)\n\n\ndef test_mnist_accuracy_over_98(tmp_path):\n (x_train, y_train), (x_test, y_test) = mnist.load_data()\n clf = ak.ImageClassifier(max_trials=1, directory=tmp_path)\n clf.fit(x_train, y_train, epochs=10)\n accuracy = clf.evaluate(x_test, y_test)[1]\n assert accuracy >= 0.98\n\n\ndef test_cifar10_accuracy_over_93(tmp_path):\n (x_train, y_train), (x_test, y_test) = cifar10.load_data()\n clf = ak.ImageClassifier(max_trials=3, directory=tmp_path)\n clf.fit(x_train, y_train, epochs=5)\n accuracy = clf.evaluate(x_test, y_test)[1]\n assert accuracy >= 0.93\n\n\ndef test_imdb_accuracy_over_92(tmp_path):\n (x_train, y_train), (x_test, y_test) = imdb_raw(num_instances=None)\n clf = ak.TextClassifier(max_trials=3, directory=tmp_path)\n clf.fit(x_train, y_train, batch_size=6, epochs=1)\n accuracy = clf.evaluate(x_test, y_test)[1]\n assert accuracy >= 0.92\n\n\ndef test_titaninc_accuracy_over_77(tmp_path):\n TRAIN_DATA_URL = (\n \"https://storage.googleapis.com/tf-datasets/titanic/train.csv\"\n )\n TEST_DATA_URL = (\n \"https://storage.googleapis.com/tf-datasets/titanic/eval.csv\"\n )\n\n train_file_path = keras.utils.get_file(\"train.csv\", TRAIN_DATA_URL)\n test_file_path = keras.utils.get_file(\"eval.csv\", TEST_DATA_URL)\n clf = ak.StructuredDataClassifier(max_trials=10, directory=tmp_path)\n\n clf.fit(train_file_path, \"survived\")\n\n accuracy = clf.evaluate(test_file_path, \"survived\")[1]\n assert accuracy >= 0.77\n" }, { "path": "benchmark/run.py", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport statistics\nimport sys\n\nfrom benchmark import experiments as exp_module\n\n\ndef generate_report(experiments):\n report = [\n \",\".join(\n [\n \"dataset_name\",\n \"average_time\",\n \"metrics_average\",\n \"metrics_standard_deviation\",\n ]\n )\n ]\n for experiment in experiments:\n total_times, metric_values = experiment.run(repeat_times=10)\n mean_time = statistics.mean(total_times)\n mean = statistics.mean(metric_values)\n std = statistics.stdev(metric_values)\n report.append(\n \",\".join([experiment.name, str(mean_time), str(mean), str(std)])\n )\n return \"\\n\".join(report)\n\n\ndef main(argv):\n task = sys.argv[1]\n path = sys.argv[2]\n report = generate_report(exp_module.get_experiments(task))\n with open(path, \"w\") as file:\n file.write(report)\n\n\nif __name__ == \"__main__\":\n main(sys.argv)\n" }, { "path": "codecov.yml", "content": "coverage:\n status:\n project:\n default:\n target: 100%\n patch:\n default:\n target: 100%\n" }, { "path": "docker/Dockerfile", "content": "FROM tensorflow/tensorflow:2.3.0\nWORKDIR /opt/autokeras\nCOPY . .\nRUN python -m pip install --no-cache-dir --editable .\nWORKDIR /work\n" }, { "path": "docker/Makefile", "content": "help:\n\t@cat Makefile\n\nDATA?=\"${HOME}/Data\"\nGPUS?=all\nDOCKER_FILE?=Dockerfile\nDOCKER=docker\nTF_VERSION=2.3.0\nTEST=tests/\nSRC?=$(shell dirname `pwd`)\n\nbuild:\n\tdocker build -t autokeras --build-arg TF_VERSION=$(TF_VERSION) -f $(DOCKER_FILE) .\n\nbash: build\n\t$(DOCKER) run --gpus $(GPUS) -it -v $(SRC):/src/workspace -v $(DATA):/data autokeras bash\n\nipython: build\n\t$(DOCKER) run --gpus $(GPUS) -it -v $(SRC):/src/workspace -v $(DATA):/data --env autokeras ipython\n\nnotebook: build\n\t$(DOCKER) run --gpus $(GPUS) -it -v $(SRC):/src/workspace -v $(DATA):/data --net=host --env autokeras\n\ntest: build\n\t$(DOCKER) run --gpus $(GPUS) -it -v $(SRC):/src/workspace -v $(DATA):/data --env autokeras py.test $(TEST)\n\n" }, { "path": "docker/README.md", "content": "# Using Autokeras via Docker\n\nThis directory contains `Dockerfile` to make it easy to get up and running with\nAutokeras via [Docker](http://www.docker.com/).\n\n## Installing Docker\n\nGeneral installation instructions are\n[on the Docker site](https://docs.docker.com/installation/), but we give some\nquick links here:\n\n* [OSX](https://docs.docker.com/installation/mac/): [docker toolbox](https://www.docker.com/toolbox)\n* [ubuntu](https://docs.docker.com/installation/ubuntulinux/)\n\n## Running the container\n\nWe are using `Makefile` to simplify docker commands within make commands.\n\nBuild the container and start a Jupyter Notebook\n\n $ make notebook\n\nBuild the container and start an iPython shell\n\n $ make ipython\n\nBuild the container and start a bash\n\n $ make bash\n\nFor GPU support install NVIDIA drivers (ideally latest) and\n[nvidia-docker](https://github.com/NVIDIA/nvidia-docker). Run using\n\n $ make notebook GPU=all # or [ipython, bash]\n\nMount a volume for external data sets\n\n $ make DATA=~/mydata\n\nPrints all make tasks\n\n $ make help\n\nNote: If you would have a problem running nvidia-docker you may try the old way\nwe have used. But it is not recommended. If you find a bug in the nvidia-docker report\nit there please and try using the nvidia-docker as described above.\n\n $ export CUDA_SO=$(\\ls /usr/lib/x86_64-linux-gnu/libcuda.* | xargs -I{} echo '-v {}:{}')\n $ export DEVICES=$(\\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')\n $ docker run -it -p 8888:8888 $CUDA_SO $DEVICES gcr.io/tensorflow/tensorflow:latest-gpu\n" }, { "path": "docker/devel.Dockerfile", "content": "FROM ubuntu:18.04\n\nRUN mkdir /app\n\nWORKDIR /app\n\n\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n python3.6 \\\n curl \\\n ca-certificates \\\n build-essential \\\n python3.6-dev \\\n python3-distutils \\\n git \\\n gcc \\\n && \\\n apt-get clean && \\\n rm -rf /var/lib/apt/lists/*\n\nRUN curl -O https://bootstrap.pypa.io/get-pip.py && \\\n python3.6 get-pip.py && \\\n rm get-pip.py\n\nRUN git clone https://github.com/keras-team/autokeras.git\n\nRUN python3.6 -m pip install -U keras\n\nRUN cd autokeras && python3.6 -m pip install -U . && cd ..\n\nRUN rm -rf autokeras\n\nCMD /bin/bash\n" }, { "path": "docker/pre-commit.Dockerfile", "content": "FROM python:3.7\n\nRUN pip install flake8 black isort\n\nWORKDIR /autokeras\nCMD [\"python\", \"docker/pre_commit.py\"]\n" }, { "path": "docker/pre_commit.py", "content": "from subprocess import CalledProcessError\nfrom subprocess import check_call\n\n\ndef check_bash_call(string):\n check_call([\"bash\", \"-c\", string])\n\n\ndef _run_format_and_flake8():\n files_changed = False\n\n try:\n check_bash_call(\"sh shell/lint.sh\")\n except CalledProcessError:\n check_bash_call(\"sh shell/format.sh\")\n files_changed = True\n\n if files_changed:\n print(\"Some files have changed.\")\n print(\"Please do git add and git commit again\")\n else:\n print(\"No formatting needed.\")\n\n if files_changed:\n exit(1)\n\n\ndef run_format_and_flake8():\n try:\n _run_format_and_flake8()\n except CalledProcessError as error:\n print(\"Pre-commit returned exit code\", error.returncode)\n exit(error.returncode)\n\n\nif __name__ == \"__main__\":\n run_format_and_flake8()\n" }, { "path": "docs/README.md", "content": "# AutoKeras Documentation\n\nThe source for AutoKeras documentation is in this directory.\nOur documentation uses extended Markdown, as implemented by [MkDocs](http://mkdocs.org).\n\n## Building the documentation\n\n- Install dependencies: `pip install -r docs/requirements.txt`\n- `pip install -e .` to make sure that Python will import your modified version of AutoKeras.\n- From the root directory, `cd` into the `docs/` folder and run:\n - `python autogen.py`\n - `mkdocs serve` # Starts a local webserver: [localhost:8000](http://localhost:8000)\n - `mkdocs build` # Builds a static site in `site/` directory\n\n## Generate contributors list\n- Prerequisites:\n - Install Pillow: `pip install Pillow`\n- Generate:\n - Run: `sh shell/contributors.sh`\n - The generated file is: `docs/templates/img/contributors.svg`" }, { "path": "docs/autogen.py", "content": "import os\nimport pathlib\nimport shutil\n\nimport keras_autodoc\nimport tutobooks\n\nPAGES = {\n \"image_classifier.md\": [\n \"autokeras.ImageClassifier\",\n \"autokeras.ImageClassifier.fit\",\n \"autokeras.ImageClassifier.predict\",\n \"autokeras.ImageClassifier.evaluate\",\n \"autokeras.ImageClassifier.export_model\",\n ],\n \"image_regressor.md\": [\n \"autokeras.ImageRegressor\",\n \"autokeras.ImageRegressor.fit\",\n \"autokeras.ImageRegressor.predict\",\n \"autokeras.ImageRegressor.evaluate\",\n \"autokeras.ImageRegressor.export_model\",\n ],\n \"text_classifier.md\": [\n \"autokeras.TextClassifier\",\n \"autokeras.TextClassifier.fit\",\n \"autokeras.TextClassifier.predict\",\n \"autokeras.TextClassifier.evaluate\",\n \"autokeras.TextClassifier.export_model\",\n ],\n \"text_regressor.md\": [\n \"autokeras.TextRegressor\",\n \"autokeras.TextRegressor.fit\",\n \"autokeras.TextRegressor.predict\",\n \"autokeras.TextRegressor.evaluate\",\n \"autokeras.TextRegressor.export_model\",\n ],\n \"structured_data_classifier.md\": [\n \"autokeras.StructuredDataClassifier\",\n \"autokeras.StructuredDataClassifier.fit\",\n \"autokeras.StructuredDataClassifier.predict\",\n \"autokeras.StructuredDataClassifier.evaluate\",\n \"autokeras.StructuredDataClassifier.export_model\",\n ],\n \"structured_data_regressor.md\": [\n \"autokeras.StructuredDataRegressor\",\n \"autokeras.StructuredDataRegressor.fit\",\n \"autokeras.StructuredDataRegressor.predict\",\n \"autokeras.StructuredDataRegressor.evaluate\",\n \"autokeras.StructuredDataRegressor.export_model\",\n ],\n \"auto_model.md\": [\n \"autokeras.AutoModel\",\n \"autokeras.AutoModel.fit\",\n \"autokeras.AutoModel.predict\",\n \"autokeras.AutoModel.evaluate\",\n \"autokeras.AutoModel.export_model\",\n ],\n \"base.md\": [\n \"autokeras.Node\",\n \"autokeras.Block\",\n \"autokeras.Block.build\",\n \"autokeras.Head\",\n ],\n \"node.md\": [\n \"autokeras.ImageInput\",\n \"autokeras.Input\",\n \"autokeras.TextInput\",\n \"autokeras.StructuredDataInput\",\n ],\n \"block.md\": [\n \"autokeras.ConvBlock\",\n \"autokeras.DenseBlock\",\n \"autokeras.Embedding\",\n \"autokeras.Merge\",\n \"autokeras.ResNetBlock\",\n \"autokeras.RNNBlock\",\n \"autokeras.SpatialReduction\",\n \"autokeras.TemporalReduction\",\n \"autokeras.XceptionBlock\",\n \"autokeras.StructuredDataBlock\",\n \"autokeras.ImageBlock\",\n \"autokeras.TextBlock\",\n \"autokeras.ImageAugmentation\",\n \"autokeras.Normalization\",\n \"autokeras.ClassificationHead\",\n \"autokeras.RegressionHead\",\n ],\n}\n\n\naliases_needed = [\n \"keras.callbacks.Callback\",\n \"keras.losses.Loss\",\n \"keras.metrics.Metric\",\n]\n\n\nROOT = \"http://autokeras.com/\"\n\nautokeras_dir = pathlib.Path(__file__).resolve().parents[1]\n\n\ndef py_to_nb_md(dest_dir):\n dir_path = \"py\"\n for file_path in os.listdir(\"py/\"):\n file_name = file_path\n py_path = os.path.join(dir_path, file_path)\n file_name_no_ext = os.path.splitext(file_name)[0]\n ext = os.path.splitext(file_name)[1]\n\n if ext != \".py\":\n continue\n\n nb_path = os.path.join(\"ipynb\", file_name_no_ext + \".ipynb\")\n md_path = os.path.join(dest_dir, \"tutorial\", file_name_no_ext + \".md\")\n\n tutobooks.py_to_md(py_path, nb_path, md_path, \"templates/img\")\n\n github_repo_dir = \"keras-team/autokeras/blob/master/docs/\"\n with open(md_path, \"r\") as md_file:\n button_lines = [\n \":material-link: \"\n \"[**View in Colab**](https://colab.research.google.com/github/\"\n + github_repo_dir\n + \"ipynb/\"\n + file_name_no_ext\n + \".ipynb\"\n + \")    \"\n # + '•'\n + \":octicons-mark-github-16: \"\n \"[**GitHub source**](https://github.com/\"\n + github_repo_dir\n + \"py/\"\n + file_name_no_ext\n + \".py)\",\n \"\\n\",\n ]\n md_content = \"\".join(button_lines) + \"\\n\" + md_file.read()\n\n with open(md_path, \"w\") as md_file:\n md_file.write(md_content)\n\n\ndef generate(dest_dir):\n template_dir = autokeras_dir / \"docs\" / \"templates\"\n doc_generator = keras_autodoc.DocumentationGenerator(\n PAGES,\n \"https://github.com/keras-team/autokeras/blob/master\",\n template_dir,\n autokeras_dir / \"examples\",\n extra_aliases=aliases_needed,\n )\n doc_generator.generate(dest_dir)\n readme = (autokeras_dir / \"README.md\").read_text()\n index = (template_dir / \"index.md\").read_text()\n index = index.replace(\"{{autogenerated}}\", readme[readme.find(\"##\") :])\n (dest_dir / \"index.md\").write_text(index, encoding=\"utf-8\")\n shutil.copyfile(\n autokeras_dir / \".github\" / \"CONTRIBUTING.md\",\n dest_dir / \"contributing.md\",\n )\n\n py_to_nb_md(dest_dir)\n\n\nif __name__ == \"__main__\":\n generate(autokeras_dir / \"docs\" / \"sources\")\n" }, { "path": "docs/ipynb/customized.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import keras\\n\",\n \"import numpy as np\\n\",\n \"import tree\\n\",\n \"from keras.datasets import mnist\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"In this tutorial, we show how to customize your search space with\\n\",\n \"[AutoModel](/auto_model/#automodel-class) and how to implement your own block\\n\",\n \"as search space. This API is mainly for advanced users who already know what\\n\",\n \"their model should look like.\\n\",\n \"\\n\",\n \"## Customized Search Space\\n\",\n \"First, let us see how we can build the following neural network using the\\n\",\n \"building blocks in AutoKeras.\\n\",\n \"\\n\",\n \"
\\n\",\n \"graph LR\\n\",\n \" id1(ImageInput) --> id2(Normalization)\\n\",\n \" id2 --> id3(Image Augmentation)\\n\",\n \" id3 --> id4(Convolutional)\\n\",\n \" id3 --> id5(ResNet V2)\\n\",\n \" id4 --> id6(Merge)\\n\",\n \" id5 --> id6\\n\",\n \" id6 --> id7(Classification Head)\\n\",\n \"
\\n\",\n \"\\n\",\n \"We can make use of the [AutoModel](/auto_model/#automodel-class) API in\\n\",\n \"AutoKeras to implemented as follows.\\n\",\n \"The usage is the same as the [Keras functional\\n\",\n \"API](https://keras.io/api/models/model/#with-the-functional-api).\\n\",\n \"Since this is just a demo, we use small amount of `max_trials` and `epochs`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.ImageInput()\\n\",\n \"output_node = ak.Normalization()(input_node)\\n\",\n \"output_node1 = ak.ConvBlock()(output_node)\\n\",\n \"output_node2 = ak.ResNetBlock(version=\\\"v2\\\")(output_node)\\n\",\n \"output_node = ak.Merge()([output_node1, output_node2])\\n\",\n \"output_node = ak.ClassificationHead()(output_node)\\n\",\n \"\\n\",\n \"auto_model = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"Whild building the model, the blocks used need to follow this topology:\\n\",\n \"`Preprocessor` -> `Block` -> `Head`. `Normalization` and `ImageAugmentation`\\n\",\n \"are `Preprocessor`s.\\n\",\n \"`ClassificationHead` is `Head`. The rest are `Block`s.\\n\",\n \"\\n\",\n \"In the code above, we use `ak.ResNetBlock(version='v2')` to specify the version\\n\",\n \"of ResNet to use. There are many other arguments to specify for each building\\n\",\n \"block. For most of the arguments, if not specified, they would be tuned\\n\",\n \"automatically. Please refer to the documentation links at the bottom of the\\n\",\n \"page for more details.\\n\",\n \"\\n\",\n \"Then, we prepare some data to run the model.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"(x_train, y_train), (x_test, y_test) = mnist.load_data()\\n\",\n \"print(x_train.shape) # (60000, 28, 28)\\n\",\n \"print(y_train.shape) # (60000,)\\n\",\n \"print(y_train[:3]) # array([7, 2, 1], dtype=uint8)\\n\",\n \"\\n\",\n \"# Feed the AutoModel with training data.\\n\",\n \"auto_model.fit(x_train[:100], y_train[:100], epochs=1)\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = auto_model.predict(x_test)\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(auto_model.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"For multiple input nodes and multiple heads search space, you can refer to\\n\",\n \"[this section](/tutorial/multi/#customized-search-space).\\n\",\n \"\\n\",\n \"## Validation Data\\n\",\n \"If you would like to provide your own validation data or change the ratio of\\n\",\n \"the validation data, please refer to the Validation Data section of the\\n\",\n \"tutorials of [Image\\n\",\n \"Classification](/tutorial/image_classification/#validation-data), [Text\\n\",\n \"Classification](/tutorial/text_classification/#validation-data), [Structured\\n\",\n \"Data\\n\",\n \"Classification](/tutorial/structured_data_classification/#validation-data),\\n\",\n \"[Multi-task and Multiple Validation](/tutorial/multi/#validation-data).\\n\",\n \"\\n\",\n \"## Implement New Block\\n\",\n \"\\n\",\n \"You can extend the [Block](/base/#block-class)\\n\",\n \"class to implement your own building blocks and use it with\\n\",\n \"[AutoModel](/auto_model/#automodel-class).\\n\",\n \"\\n\",\n \"The first step is to learn how to write a build function for\\n\",\n \"[KerasTuner](https://keras-team.github.io/keras-tuner/#usage-the-basics). You\\n\",\n \"need to override the [build function](/base/#build-method) of the block. The\\n\",\n \"following example shows how to implement a single Dense layer block whose\\n\",\n \"number of neurons is tunable.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"class SingleDenseLayerBlock(ak.Block):\\n\",\n \" def build(self, hp, inputs=None):\\n\",\n \" # Get the input_node from inputs.\\n\",\n \" input_node = tree.flatten(inputs)[0]\\n\",\n \" layer = keras.layers.Dense(\\n\",\n \" hp.Int(\\\"num_units\\\", min_value=32, max_value=512, step=32)\\n\",\n \" )\\n\",\n \" output_node = layer(input_node)\\n\",\n \" return output_node\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can connect it with other blocks and build it into an\\n\",\n \"[AutoModel](/auto_model/#automodel-class).\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Build the AutoModel\\n\",\n \"input_node = ak.Input()\\n\",\n \"output_node = SingleDenseLayerBlock()(input_node)\\n\",\n \"output_node = ak.RegressionHead()(output_node)\\n\",\n \"auto_model = ak.AutoModel(input_node, output_node, overwrite=True, max_trials=1)\\n\",\n \"# Prepare Data\\n\",\n \"num_instances = 100\\n\",\n \"x_train = np.random.rand(num_instances, 20).astype(np.float32)\\n\",\n \"y_train = np.random.rand(num_instances, 1).astype(np.float32)\\n\",\n \"x_test = np.random.rand(num_instances, 20).astype(np.float32)\\n\",\n \"y_test = np.random.rand(num_instances, 1).astype(np.float32)\\n\",\n \"# Train the model\\n\",\n \"auto_model.fit(x_train, y_train, epochs=1)\\n\",\n \"print(auto_model.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"\\n\",\n \"[AutoModel](/auto_model/#automodel-class)\\n\",\n \"\\n\",\n \"**Nodes**:\\n\",\n \"[ImageInput](/node/#imageinput-class),\\n\",\n \"[Input](/node/#input-class),\\n\",\n \"[TextInput](/node/#textinput-class).\\n\",\n \"[StructuredDataInput](/node/#structureddatainput-class),\\n\",\n \"\\n\",\n \"**Preprocessors**:\\n\",\n \"[FeatureEngineering](/block/#featureengineering-class),\\n\",\n \"[ImageAugmentation](/block/#imageaugmentation-class),\\n\",\n \"[LightGBM](/block/#lightgbm-class),\\n\",\n \"[Normalization](/block/#normalization-class),\\n\",\n \"\\n\",\n \"**Blocks**:\\n\",\n \"[ConvBlock](/block/#convblock-class),\\n\",\n \"[DenseBlock](/block/#denseblock-class),\\n\",\n \"[Embedding](/block/#embedding-class),\\n\",\n \"[Merge](/block/#merge-class),\\n\",\n \"[ResNetBlock](/block/#resnetblock-class),\\n\",\n \"[RNNBlock](/block/#rnnblock-class),\\n\",\n \"[SpatialReduction](/block/#spatialreduction-class),\\n\",\n \"[TemporalReduction](/block/#temporalreduction-class),\\n\",\n \"[XceptionBlock](/block/#xceptionblock-class),\\n\",\n \"[ImageBlock](/block/#imageblock-class),\\n\",\n \"[TextBlock](/block/#textblock-class).\\n\",\n \"[StructuredDataBlock](/block/#structureddatablock-class),\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"customized\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/export.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"from keras.datasets import mnist\\n\",\n \"from keras.models import load_model\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can easily export your model the best model found by AutoKeras as a Keras\\n\",\n \"Model.\\n\",\n \"\\n\",\n \"The following example uses [ImageClassifier](/image_classifier) as an example.\\n\",\n \"All the tasks and the [AutoModel](/auto_model/#automodel-class) has this\\n\",\n \"[export_model](/auto_model/#export_model-method) function.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"(x_train, y_train), (x_test, y_test) = mnist.load_data()\\n\",\n \"\\n\",\n \"# Initialize the image classifier.\\n\",\n \"clf = ak.ImageClassifier(\\n\",\n \" overwrite=True, max_trials=1\\n\",\n \") # Try only 1 model.(Increase accordingly)\\n\",\n \"# Feed the image classifier with training data.\\n\",\n \"clf.fit(x_train, y_train, epochs=1) # Change no of epochs to improve the model\\n\",\n \"# Export as a Keras Model.\\n\",\n \"model = clf.export_model()\\n\",\n \"\\n\",\n \"print(type(model)) # \\n\",\n \"\\n\",\n \"model.save(\\\"model_autokeras.keras\\\")\\n\",\n \"\\n\",\n \"\\n\",\n \"loaded_model = load_model(\\n\",\n \" \\\"model_autokeras.keras\\\", custom_objects=ak.CUSTOM_OBJECTS\\n\",\n \")\\n\",\n \"\\n\",\n \"predicted_y = loaded_model.predict(np.expand_dims(x_test, -1))\\n\",\n \"print(predicted_y)\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"export\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/image_classification.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"from keras.datasets import mnist\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the MNIST dataset as an\\n\",\n \"example\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"(x_train, y_train), (x_test, y_test) = mnist.load_data()\\n\",\n \"x_train = x_train[:100]\\n\",\n \"y_train = y_train[:100]\\n\",\n \"x_test = x_test[:100]\\n\",\n \"y_test = y_test[:100]\\n\",\n \"print(x_train.shape) # (60000, 28, 28)\\n\",\n \"print(y_train.shape) # (60000,)\\n\",\n \"print(y_train[:3]) # array([7, 2, 1], dtype=uint8)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the ImageClassifier.\\n\",\n \"It is recommended have more trials for more complicated datasets.\\n\",\n \"This is just a quick demo of MNIST, so we set max_trials to 1.\\n\",\n \"For the same reason, we set epochs to 10.\\n\",\n \"You can also leave the epochs unspecified for an adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the image classifier.\\n\",\n \"clf = ak.ImageClassifier(overwrite=True, max_trials=1)\\n\",\n \"# Feed the image classifier with training data.\\n\",\n \"clf.fit(x_train, y_train, epochs=1)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = clf.predict(x_test)\\n\",\n \"print(predicted_y)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(clf.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use validation_split to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=1,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set instead of splitting it from the\\n\",\n \"training data with validation_data.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 50000\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" epochs=1,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Customized Search Space\\n\",\n \"For advanced users, you may customize your search space by using AutoModel\\n\",\n \"instead of ImageClassifier. You can configure the ImageBlock for some\\n\",\n \"high-level configurations, e.g., block_type for the type of neural network to\\n\",\n \"search, normalize for whether to do data normalization, augment for whether to\\n\",\n \"do data augmentation. You can also do not specify these arguments, which would\\n\",\n \"leave the different choices to be tuned automatically. See the following\\n\",\n \"example for detail.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.ImageInput()\\n\",\n \"output_node = ak.ImageBlock(\\n\",\n \" # Only search ResNet architectures.\\n\",\n \" block_type=\\\"resnet\\\",\\n\",\n \" # Normalize the dataset.\\n\",\n \" normalize=True,\\n\",\n \" # Do not do data augmentation.\\n\",\n \" augment=False,\\n\",\n \")(input_node)\\n\",\n \"output_node = ak.ClassificationHead()(output_node)\\n\",\n \"clf = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"clf.fit(x_train, y_train, epochs=1)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The usage of AutoModel is similar to the functional API of Keras. Basically, you\\n\",\n \"are building a graph, whose edges are blocks and the nodes are intermediate\\n\",\n \"outputs of blocks. To add an edge from input_node to output_node with\\n\",\n \"output_node = ak.[some_block]([block_args])(input_node).\\n\",\n \"\\n\",\n \"You can even also use more fine grained blocks to customize the search space\\n\",\n \"even further. See the following example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.ImageInput()\\n\",\n \"output_node = ak.Normalization()(input_node)\\n\",\n \"output_node = ak.ImageAugmentation(horizontal_flip=False)(output_node)\\n\",\n \"output_node = ak.ResNetBlock(version=\\\"v2\\\")(output_node)\\n\",\n \"output_node = ak.ClassificationHead()(output_node)\\n\",\n \"clf = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"clf.fit(x_train, y_train, epochs=1)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[ImageClassifier](/image_classifier),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[ImageBlock](/block/#imageblock-class),\\n\",\n \"[Normalization](/block/#normalization-class),\\n\",\n \"[ImageAugmentation](/block/#image-augmentation-class),\\n\",\n \"[ResNetBlock](/block/#resnetblock-class),\\n\",\n \"[ImageInput](/node/#imageinput-class),\\n\",\n \"[ClassificationHead](/block/#classificationhead-class).\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"image_classification\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/image_regression.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"from keras.datasets import mnist\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"To make this tutorial easy to follow, we just treat MNIST dataset as a\\n\",\n \"regression dataset. It means we will treat prediction targets of MNIST dataset,\\n\",\n \"which are integers ranging from 0 to 9 as numerical values, so that they can be\\n\",\n \"directly used as the regression targets.\\n\",\n \"\\n\",\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the MNIST dataset as an\\n\",\n \"example\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"(x_train, y_train), (x_test, y_test) = mnist.load_data()\\n\",\n \"x_train = x_train[:100]\\n\",\n \"y_train = y_train[:100]\\n\",\n \"x_test = x_test[:100]\\n\",\n \"y_test = y_test[:100]\\n\",\n \"print(x_train.shape) # (60000, 28, 28)\\n\",\n \"print(y_train.shape) # (60000,)\\n\",\n \"print(y_train[:3]) # array([7, 2, 1], dtype=uint8)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the ImageRegressor. It is recommended have more\\n\",\n \"trials for more complicated datasets. This is just a quick demo of MNIST, so\\n\",\n \"we set max_trials to 1. For the same reason, we set epochs to 1. You can also\\n\",\n \"leave the epochs unspecified for an adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the image regressor.\\n\",\n \"reg = ak.ImageRegressor(overwrite=True, max_trials=1)\\n\",\n \"# Feed the image regressor with training data.\\n\",\n \"reg.fit(x_train, y_train, epochs=1)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = reg.predict(x_test)\\n\",\n \"print(predicted_y)\\n\",\n \"\\n\",\n \"\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(reg.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use validation_split to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=1,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set instead of splitting it from the\\n\",\n \"training data with validation_data.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 50000\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" epochs=2,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Customized Search Space\\n\",\n \"For advanced users, you may customize your search space by using AutoModel\\n\",\n \"instead of ImageRegressor. You can configure the ImageBlock for some high-level\\n\",\n \"configurations, e.g., block_type for the type of neural network to search,\\n\",\n \"normalize for whether to do data normalization, augment for whether to do data\\n\",\n \"augmentation. You can also do not specify these arguments, which would leave\\n\",\n \"the different choices to be tuned automatically. See the following example for\\n\",\n \"detail.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.ImageInput()\\n\",\n \"output_node = ak.ImageBlock(\\n\",\n \" # Only search ResNet architectures.\\n\",\n \" block_type=\\\"resnet\\\",\\n\",\n \" # Normalize the dataset.\\n\",\n \" normalize=False,\\n\",\n \" # Do not do data augmentation.\\n\",\n \" augment=False,\\n\",\n \")(input_node)\\n\",\n \"output_node = ak.RegressionHead()(output_node)\\n\",\n \"reg = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"reg.fit(x_train, y_train, epochs=1)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The usage of AutoModel is similar to the functional API of Keras. Basically,\\n\",\n \"you are building a graph, whose edges are blocks and the nodes are intermediate\\n\",\n \"outputs of blocks. To add an edge from input_node to output_node with\\n\",\n \"output_node = ak.[some_block]([block_args])(input_node).\\n\",\n \"\\n\",\n \"You can even also use more fine grained blocks to customize the search space\\n\",\n \"even further. See the following example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.ImageInput()\\n\",\n \"output_node = ak.Normalization()(input_node)\\n\",\n \"output_node = ak.ImageAugmentation(horizontal_flip=False)(output_node)\\n\",\n \"output_node = ak.ResNetBlock(version=\\\"v2\\\")(output_node)\\n\",\n \"output_node = ak.RegressionHead()(output_node)\\n\",\n \"reg = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"reg.fit(x_train, y_train, epochs=1)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[ImageRegressor](/image_regressor),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[ImageBlock](/block/#imageblock-class),\\n\",\n \"[Normalization](/block/#normalization-class),\\n\",\n \"[ImageAugmentation](/block/#image-augmentation-class),\\n\",\n \"[ResNetBlock](/block/#resnetblock-class),\\n\",\n \"[ImageInput](/node/#imageinput-class),\\n\",\n \"[RegressionHead](/block/#regressionhead-class).\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"image_regression\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/multi.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import numpy as np\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"In this tutorial we are making use of the\\n\",\n \"[AutoModel](/auto_model/#automodel-class)\\n\",\n \" API to show how to handle multi-modal data and multi-task.\\n\",\n \"\\n\",\n \"## What is multi-modal?\\n\",\n \"\\n\",\n \"Multi-modal data means each data instance has multiple forms of information.\\n\",\n \"For example, a photo can be saved as a image. Besides the image, it may also\\n\",\n \"have when and where it was taken as its attributes, which can be represented as\\n\",\n \"numerical data.\\n\",\n \"\\n\",\n \"## What is multi-task?\\n\",\n \"\\n\",\n \"Multi-task here we refer to we want to predict multiple targets with the same\\n\",\n \"input features. For example, we not only want to classify an image according to\\n\",\n \"its content, but we also want to regress its quality as a float number between\\n\",\n \"0 and 1.\\n\",\n \"\\n\",\n \"The following diagram shows an example of multi-modal and multi-task neural\\n\",\n \"network model.\\n\",\n \"\\n\",\n \"
\\n\",\n \"graph TD\\n\",\n \" id1(ImageInput) --> id3(Some Neural Network Model)\\n\",\n \" id2(Input) --> id3\\n\",\n \" id3 --> id4(ClassificationHead)\\n\",\n \" id3 --> id5(RegressionHead)\\n\",\n \"
\\n\",\n \"\\n\",\n \"It has two inputs the images and the numerical input data. Each image is\\n\",\n \"associated with a set of attributes in the numerical input data. From these\\n\",\n \"data, we are trying to predict the classification label and the regression value\\n\",\n \"at the same time.\\n\",\n \"\\n\",\n \"## Data Preparation\\n\",\n \"\\n\",\n \"To illustrate our idea, we generate some random image and numerical data as\\n\",\n \"the multi-modal data.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"num_instances = 10\\n\",\n \"# Generate image data.\\n\",\n \"image_data = np.random.rand(num_instances, 32, 32, 3).astype(np.float32)\\n\",\n \"# Generate numerical data.\\n\",\n \"numerical_data = np.random.rand(num_instances, 20).astype(np.float32)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"We also generate some multi-task targets for classification and regression.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Generate regression targets.\\n\",\n \"regression_target = np.random.rand(num_instances, 1).astype(np.float32)\\n\",\n \"# Generate classification labels of five classes.\\n\",\n \"classification_target = np.random.randint(5, size=num_instances)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Build and Train the Model\\n\",\n \"Then we initialize the multi-modal and multi-task model with\\n\",\n \"[AutoModel](/auto_model/#automodel-class).\\n\",\n \"Since this is just a demo, we use small amount of `max_trials` and `epochs`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the multi with multiple inputs and outputs.\\n\",\n \"model = ak.AutoModel(\\n\",\n \" inputs=[ak.ImageInput(), ak.Input()],\\n\",\n \" outputs=[\\n\",\n \" ak.RegressionHead(metrics=[\\\"mae\\\"]),\\n\",\n \" ak.ClassificationHead(\\n\",\n \" loss=\\\"categorical_crossentropy\\\", metrics=[\\\"accuracy\\\"]\\n\",\n \" ),\\n\",\n \" ],\\n\",\n \" overwrite=True,\\n\",\n \" max_trials=2,\\n\",\n \")\\n\",\n \"# Fit the model with prepared data.\\n\",\n \"model.fit(\\n\",\n \" [image_data, numerical_data],\\n\",\n \" [regression_target, classification_target],\\n\",\n \" epochs=1,\\n\",\n \" batch_size=3,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data.\\n\",\n \"As shown in the example below, you can use `validation_split` to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"model.fit(\\n\",\n \" [image_data, numerical_data],\\n\",\n \" [regression_target, classification_target],\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=1,\\n\",\n \" batch_size=3,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set\\n\",\n \"instead of splitting it from the training data with `validation_data`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 5\\n\",\n \"\\n\",\n \"image_val = image_data[split:]\\n\",\n \"numerical_val = numerical_data[split:]\\n\",\n \"regression_val = regression_target[split:]\\n\",\n \"classification_val = classification_target[split:]\\n\",\n \"\\n\",\n \"image_data = image_data[:split]\\n\",\n \"numerical_data = numerical_data[:split]\\n\",\n \"regression_target = regression_target[:split]\\n\",\n \"classification_target = classification_target[:split]\\n\",\n \"\\n\",\n \"model.fit(\\n\",\n \" [image_data, numerical_data],\\n\",\n \" [regression_target, classification_target],\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(\\n\",\n \" [image_val, numerical_val],\\n\",\n \" [regression_val, classification_val],\\n\",\n \" ),\\n\",\n \" epochs=1,\\n\",\n \" batch_size=3,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Customized Search Space\\n\",\n \"You can customize your search space.\\n\",\n \"The following figure shows the search space we want to define.\\n\",\n \"\\n\",\n \"
\\n\",\n \"graph LR\\n\",\n \" id1(ImageInput) --> id2(Normalization)\\n\",\n \" id2 --> id3(Image Augmentation)\\n\",\n \" id3 --> id4(Convolutional)\\n\",\n \" id3 --> id5(ResNet V2)\\n\",\n \" id4 --> id6(Merge)\\n\",\n \" id5 --> id6\\n\",\n \" id7(Input) --> id9(DenseBlock)\\n\",\n \" id6 --> id10(Merge)\\n\",\n \" id9 --> id10\\n\",\n \" id10 --> id11(Classification Head)\\n\",\n \" id10 --> id12(Regression Head)\\n\",\n \"
\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node1 = ak.ImageInput()\\n\",\n \"output_node = ak.Normalization()(input_node1)\\n\",\n \"output_node = ak.ImageAugmentation()(output_node)\\n\",\n \"output_node1 = ak.ConvBlock()(output_node)\\n\",\n \"output_node2 = ak.ResNetBlock(version=\\\"v2\\\")(output_node)\\n\",\n \"output_node1 = ak.Merge()([output_node1, output_node2])\\n\",\n \"\\n\",\n \"input_node2 = ak.Input()\\n\",\n \"output_node2 = ak.DenseBlock()(input_node2)\\n\",\n \"\\n\",\n \"output_node = ak.Merge()([output_node1, output_node2])\\n\",\n \"output_node1 = ak.ClassificationHead()(output_node)\\n\",\n \"output_node2 = ak.RegressionHead()(output_node)\\n\",\n \"\\n\",\n \"auto_model = ak.AutoModel(\\n\",\n \" inputs=[input_node1, input_node2],\\n\",\n \" outputs=[output_node1, output_node2],\\n\",\n \" overwrite=True,\\n\",\n \" max_trials=2,\\n\",\n \")\\n\",\n \"\\n\",\n \"image_data = np.random.rand(num_instances, 32, 32, 3).astype(np.float32)\\n\",\n \"numerical_data = np.random.rand(num_instances, 20).astype(np.float32)\\n\",\n \"regression_target = np.random.rand(num_instances, 1).astype(np.float32)\\n\",\n \"classification_target = np.random.randint(5, size=num_instances)\\n\",\n \"\\n\",\n \"auto_model.fit(\\n\",\n \" [image_data, numerical_data],\\n\",\n \" [classification_target, regression_target],\\n\",\n \" batch_size=3,\\n\",\n \" epochs=1,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[ImageInput](/node/#imageinput-class),\\n\",\n \"[Input](/node/#input-class),\\n\",\n \"[DenseBlock](/block/#denseblock-class),\\n\",\n \"[RegressionHead](/block/#regressionhead-class),\\n\",\n \"[ClassificationHead](/block/#classificationhead-class),\\n\",\n \"[CategoricalToNumerical](/block/#categoricaltonumerical-class).\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"multi\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/structured_data_classification.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import keras\\n\",\n \"import pandas as pd\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the [Titanic\\n\",\n \"dataset](https://www.kaggle.com/c/titanic) as an example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"TRAIN_DATA_URL = \\\"https://storage.googleapis.com/tf-datasets/titanic/train.csv\\\"\\n\",\n \"TEST_DATA_URL = \\\"https://storage.googleapis.com/tf-datasets/titanic/eval.csv\\\"\\n\",\n \"\\n\",\n \"train_file_path = keras.utils.get_file(\\\"train.csv\\\", TRAIN_DATA_URL)\\n\",\n \"test_file_path = keras.utils.get_file(\\\"eval.csv\\\", TEST_DATA_URL)\\n\",\n \"\\n\",\n \"# Load data into numpy arrays\\n\",\n \"train_df = pd.read_csv(train_file_path)\\n\",\n \"test_df = pd.read_csv(test_file_path)\\n\",\n \"\\n\",\n \"y_train = train_df[\\\"survived\\\"].values\\n\",\n \"x_train = train_df.drop(\\\"survived\\\", axis=1).values\\n\",\n \"\\n\",\n \"y_test = test_df[\\\"survived\\\"].values\\n\",\n \"x_test = test_df.drop(\\\"survived\\\", axis=1).values\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the\\n\",\n \"[StructuredDataClassifier](/structured_data_classifier).\\n\",\n \"As a quick demo, we set epochs to 10.\\n\",\n \"You can also leave the epochs unspecified for an adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the structured data classifier.\\n\",\n \"clf = ak.StructuredDataClassifier(\\n\",\n \" overwrite=True, max_trials=3\\n\",\n \") # It tries 3 different models.\\n\",\n \"# Feed the structured data classifier with training data.\\n\",\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" epochs=10,\\n\",\n \")\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = clf.predict(x_test)\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(clf.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also specify the column names and types for the data as follows. The\\n\",\n \"`column_names` is optional if the training data already have the column names,\\n\",\n \"e.g. pandas.DataFrame, CSV file. Any column, whose type is not specified will\\n\",\n \"be inferred from the training data.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the structured data classifier.\\n\",\n \"clf = ak.StructuredDataClassifier(\\n\",\n \" column_names=[\\n\",\n \" \\\"sex\\\",\\n\",\n \" \\\"age\\\",\\n\",\n \" \\\"n_siblings_spouses\\\",\\n\",\n \" \\\"parch\\\",\\n\",\n \" \\\"fare\\\",\\n\",\n \" \\\"class\\\",\\n\",\n \" \\\"deck\\\",\\n\",\n \" \\\"embark_town\\\",\\n\",\n \" \\\"alone\\\",\\n\",\n \" ],\\n\",\n \" column_types={\\\"sex\\\": \\\"categorical\\\", \\\"fare\\\": \\\"numerical\\\"},\\n\",\n \" max_trials=10, # It tries 10 different models.\\n\",\n \" overwrite=True,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use `validation_split` to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=10,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set\\n\",\n \"instead of splitting it from the training data with `validation_data`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 500\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" epochs=10,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[StructuredDataClassifier](/structured_data_classifier),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[StructuredDataBlock](/block/#structureddatablock-class),\\n\",\n \"[DenseBlock](/block/#denseblock-class),\\n\",\n \"[StructuredDataInput](/node/#structureddatainput-class),\\n\",\n \"[ClassificationHead](/block/#classificationhead-class),\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"structured_data_classification\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/structured_data_regression.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"from sklearn.datasets import fetch_california_housing\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the [California housing\\n\",\n \"dataset](\\n\",\n \"https://scikit-learn.org/stable/datasets/real_world.html#california-housing-dataset)\\n\",\n \"as an example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"house_dataset = fetch_california_housing()\\n\",\n \"train_size = int(house_dataset.data.shape[0] * 0.9)\\n\",\n \"\\n\",\n \"x_train = house_dataset.data[:train_size]\\n\",\n \"y_train = house_dataset.target[:train_size]\\n\",\n \"x_test = house_dataset.data[train_size:]\\n\",\n \"y_test = house_dataset.target[train_size:]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the\\n\",\n \"[StructuredDataRegressor](/structured_data_regressor).\\n\",\n \"As a quick demo, we set epochs to 10.\\n\",\n \"You can also leave the epochs unspecified for an adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the structured data regressor.\\n\",\n \"reg = ak.StructuredDataRegressor(\\n\",\n \" overwrite=True, max_trials=3\\n\",\n \") # It tries 3 different models.\\n\",\n \"# Feed the structured data regressor with training data.\\n\",\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" epochs=10,\\n\",\n \")\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = reg.predict(x_test)\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(reg.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also specify the column names and types for the data as follows. The\\n\",\n \"`column_names` is optional if the training data already have the column names,\\n\",\n \"e.g. pandas.DataFrame, CSV file. Any column, whose type is not specified will\\n\",\n \"be inferred from the training data.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the structured data regressor.\\n\",\n \"reg = ak.StructuredDataRegressor(\\n\",\n \" column_names=[\\n\",\n \" \\\"MedInc\\\",\\n\",\n \" \\\"HouseAge\\\",\\n\",\n \" \\\"AveRooms\\\",\\n\",\n \" \\\"AveBedrms\\\",\\n\",\n \" \\\"Population\\\",\\n\",\n \" \\\"AveOccup\\\",\\n\",\n \" \\\"Latitude\\\",\\n\",\n \" \\\"Longitude\\\",\\n\",\n \" ],\\n\",\n \" column_types={\\\"MedInc\\\": \\\"numerical\\\", \\\"Latitude\\\": \\\"numerical\\\"},\\n\",\n \" max_trials=10, # It tries 10 different models.\\n\",\n \" overwrite=True,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use `validation_split` to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=10,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set\\n\",\n \"instead of splitting it from the training data with `validation_data`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 500\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" epochs=10,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[StructuredDataRegressor](/structured_data_regressor),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[StructuredDataBlock](/block/#structureddatablock-class),\\n\",\n \"[DenseBlock](/block/#denseblock-class),\\n\",\n \"[StructuredDataInput](/node/#structureddatainput-class),\\n\",\n \"[RegressionHead](/block/#regressionhead-class),\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"structured_data_regression\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/text_classification.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"\\n\",\n \"import keras\\n\",\n \"import numpy as np\\n\",\n \"from sklearn.datasets import load_files\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the [IMDB\\n\",\n \"dataset](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification)\\n\",\n \"as an example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"dataset = keras.utils.get_file(\\n\",\n \" fname=\\\"aclImdb.tar.gz\\\",\\n\",\n \" origin=\\\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\\\",\\n\",\n \" extract=True,\\n\",\n \")\\n\",\n \"\\n\",\n \"# set path to dataset\\n\",\n \"IMDB_DATADIR = os.path.join(\\n\",\n \" os.path.dirname(dataset), \\\"aclImdb_extracted\\\", \\\"aclImdb\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"classes = [\\\"pos\\\", \\\"neg\\\"]\\n\",\n \"train_data = load_files(\\n\",\n \" os.path.join(IMDB_DATADIR, \\\"train\\\"), shuffle=True, categories=classes\\n\",\n \")\\n\",\n \"test_data = load_files(\\n\",\n \" os.path.join(IMDB_DATADIR, \\\"test\\\"), shuffle=False, categories=classes\\n\",\n \")\\n\",\n \"\\n\",\n \"x_train = np.array(train_data.data)[:100]\\n\",\n \"y_train = np.array(train_data.target)[:100]\\n\",\n \"x_test = np.array(test_data.data)[:100]\\n\",\n \"y_test = np.array(test_data.target)[:100]\\n\",\n \"\\n\",\n \"print(x_train.shape) # (25000,)\\n\",\n \"print(y_train.shape) # (25000, 1)\\n\",\n \"print(x_train[0][:50]) # this film was just brilliant casting\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the [TextClassifier](/text_classifier). As a quick\\n\",\n \"demo, we set epochs to 2. You can also leave the epochs unspecified for an\\n\",\n \"adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the text classifier.\\n\",\n \"clf = ak.TextClassifier(\\n\",\n \" overwrite=True, max_trials=1\\n\",\n \") # It only tries 1 model as a quick demo.\\n\",\n \"# Feed the text classifier with training data.\\n\",\n \"clf.fit(x_train, y_train, epochs=1, batch_size=2)\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = clf.predict(x_test)\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(clf.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use `validation_split` to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \" epochs=1,\\n\",\n \" batch_size=2,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set instead of splitting it from the\\n\",\n \"training data with `validation_data`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 5\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"clf.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" epochs=1,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" batch_size=2,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Customized Search Space\\n\",\n \"For advanced users, you may customize your search space by using\\n\",\n \"[AutoModel](/auto_model/#automodel-class) instead of\\n\",\n \"[TextClassifier](/text_classifier). You can configure the\\n\",\n \"[TextBlock](/block/#textblock-class) for some high-level configurations. You can\\n\",\n \"also do not specify these arguments, which would leave the different choices to\\n\",\n \"be tuned automatically. See the following example for detail.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.TextInput()\\n\",\n \"output_node = ak.TextBlock()(input_node)\\n\",\n \"output_node = ak.ClassificationHead()(output_node)\\n\",\n \"clf = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"clf.fit(x_train, y_train, epochs=1, batch_size=2)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[TextClassifier](/text_classifier),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[ConvBlock](/block/#convblock-class),\\n\",\n \"[TextInput](/node/#textinput-class),\\n\",\n \"[ClassificationHead](/block/#classificationhead-class).\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"text_classification\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/ipynb/text_regression.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"!export KERAS_BACKEND=\\\"torch\\\"\\n\",\n \"!pip install autokeras\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"\\n\",\n \"import keras\\n\",\n \"import numpy as np\\n\",\n \"from sklearn.datasets import load_files\\n\",\n \"\\n\",\n \"import autokeras as ak\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"To make this tutorial easy to follow, we just treat IMDB dataset as a\\n\",\n \"regression dataset. It means we will treat prediction targets of IMDB dataset,\\n\",\n \"which are 0s and 1s as numerical values, so that they can be directly used as\\n\",\n \"the regression targets.\\n\",\n \"\\n\",\n \"## A Simple Example\\n\",\n \"The first step is to prepare your data. Here we use the [IMDB\\n\",\n \"dataset](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification)\\n\",\n \"as an example.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"dataset = keras.utils.get_file(\\n\",\n \" fname=\\\"aclImdb.tar.gz\\\",\\n\",\n \" origin=\\\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\\\",\\n\",\n \" extract=True,\\n\",\n \")\\n\",\n \"\\n\",\n \"# set path to dataset\\n\",\n \"IMDB_DATADIR = os.path.join(\\n\",\n \" os.path.dirname(dataset), \\\"aclImdb_extracted\\\", \\\"aclImdb\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"classes = [\\\"pos\\\", \\\"neg\\\"]\\n\",\n \"train_data = load_files(\\n\",\n \" os.path.join(IMDB_DATADIR, \\\"train\\\"), shuffle=True, categories=classes\\n\",\n \")\\n\",\n \"test_data = load_files(\\n\",\n \" os.path.join(IMDB_DATADIR, \\\"test\\\"), shuffle=False, categories=classes\\n\",\n \")\\n\",\n \"\\n\",\n \"x_train = np.array(train_data.data)[:100]\\n\",\n \"y_train = np.array(train_data.target)[:100]\\n\",\n \"x_test = np.array(test_data.data)[:100]\\n\",\n \"y_test = np.array(test_data.target)[:100]\\n\",\n \"\\n\",\n \"print(x_train.shape) # (25000,)\\n\",\n \"print(y_train.shape) # (25000, 1)\\n\",\n \"print(x_train[0][:50]) # this film was just brilliant casting \"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"The second step is to run the [TextRegressor](/text_regressor). As a quick\\n\",\n \"demo, we set epochs to 2. You can also leave the epochs unspecified for an\\n\",\n \"adaptive number of epochs.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"# Initialize the text regressor.\\n\",\n \"reg = ak.TextRegressor(\\n\",\n \" overwrite=True, max_trials=1 # It tries 10 different models.\\n\",\n \")\\n\",\n \"# Feed the text regressor with training data.\\n\",\n \"reg.fit(x_train, y_train, epochs=1, batch_size=2)\\n\",\n \"# Predict with the best model.\\n\",\n \"predicted_y = reg.predict(x_test)\\n\",\n \"# Evaluate the best model with testing data.\\n\",\n \"print(reg.evaluate(x_test, y_test))\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Validation Data\\n\",\n \"By default, AutoKeras use the last 20% of training data as validation data. As\\n\",\n \"shown in the example below, you can use `validation_split` to specify the\\n\",\n \"percentage.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" # Split the training data and use the last 15% as validation data.\\n\",\n \" validation_split=0.15,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"You can also use your own validation set instead of splitting it from the\\n\",\n \"training data with `validation_data`.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"split = 5\\n\",\n \"x_val = x_train[split:]\\n\",\n \"y_val = y_train[split:]\\n\",\n \"x_train = x_train[:split]\\n\",\n \"y_train = y_train[:split]\\n\",\n \"reg.fit(\\n\",\n \" x_train,\\n\",\n \" y_train,\\n\",\n \" epochs=1,\\n\",\n \" # Use your own validation set.\\n\",\n \" validation_data=(x_val, y_val),\\n\",\n \" batch_size=2,\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Customized Search Space\\n\",\n \"For advanced users, you may customize your search space by using\\n\",\n \"[AutoModel](/auto_model/#automodel-class) instead of\\n\",\n \"[TextRegressor](/text_regressor). You can configure the\\n\",\n \"[TextBlock](/block/#textblock-class) for some high-level configurations. You can\\n\",\n \"also do not specify these arguments, which would leave the different choices to\\n\",\n \"be tuned automatically. See the following example for detail.\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab_type\": \"code\"\n },\n \"outputs\": [],\n \"source\": [\n \"input_node = ak.TextInput()\\n\",\n \"output_node = ak.TextBlock()(input_node)\\n\",\n \"output_node = ak.RegressionHead()(output_node)\\n\",\n \"reg = ak.AutoModel(\\n\",\n \" inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\\n\",\n \")\\n\",\n \"reg.fit(x_train, y_train, epochs=1, batch_size=2)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\"\n },\n \"source\": [\n \"## Reference\\n\",\n \"[TextRegressor](/text_regressor),\\n\",\n \"[AutoModel](/auto_model/#automodel-class),\\n\",\n \"[TextBlock](/block/#textblock-class),\\n\",\n \"[ConvBlock](/block/#convblock-class),\\n\",\n \"[TextInput](/node/#textinput-class),\\n\",\n \"[RegressionHead](/block/#regressionhead-class).\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"text_regression\",\n \"private_outputs\": false,\n \"provenance\": [],\n \"toc_visible\": true\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0\n}" }, { "path": "docs/keras_autodoc/__init__.py", "content": "from .autogen import DocumentationGenerator\nfrom .gathering_members import get_classes\nfrom .gathering_members import get_functions\nfrom .gathering_members import get_methods\n" }, { "path": "docs/keras_autodoc/autogen.py", "content": "import pathlib\nimport shutil\nfrom inspect import getdoc\nfrom inspect import isclass\nfrom typing import Dict\nfrom typing import List\nfrom typing import Union\nfrom typing import get_type_hints\n\nfrom . import utils\nfrom .docstring import process_docstring\nfrom .examples import copy_examples\nfrom .get_signatures import get_signature\n\n\nclass DocumentationGenerator:\n \"\"\"Generates the documentation.\n\n # Arguments\n\n pages: A dictionary. The keys are the files' paths, the values\n are lists of strings, functions /classes / methods names\n with dotted access to the object. For example,\n `pages = {'my_file.md': ['keras.layers.Dense']}` is valid.\n project_url: The url pointing to the module directory of your project on\n GitHub. This will be used to make a `[Sources]` link.\n template_dir: Where to put the markdown files which will be copied and\n filled in the destination directory. You should put files like\n `index.md` inside. If you want a markdown file to be filled with\n the docstring of a function, use the `{{autogenerated}}` tag inside,\n and then add the markdown file to the `pages` dictionary.\n example_dir: Where you store examples in your project. Usually\n standalone files with a markdown docstring at the top. Will be\n inserted in the docs.\n extra_aliases: When displaying type hints, it's possible that the full\n dotted path is displayed instead of alias. The aliases present in\n `pages` are used, but it may happen if you're using a third-party\n library. For example `tensorflow.python.ops.variables.Variable` is\n displayed instead of `tensorflow.Variable`. Here you have two\n solutions, either you provide the import keras-autodoc should\n follow:\n `extra_aliases=[\"tensorflow.Variable\"]`, either you provide a\n mapping to use\n `extra_aliases={\"tensorflow.python.ops.variables.Variable\":\n \"tf.Variable\"}`. The second option should be used if you want more\n control and that you don't want to respect the alias corresponding\n to the import (you can't do `import tf.Variable`). When giving a\n list, keras-autodoc will try to import the object from the string to\n understand what object you want to replace.\n max_signature_line_length: When displaying class and function\n signatures, keras-autodoc formats them using Black. This parameter\n controls the maximum line length of these signatures, and is passed\n directly through to Black.\n titles_size: `\"#\"` signs to put before a title in the generated\n markdown.\n \"\"\"\n\n def __init__(\n self,\n pages: Dict[str, list] = {},\n project_url: Union[str, Dict[str, str]] = None,\n template_dir=None,\n examples_dir=None,\n extra_aliases: Union[List[str], Dict[str, str]] = None,\n max_signature_line_length: int = 110,\n titles_size=\"###\",\n ):\n self.pages = pages\n self.project_url = project_url\n self.template_dir = template_dir\n self.examples_dir = examples_dir\n self.class_aliases = {}\n self._fill_aliases(extra_aliases)\n self.max_signature_line_length = max_signature_line_length\n self.titles_size = titles_size\n\n def generate(self, dest_dir):\n \"\"\"Generate the docs.\n\n # Arguments\n\n dest_dir: Where to put the resulting markdown files.\n \"\"\"\n dest_dir = pathlib.Path(dest_dir)\n print(\"Cleaning up existing sources directory.\")\n if dest_dir.exists():\n shutil.rmtree(dest_dir)\n\n print(\"Populating sources directory with templates.\")\n if self.template_dir:\n shutil.copytree(self.template_dir, dest_dir)\n\n for file_path, elements in self.pages.items():\n markdown_text = \"\"\n for element in elements:\n markdown_text += self._render(element)\n utils.insert_in_file(markdown_text, dest_dir / file_path)\n\n if self.examples_dir is not None:\n copy_examples(self.examples_dir, dest_dir / \"examples\")\n\n def process_docstring(self, docstring, types: dict = None):\n \"\"\"Can be overridden.\"\"\"\n processsed = process_docstring(docstring, types, self.class_aliases)\n return processsed\n\n def process_signature(self, signature):\n \"\"\"Can be overridden.\"\"\"\n return signature\n\n def _render(self, element):\n if isinstance(element, str):\n object_ = utils.import_object(element)\n if utils.ismethod(object_):\n # we remove the modules when displaying the methods\n signature_override = \".\".join(element.split(\".\")[-2:])\n else:\n signature_override = element\n else:\n signature_override = None\n object_ = element\n\n return self._render_from_object(object_, signature_override)\n\n def _render_from_object(self, object_, signature_override: str):\n subblocks = []\n if self.project_url is not None:\n subblocks.append(utils.make_source_link(object_, self.project_url))\n signature = get_signature(\n object_, signature_override, self.max_signature_line_length\n )\n signature = self.process_signature(signature)\n subblocks.append(f\"{self.titles_size} {object_.__name__}\\n\")\n subblocks.append(utils.code_snippet(signature))\n\n docstring = getdoc(object_)\n if docstring:\n if isclass(object_):\n type_hints = get_type_hints(object_.__init__)\n else:\n type_hints = get_type_hints(object_)\n docstring = self.process_docstring(docstring, type_hints)\n subblocks.append(docstring)\n return \"\\n\\n\".join(subblocks) + \"\\n\\n----\\n\\n\"\n\n def _fill_aliases(self, extra_aliases):\n for list_elements in self.pages.values():\n for element_as_str in list_elements:\n element = utils.import_object(element_as_str)\n if not isclass(element):\n continue\n true_dotted_path = utils.get_dotted_path(element)\n self.class_aliases[true_dotted_path] = element_as_str\n\n if isinstance(extra_aliases, dict):\n self.class_aliases.update(extra_aliases)\n elif isinstance(extra_aliases, list):\n for alias in extra_aliases:\n full_dotted_path = utils.get_dotted_path(\n utils.import_object(alias)\n )\n self.class_aliases[full_dotted_path] = alias\n" }, { "path": "docs/keras_autodoc/docstring.py", "content": "import itertools\nimport re\n\nfrom sphinx.util.typing import stringify_annotation\n\nfrom . import utils\n\n\ndef get_code_blocks(docstring):\n code_blocks = {}\n tmp = docstring[:]\n while \"```\" in tmp:\n tmp = tmp[tmp.find(\"```\") :]\n index = tmp[3:].find(\"```\") + 6\n snippet = tmp[:index]\n # Place marker in docstring for later reinjection.\n token = f\"$KERAS_AUTODOC_CODE_BLOCK_{len(code_blocks)}\"\n docstring = docstring.replace(snippet, token)\n code_blocks[token] = snippet\n tmp = tmp[index:]\n\n return code_blocks, docstring\n\n\ndef get_section_end(docstring, section_start):\n regex_indented_sections_end = re.compile(r\"\\S\\n+(\\S|$)\")\n end = re.search(regex_indented_sections_end, docstring[section_start:])\n section_end = section_start + end.end()\n if section_end == len(docstring):\n return section_end\n else:\n return section_end - 2\n\n\ndef get_google_style_sections_without_code(docstring):\n regex_indented_sections_start = re.compile(r\"\\n# .+?\\n\")\n\n google_style_sections = {}\n for i in itertools.count():\n match = re.search(regex_indented_sections_start, docstring)\n if match is None:\n break\n section_start = match.start() + 1\n section_end = get_section_end(docstring, section_start)\n google_style_section = docstring[section_start:section_end]\n token = f\"KERAS_AUTODOC_GOOGLE_STYLE_SECTION_{i}\"\n google_style_sections[token] = google_style_section\n docstring = utils.insert_in_string(\n docstring, token, section_start, section_end\n )\n return google_style_sections, docstring\n\n\ndef get_google_style_sections(docstring):\n # First, extract code blocks and process them.\n # The parsing is easier if the #, : and other symbols aren't there.\n code_blocks, docstring = get_code_blocks(docstring)\n\n google_style_sections, docstring = get_google_style_sections_without_code(\n docstring\n )\n\n docstring = reinject_strings(docstring, code_blocks)\n for section_token, section in google_style_sections.items():\n google_style_sections[section_token] = reinject_strings(\n section, code_blocks\n )\n return google_style_sections, docstring\n\n\ndef to_markdown(\n google_style_section: str, types: dict = None, aliases=None\n) -> str:\n end_first_line = google_style_section.find(\"\\n\")\n section_title = google_style_section[2:end_first_line]\n section_body = google_style_section[end_first_line + 1 :]\n section_body = utils.remove_indentation(section_body.strip())\n\n # it's a list of elements, a special formatting is applied.\n if section_title == \"Arguments\":\n section_body = format_as_markdown_list(section_body, types, aliases)\n elif section_title in (\"Attributes\", \"Raises\"):\n section_body = format_as_markdown_list(section_body)\n\n if section_body:\n return f\"__{section_title}__\\n\\n{section_body}\\n\"\n else:\n return f\"__{section_title}__\\n\"\n\n\ndef format_as_markdown_list(\n section_body, types: dict = None, aliases: dict = None\n):\n section_body = re.sub(r\"\\n([^ ].*?):\", r\"\\n- __\\1__:\", section_body)\n section_body = re.sub(r\"^([^ ].*?):\", r\"- __\\1__:\", section_body)\n\n # Optionally add type annotations to docstring\n if types:\n for arg, arg_type in types.items():\n type_hint_str = apply_aliases(\n stringify_annotation(arg_type), aliases\n )\n section_body = re.sub(\n rf\"(- __{arg}__)\", rf\"\\1 `{type_hint_str}`\", section_body\n )\n\n return section_body\n\n\ndef apply_aliases(string: str, aliases: dict):\n for dotted_path, alias in aliases.items():\n string = string.replace(dotted_path, alias)\n return string\n\n\ndef reinject_strings(target, strings_to_inject):\n for token, string_to_inject in strings_to_inject.items():\n target = target.replace(token, string_to_inject)\n return target\n\n\ndef process_docstring(docstring, types: dict = None, aliases=None):\n if docstring[-1] != \"\\n\":\n docstring += \"\\n\"\n google_style_sections, docstring = get_google_style_sections(docstring)\n\n for token, google_style_section in google_style_sections.items():\n markdown_section = to_markdown(google_style_section, types, aliases)\n docstring = docstring.replace(token, markdown_section)\n return docstring\n" }, { "path": "docs/keras_autodoc/examples.py", "content": "import os\nimport pathlib\n\n\ndef copy_examples(examples_dir, destination_dir):\n \"\"\"Copy the examples directory in the documentation.\n\n Prettify files by extracting the docstrings written in Markdown.\n \"\"\"\n pathlib.Path(destination_dir).mkdir(exist_ok=True)\n for file in os.listdir(examples_dir):\n if not file.endswith(\".py\"):\n continue\n module_path = os.path.join(examples_dir, file)\n docstring, starting_line = get_module_docstring(module_path)\n destination_file = os.path.join(destination_dir, file[:-2] + \"md\")\n with open(destination_file, \"w+\", encoding=\"utf-8\") as f_out, open(\n examples_dir / file, \"r+\", encoding=\"utf-8\"\n ) as f_in:\n f_out.write(docstring + \"\\n\\n\")\n\n # skip docstring\n for _ in range(starting_line):\n next(f_in)\n\n f_out.write(\"```python\\n\")\n # next line might be empty.\n line = next(f_in)\n if line != \"\\n\":\n f_out.write(line)\n\n # copy the rest of the file.\n for line in f_in:\n f_out.write(line)\n f_out.write(\"```\")\n\n\ndef get_module_docstring(filepath):\n \"\"\"Extract the module docstring.\n\n Also finds the line at which the docstring ends.\n \"\"\"\n co = compile(open(filepath, encoding=\"utf-8\").read(), filepath, \"exec\")\n if co.co_consts and isinstance(co.co_consts[0], str):\n docstring = co.co_consts[0]\n else:\n print(\"Could not get the docstring from \" + filepath)\n docstring = \"\"\n return docstring, co.co_firstlineno\n" }, { "path": "docs/keras_autodoc/gathering_members.py", "content": "import inspect\nfrom inspect import isclass\nfrom inspect import isfunction\nfrom inspect import isroutine\nfrom typing import List\n\nfrom .utils import import_object\n\n\ndef get_classes(module, exclude: List[str] = None, return_strings: bool = True):\n \"\"\"Get all the classes of a module.\n\n # Arguments\n\n module: The module to fetch the classes from. If it's a string, it\n should be in the dotted format. `'keras.layers'` for example.\n exclude: The names which will be excluded from the returned list. For\n example, `get_classes('keras.layers', exclude=['Dense', 'Conv2D'])`.\n return_strings: If False, the actual classes will be returned. Note that\n if you use aliases when building your docs, you should use strings.\n This is because the computed signature uses\n `__name__` and `__module__` if you don't provide a string as input.\n\n # Returns\n\n A list of strings or a list of classes.\n \"\"\"\n return _get_all_module_element(module, exclude, return_strings, True)\n\n\ndef get_functions(\n module, exclude: List[str] = None, return_strings: bool = True\n):\n \"\"\"Get all the functions of a module.\n\n # Arguments\n\n module: The module to fetch the functions from. If it's a string, it\n should be in the dotted format. `'keras.backend'` for example.\n exclude: The names which will be excluded from the returned list. For\n example, `get_functions('keras.backend', exclude=['max'])`.\n return_strings: If False, the actual functions will be returned. Note\n that if you use aliases when building your docs, you should use\n strings. This is because the computed signature uses `__name__` and\n `__module__` if you don't provide a string as input.\n\n # Returns\n\n A list of strings or a list of functions.\n \"\"\"\n return _get_all_module_element(module, exclude, return_strings, False)\n\n\ndef get_methods(cls, exclude=None, return_strings=True):\n \"\"\"Get all the method of a class.\n\n # Arguments\n\n cls: The class to fetch the methods from. If it's a\n string, it should be in the dotted format. `'keras.layers.Dense'`\n for example.\n exclude: The names which will be excluded from the returned list. For\n example, `get_methods('keras.Model', exclude=['save'])`.\n return_strings: If False, the actual methods will be returned. Note that\n if you use aliases when building your docs, you should use strings.\n This is because the computed signature uses\n `__name__` and `__module__` if you don't provide a string as input.\n\n # Returns\n\n A list of strings or a list of methods.\n \"\"\"\n if isinstance(cls, str):\n cls_str = cls\n cls = import_object(cls)\n else:\n cls_str = f\"{cls.__module__}.{cls.__name__}\"\n exclude = exclude or []\n methods = []\n for _, method in inspect.getmembers(cls, predicate=isroutine):\n if method.__name__[0] == \"_\" or method.__name__ in exclude:\n continue\n if return_strings:\n methods.append(f\"{cls_str}.{method.__name__}\")\n else:\n methods.append(method)\n return methods\n\n\ndef _get_all_module_element(module, exclude, return_strings, class_):\n if isinstance(module, str):\n module = import_object(module)\n exclude = exclude or []\n module_data = []\n for name in dir(module):\n module_member = getattr(module, name)\n if not (isfunction(module_member) or isclass(module_member)):\n continue\n if name[0] == \"_\" or name in exclude:\n continue\n if module.__name__ not in module_member.__module__:\n continue\n if module_member in module_data:\n continue\n if class_ and not isclass(module_member):\n continue\n if not class_ and not isfunction(module_member):\n continue\n if return_strings:\n module_data.append(f\"{module.__name__}.{name}\")\n else:\n module_data.append(module_member)\n module_data.sort(key=id)\n return module_data\n" }, { "path": "docs/keras_autodoc/get_signatures.py", "content": "import inspect\nimport warnings\n\nimport black\nfrom sphinx.util.inspect import signature\nfrom sphinx.util.inspect import stringify_signature\n\nfrom . import utils\n\n\ndef get_signature_start(function):\n \"\"\"For the Dense layer, it should return the string 'keras.layers.Dense'\"\"\"\n if utils.ismethod(function):\n prefix = f\"{utils.get_class_from_method(function).__name__}.\"\n else:\n try:\n prefix = f\"{function.__module__}.\"\n except AttributeError:\n warnings.warn(\n f\"function {function} has no module. \"\n f\"It will not be included in the signature.\"\n )\n prefix = \"\"\n\n return f\"{prefix}{function.__name__}\"\n\n\ndef get_signature_end(function):\n sig = signature(function)\n signature_end = stringify_signature(sig, show_annotation=False)\n if utils.ismethod(function):\n signature_end = signature_end.replace(\"(self, \", \"(\")\n signature_end = signature_end.replace(\"(self)\", \"()\")\n return signature_end\n\n\ndef get_function_signature(function, override=None, max_line_length: int = 110):\n if override is None:\n signature_start = get_signature_start(function)\n else:\n signature_start = override\n signature_end = get_signature_end(function)\n return format_signature(signature_start, signature_end, max_line_length)\n\n\ndef get_class_signature(cls, override=None, max_line_length: int = 110):\n if override is None:\n signature_start = f\"{cls.__module__}.{cls.__name__}\"\n else:\n signature_start = override\n signature_end = get_signature_end(cls.__init__)\n return format_signature(signature_start, signature_end, max_line_length)\n\n\ndef get_signature(object_, override=None, max_line_length: int = 110):\n if inspect.isclass(object_):\n return get_class_signature(object_, override, max_line_length)\n else:\n return get_function_signature(object_, override, max_line_length)\n\n\ndef format_signature(\n signature_start: str, signature_end: str, max_line_length: int = 110\n):\n \"\"\"pretty formatting to avoid long signatures on one single line\"\"\"\n\n # first, we make it look like a real function declaration.\n fake_signature_start = \"x\" * len(signature_start)\n fake_signature = fake_signature_start + signature_end\n fake_python_code = f\"def {fake_signature}:\\n pass\\n\"\n\n # we format with black\n mode = black.FileMode(line_length=max_line_length)\n formatted_fake_python_code = black.format_str(fake_python_code, mode=mode)\n\n # we make the final, multiline signature\n new_signature_end = extract_signature_end(formatted_fake_python_code)\n return signature_start + new_signature_end\n\n\ndef extract_signature_end(function_definition):\n start = function_definition.find(\"(\")\n stop = function_definition.rfind(\")\")\n return function_definition[start : stop + 1]\n" }, { "path": "docs/keras_autodoc/utils.py", "content": "import importlib\nimport inspect\nimport os\nimport re\n\n\ndef count_leading_spaces(s):\n ws = re.search(r\"\\S\", s)\n if ws:\n return ws.start()\n else:\n return 0\n\n\ndef insert_in_file(markdown_text, file_path):\n \"\"\"Save module page.\n\n Either insert content into existing page,\n or create page otherwise.\"\"\"\n if file_path.exists():\n template = file_path.read_text(encoding=\"utf-8\")\n if \"{{autogenerated}}\" not in template:\n raise RuntimeError(\n f\"Template found for {file_path} but missing \"\n f\"{{autogenerated}} tag.\"\n )\n markdown_text = template.replace(\"{{autogenerated}}\", markdown_text)\n print(\"...inserting autogenerated content into template:\", file_path)\n else:\n print(\"...creating new page with autogenerated content:\", file_path)\n os.makedirs(file_path.parent, exist_ok=True)\n file_path.write_text(markdown_text, encoding=\"utf-8\")\n\n\ndef code_snippet(snippet):\n return f\"```python\\n{snippet}\\n```\\n\"\n\n\ndef make_source_link(cls, project_url):\n if isinstance(project_url, dict):\n base_module = cls.__module__.split(\".\")[0]\n project_url = project_url[base_module]\n path = cls.__module__.replace(\".\", \"/\")\n line = inspect.getsourcelines(cls)[-1]\n return (\n f''\n f\"[[source]]({project_url}/{path}.py#L{line})\"\n f\"\"\n )\n\n\ndef format_classes_list(classes, page_name):\n for i in range(len(classes)):\n if not isinstance(classes[i], (list, tuple)):\n classes[i] = (classes[i], [])\n for class_, class_methods in classes:\n if not inspect.isclass(class_):\n # TODO: add a test for this\n raise TypeError(\n f\"{class_} was given in the class list \"\n f\"of {page_name} but {class_} is not a Python class.\"\n )\n return classes\n\n\ndef get_class_from_method(meth):\n \"\"\"See\n https://stackoverflow.com/questions/3589311/\n get-defining-class-of-unbound-method-object-in-python-3/\n 25959545#25959545\n \"\"\"\n if inspect.ismethod(meth):\n for cls in inspect.getmro(meth.__self__.__class__):\n if cls.__dict__.get(meth.__name__) is meth:\n return cls\n meth = meth.__func__ # fallback to __qualname__ parsing\n if inspect.isfunction(meth):\n cls = getattr(\n inspect.getmodule(meth),\n meth.__qualname__.split(\".\", 1)[0].rsplit(\".\", 1)[0],\n )\n if isinstance(cls, type):\n return cls\n return getattr(\n meth, \"__objclass__\", None\n ) # handle special descriptor objects\n\n\ndef ismethod(function):\n return get_class_from_method(function) is not None\n\n\ndef import_object(string: str):\n \"\"\"Import an object from a string.\n\n The object can be a function, class or method.\n For example: `'keras.layers.Dense.get_weights'` is valid.\n \"\"\"\n last_object_got = None\n seen_names = []\n for name in string.split(\".\"):\n seen_names.append(name)\n try:\n last_object_got = importlib.import_module(\".\".join(seen_names))\n except ModuleNotFoundError:\n last_object_got = getattr(last_object_got, name)\n return last_object_got\n\n\ndef get_type(object_) -> str:\n if inspect.isclass(object_):\n return \"class\"\n elif ismethod(object_):\n return \"method\"\n elif inspect.isfunction(object_):\n return \"function\"\n else:\n raise TypeError(\n f\"{object_} is detected as neither a class, a method nor\"\n f\"a function.\"\n )\n\n\ndef insert_in_string(target, string_to_insert, start, end):\n target_start_cut = target[:start]\n target_end_cut = target[end:]\n return target_start_cut + string_to_insert + target_end_cut\n\n\ndef remove_indentation(string):\n string = string.replace(\"\\n \", \"\\n\")\n if string[:4] == \" \":\n string = string[4:]\n return string\n\n\ndef get_dotted_path(class_):\n return f\"{class_.__module__}.{class_.__qualname__}\"\n" }, { "path": "docs/mkdocs.yml", "content": "site_name: AutoKeras\ntheme:\n favicon: '/img/favicon.png'\n logo: '/img/logo_white.svg'\n name: 'material'\n\ndocs_dir: sources\nrepo_url: https://github.com/keras-team/autokeras\nsite_url: http://autokeras.com\nedit_uri: \"\"\nsite_description: 'Documentation for AutoKeras.'\nmarkdown_extensions:\n - codehilite\n - pymdownx.superfences:\n custom_fences:\n - name: mermaid\n class: mermaid\n format: !!python/name:pymdownx.superfences.fence_div_format\n - pymdownx.emoji:\n emoji_index: !!python/name:material.extensions.emoji.twemoji\n emoji_generator: !!python/name:material.extensions.emoji.to_svg\n - admonition\n\nextra:\n analytics:\n provider: google\n property: G-GTF9QP8DFD\n\nextra_css:\n - stylesheets/extra.css\n\nextra_javascript:\n - https://unpkg.com/mermaid@8.4.4/dist/mermaid.min.js\n\nnav:\n - Home: index.md\n - Installation: install.md\n - Tutorials:\n - Overview: tutorial/overview.md\n - Image Classification: tutorial/image_classification.md\n - Image Regression: tutorial/image_regression.md\n - Text Classification: tutorial/text_classification.md\n - Text Regression: tutorial/text_regression.md\n - Structured Data Classification: tutorial/structured_data_classification.md\n - Structured Data Regression: tutorial/structured_data_regression.md\n - Multi-Modal and Multi-Task: tutorial/multi.md\n - Customized Model: tutorial/customized.md\n - Export Model: tutorial/export.md\n - Load Data from Disk: tutorial/load.md\n - FAQ: tutorial/faq.md\n - Contributing Guide: contributing.md\n - Documentation:\n - ImageClassifier: image_classifier.md\n - ImageRegressor: image_regressor.md\n - TextClassifier: text_classifier.md\n - TextRegressor: text_regressor.md\n - StructuredDataClassifier: structured_data_classifier.md\n - StructuredDataRegressor: structured_data_regressor.md\n - AutoModel: auto_model.md\n - Base Class: base.md\n - Node: node.md\n - Block: block.md\n - Utils: utils.md\n - About: about.md\n" }, { "path": "docs/py/customized.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport keras\nimport numpy as np\nimport tree\nfrom keras.datasets import mnist\n\nimport autokeras as ak\n\n\"\"\"\nIn this tutorial, we show how to customize your search space with\n[AutoModel](/auto_model/#automodel-class) and how to implement your own block\nas search space. This API is mainly for advanced users who already know what\ntheir model should look like.\n\n## Customized Search Space\nFirst, let us see how we can build the following neural network using the\nbuilding blocks in AutoKeras.\n\n
\ngraph LR\n id1(ImageInput) --> id2(Normalization)\n id2 --> id3(Image Augmentation)\n id3 --> id4(Convolutional)\n id3 --> id5(ResNet V2)\n id4 --> id6(Merge)\n id5 --> id6\n id6 --> id7(Classification Head)\n
\n\nWe can make use of the [AutoModel](/auto_model/#automodel-class) API in\nAutoKeras to implemented as follows.\nThe usage is the same as the [Keras functional\nAPI](https://keras.io/api/models/model/#with-the-functional-api).\nSince this is just a demo, we use small amount of `max_trials` and `epochs`.\n\"\"\"\n\n\ninput_node = ak.ImageInput()\noutput_node = ak.Normalization()(input_node)\noutput_node1 = ak.ConvBlock()(output_node)\noutput_node2 = ak.ResNetBlock(version=\"v2\")(output_node)\noutput_node = ak.Merge()([output_node1, output_node2])\noutput_node = ak.ClassificationHead()(output_node)\n\nauto_model = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\n\n\"\"\"\nWhild building the model, the blocks used need to follow this topology:\n`Preprocessor` -> `Block` -> `Head`. `Normalization` and `ImageAugmentation`\nare `Preprocessor`s.\n`ClassificationHead` is `Head`. The rest are `Block`s.\n\nIn the code above, we use `ak.ResNetBlock(version='v2')` to specify the version\nof ResNet to use. There are many other arguments to specify for each building\nblock. For most of the arguments, if not specified, they would be tuned\nautomatically. Please refer to the documentation links at the bottom of the\npage for more details.\n\nThen, we prepare some data to run the model.\n\"\"\"\n\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nprint(x_train.shape) # (60000, 28, 28)\nprint(y_train.shape) # (60000,)\nprint(y_train[:3]) # array([7, 2, 1], dtype=uint8)\n\n# Feed the AutoModel with training data.\nauto_model.fit(x_train[:100], y_train[:100], epochs=1)\n# Predict with the best model.\npredicted_y = auto_model.predict(x_test)\n# Evaluate the best model with testing data.\nprint(auto_model.evaluate(x_test, y_test))\n\n\"\"\"\nFor multiple input nodes and multiple heads search space, you can refer to\n[this section](/tutorial/multi/#customized-search-space).\n\n## Validation Data\nIf you would like to provide your own validation data or change the ratio of\nthe validation data, please refer to the Validation Data section of the\ntutorials of [Image\nClassification](/tutorial/image_classification/#validation-data), [Text\nClassification](/tutorial/text_classification/#validation-data), [Structured\nData\nClassification](/tutorial/structured_data_classification/#validation-data),\n[Multi-task and Multiple Validation](/tutorial/multi/#validation-data).\n\n## Implement New Block\n\nYou can extend the [Block](/base/#block-class)\nclass to implement your own building blocks and use it with\n[AutoModel](/auto_model/#automodel-class).\n\nThe first step is to learn how to write a build function for\n[KerasTuner](https://keras-team.github.io/keras-tuner/#usage-the-basics). You\nneed to override the [build function](/base/#build-method) of the block. The\nfollowing example shows how to implement a single Dense layer block whose\nnumber of neurons is tunable.\n\"\"\"\n\n\nclass SingleDenseLayerBlock(ak.Block):\n def build(self, hp, inputs=None):\n # Get the input_node from inputs.\n input_node = tree.flatten(inputs)[0]\n layer = keras.layers.Dense(\n hp.Int(\"num_units\", min_value=32, max_value=512, step=32)\n )\n output_node = layer(input_node)\n return output_node\n\n\n\"\"\"\nYou can connect it with other blocks and build it into an\n[AutoModel](/auto_model/#automodel-class).\n\"\"\"\n\n# Build the AutoModel\ninput_node = ak.Input()\noutput_node = SingleDenseLayerBlock()(input_node)\noutput_node = ak.RegressionHead()(output_node)\nauto_model = ak.AutoModel(input_node, output_node, overwrite=True, max_trials=1)\n# Prepare Data\nnum_instances = 100\nx_train = np.random.rand(num_instances, 20).astype(np.float32)\ny_train = np.random.rand(num_instances, 1).astype(np.float32)\nx_test = np.random.rand(num_instances, 20).astype(np.float32)\ny_test = np.random.rand(num_instances, 1).astype(np.float32)\n# Train the model\nauto_model.fit(x_train, y_train, epochs=1)\nprint(auto_model.evaluate(x_test, y_test))\n\n\"\"\"\n## Reference\n\n[AutoModel](/auto_model/#automodel-class)\n\n**Nodes**:\n[ImageInput](/node/#imageinput-class),\n[Input](/node/#input-class),\n[TextInput](/node/#textinput-class).\n[StructuredDataInput](/node/#structureddatainput-class),\n\n**Preprocessors**:\n[FeatureEngineering](/block/#featureengineering-class),\n[ImageAugmentation](/block/#imageaugmentation-class),\n[LightGBM](/block/#lightgbm-class),\n[Normalization](/block/#normalization-class),\n\n**Blocks**:\n[ConvBlock](/block/#convblock-class),\n[DenseBlock](/block/#denseblock-class),\n[Embedding](/block/#embedding-class),\n[Merge](/block/#merge-class),\n[ResNetBlock](/block/#resnetblock-class),\n[RNNBlock](/block/#rnnblock-class),\n[SpatialReduction](/block/#spatialreduction-class),\n[TemporalReduction](/block/#temporalreduction-class),\n[XceptionBlock](/block/#xceptionblock-class),\n[ImageBlock](/block/#imageblock-class),\n[TextBlock](/block/#textblock-class).\n[StructuredDataBlock](/block/#structureddatablock-class),\n\"\"\"\n" }, { "path": "docs/py/export.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport numpy as np\nfrom keras.datasets import mnist\nfrom keras.models import load_model\n\nimport autokeras as ak\n\n\"\"\"\nYou can easily export your model the best model found by AutoKeras as a Keras\nModel.\n\nThe following example uses [ImageClassifier](/image_classifier) as an example.\nAll the tasks and the [AutoModel](/auto_model/#automodel-class) has this\n[export_model](/auto_model/#export_model-method) function.\n\n\"\"\"\n\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\n\n# Initialize the image classifier.\nclf = ak.ImageClassifier(\n overwrite=True, max_trials=1\n) # Try only 1 model.(Increase accordingly)\n# Feed the image classifier with training data.\nclf.fit(x_train, y_train, epochs=1) # Change no of epochs to improve the model\n# Export as a Keras Model.\nmodel = clf.export_model()\n\nprint(type(model)) # \n\nmodel.save(\"model_autokeras.keras\")\n\n\nloaded_model = load_model(\n \"model_autokeras.keras\", custom_objects=ak.CUSTOM_OBJECTS\n)\n\npredicted_y = loaded_model.predict(np.expand_dims(x_test, -1))\nprint(predicted_y)\n" }, { "path": "docs/py/image_classification.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nfrom keras.datasets import mnist\n\nimport autokeras as ak\n\n\"\"\"\n## A Simple Example\nThe first step is to prepare your data. Here we use the MNIST dataset as an\nexample\n\"\"\"\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nx_train = x_train[:100]\ny_train = y_train[:100]\nx_test = x_test[:100]\ny_test = y_test[:100]\nprint(x_train.shape) # (60000, 28, 28)\nprint(y_train.shape) # (60000,)\nprint(y_train[:3]) # array([7, 2, 1], dtype=uint8)\n\n\"\"\"\nThe second step is to run the ImageClassifier.\nIt is recommended have more trials for more complicated datasets.\nThis is just a quick demo of MNIST, so we set max_trials to 1.\nFor the same reason, we set epochs to 10.\nYou can also leave the epochs unspecified for an adaptive number of epochs.\n\"\"\"\n\n# Initialize the image classifier.\nclf = ak.ImageClassifier(overwrite=True, max_trials=1)\n# Feed the image classifier with training data.\nclf.fit(x_train, y_train, epochs=1)\n\n\n# Predict with the best model.\npredicted_y = clf.predict(x_test)\nprint(predicted_y)\n\n\n# Evaluate the best model with testing data.\nprint(clf.evaluate(x_test, y_test))\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use validation_split to specify the\npercentage.\n\"\"\"\n\nclf.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=1,\n)\n\n\"\"\"\nYou can also use your own validation set instead of splitting it from the\ntraining data with validation_data.\n\"\"\"\n\nsplit = 50000\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nclf.fit(\n x_train,\n y_train,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n epochs=1,\n)\n\n\"\"\"\n## Customized Search Space\nFor advanced users, you may customize your search space by using AutoModel\ninstead of ImageClassifier. You can configure the ImageBlock for some\nhigh-level configurations, e.g., block_type for the type of neural network to\nsearch, normalize for whether to do data normalization, augment for whether to\ndo data augmentation. You can also do not specify these arguments, which would\nleave the different choices to be tuned automatically. See the following\nexample for detail.\n\"\"\"\n\ninput_node = ak.ImageInput()\noutput_node = ak.ImageBlock(\n # Only search ResNet architectures.\n block_type=\"resnet\",\n # Normalize the dataset.\n normalize=True,\n # Do not do data augmentation.\n augment=False,\n)(input_node)\noutput_node = ak.ClassificationHead()(output_node)\nclf = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nclf.fit(x_train, y_train, epochs=1)\n\n\"\"\"\nThe usage of AutoModel is similar to the functional API of Keras. Basically, you\nare building a graph, whose edges are blocks and the nodes are intermediate\noutputs of blocks. To add an edge from input_node to output_node with\noutput_node = ak.[some_block]([block_args])(input_node).\n\nYou can even also use more fine grained blocks to customize the search space\neven further. See the following example.\n\"\"\"\n\ninput_node = ak.ImageInput()\noutput_node = ak.Normalization()(input_node)\noutput_node = ak.ImageAugmentation(horizontal_flip=False)(output_node)\noutput_node = ak.ResNetBlock(version=\"v2\")(output_node)\noutput_node = ak.ClassificationHead()(output_node)\nclf = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nclf.fit(x_train, y_train, epochs=1)\n\n\"\"\"\n## Reference\n[ImageClassifier](/image_classifier),\n[AutoModel](/auto_model/#automodel-class),\n[ImageBlock](/block/#imageblock-class),\n[Normalization](/block/#normalization-class),\n[ImageAugmentation](/block/#image-augmentation-class),\n[ResNetBlock](/block/#resnetblock-class),\n[ImageInput](/node/#imageinput-class),\n[ClassificationHead](/block/#classificationhead-class).\n\"\"\"\n" }, { "path": "docs/py/image_regression.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nfrom keras.datasets import mnist\n\nimport autokeras as ak\n\n\"\"\"\nTo make this tutorial easy to follow, we just treat MNIST dataset as a\nregression dataset. It means we will treat prediction targets of MNIST dataset,\nwhich are integers ranging from 0 to 9 as numerical values, so that they can be\ndirectly used as the regression targets.\n\n## A Simple Example\nThe first step is to prepare your data. Here we use the MNIST dataset as an\nexample\n\"\"\"\n\n\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nx_train = x_train[:100]\ny_train = y_train[:100]\nx_test = x_test[:100]\ny_test = y_test[:100]\nprint(x_train.shape) # (60000, 28, 28)\nprint(y_train.shape) # (60000,)\nprint(y_train[:3]) # array([7, 2, 1], dtype=uint8)\n\n\"\"\"\nThe second step is to run the ImageRegressor. It is recommended have more\ntrials for more complicated datasets. This is just a quick demo of MNIST, so\nwe set max_trials to 1. For the same reason, we set epochs to 1. You can also\nleave the epochs unspecified for an adaptive number of epochs.\n\"\"\"\n\n# Initialize the image regressor.\nreg = ak.ImageRegressor(overwrite=True, max_trials=1)\n# Feed the image regressor with training data.\nreg.fit(x_train, y_train, epochs=1)\n\n\n# Predict with the best model.\npredicted_y = reg.predict(x_test)\nprint(predicted_y)\n\n\n# Evaluate the best model with testing data.\nprint(reg.evaluate(x_test, y_test))\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use validation_split to specify the\npercentage.\n\"\"\"\n\nreg.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=1,\n)\n\n\"\"\"\nYou can also use your own validation set instead of splitting it from the\ntraining data with validation_data.\n\"\"\"\n\nsplit = 50000\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nreg.fit(\n x_train,\n y_train,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n epochs=2,\n)\n\n\"\"\"\n## Customized Search Space\nFor advanced users, you may customize your search space by using AutoModel\ninstead of ImageRegressor. You can configure the ImageBlock for some high-level\nconfigurations, e.g., block_type for the type of neural network to search,\nnormalize for whether to do data normalization, augment for whether to do data\naugmentation. You can also do not specify these arguments, which would leave\nthe different choices to be tuned automatically. See the following example for\ndetail.\n\"\"\"\n\ninput_node = ak.ImageInput()\noutput_node = ak.ImageBlock(\n # Only search ResNet architectures.\n block_type=\"resnet\",\n # Normalize the dataset.\n normalize=False,\n # Do not do data augmentation.\n augment=False,\n)(input_node)\noutput_node = ak.RegressionHead()(output_node)\nreg = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nreg.fit(x_train, y_train, epochs=1)\n\n\"\"\"\nThe usage of AutoModel is similar to the functional API of Keras. Basically,\nyou are building a graph, whose edges are blocks and the nodes are intermediate\noutputs of blocks. To add an edge from input_node to output_node with\noutput_node = ak.[some_block]([block_args])(input_node).\n\nYou can even also use more fine grained blocks to customize the search space\neven further. See the following example.\n\"\"\"\n\ninput_node = ak.ImageInput()\noutput_node = ak.Normalization()(input_node)\noutput_node = ak.ImageAugmentation(horizontal_flip=False)(output_node)\noutput_node = ak.ResNetBlock(version=\"v2\")(output_node)\noutput_node = ak.RegressionHead()(output_node)\nreg = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nreg.fit(x_train, y_train, epochs=1)\n\n\"\"\"\n## Reference\n[ImageRegressor](/image_regressor),\n[AutoModel](/auto_model/#automodel-class),\n[ImageBlock](/block/#imageblock-class),\n[Normalization](/block/#normalization-class),\n[ImageAugmentation](/block/#image-augmentation-class),\n[ResNetBlock](/block/#resnetblock-class),\n[ImageInput](/node/#imageinput-class),\n[RegressionHead](/block/#regressionhead-class).\n\"\"\"\n" }, { "path": "docs/py/multi.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport numpy as np\n\nimport autokeras as ak\n\n\"\"\"\nIn this tutorial we are making use of the\n[AutoModel](/auto_model/#automodel-class)\n API to show how to handle multi-modal data and multi-task.\n\n## What is multi-modal?\n\nMulti-modal data means each data instance has multiple forms of information.\nFor example, a photo can be saved as a image. Besides the image, it may also\nhave when and where it was taken as its attributes, which can be represented as\nnumerical data.\n\n## What is multi-task?\n\nMulti-task here we refer to we want to predict multiple targets with the same\ninput features. For example, we not only want to classify an image according to\nits content, but we also want to regress its quality as a float number between\n0 and 1.\n\nThe following diagram shows an example of multi-modal and multi-task neural\nnetwork model.\n\n
\ngraph TD\n id1(ImageInput) --> id3(Some Neural Network Model)\n id2(Input) --> id3\n id3 --> id4(ClassificationHead)\n id3 --> id5(RegressionHead)\n
\n\nIt has two inputs the images and the numerical input data. Each image is\nassociated with a set of attributes in the numerical input data. From these\ndata, we are trying to predict the classification label and the regression value\nat the same time.\n\n## Data Preparation\n\nTo illustrate our idea, we generate some random image and numerical data as\nthe multi-modal data.\n\"\"\"\n\n\nnum_instances = 10\n# Generate image data.\nimage_data = np.random.rand(num_instances, 32, 32, 3).astype(np.float32)\n# Generate numerical data.\nnumerical_data = np.random.rand(num_instances, 20).astype(np.float32)\n\n\"\"\"\nWe also generate some multi-task targets for classification and regression.\n\"\"\"\n\n# Generate regression targets.\nregression_target = np.random.rand(num_instances, 1).astype(np.float32)\n# Generate classification labels of five classes.\nclassification_target = np.random.randint(5, size=num_instances)\n\n\"\"\"\n## Build and Train the Model\nThen we initialize the multi-modal and multi-task model with\n[AutoModel](/auto_model/#automodel-class).\nSince this is just a demo, we use small amount of `max_trials` and `epochs`.\n\"\"\"\n\n\n# Initialize the multi with multiple inputs and outputs.\nmodel = ak.AutoModel(\n inputs=[ak.ImageInput(), ak.Input()],\n outputs=[\n ak.RegressionHead(metrics=[\"mae\"]),\n ak.ClassificationHead(\n loss=\"categorical_crossentropy\", metrics=[\"accuracy\"]\n ),\n ],\n overwrite=True,\n max_trials=2,\n)\n# Fit the model with prepared data.\nmodel.fit(\n [image_data, numerical_data],\n [regression_target, classification_target],\n epochs=1,\n batch_size=3,\n)\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data.\nAs shown in the example below, you can use `validation_split` to specify the\npercentage.\n\"\"\"\n\nmodel.fit(\n [image_data, numerical_data],\n [regression_target, classification_target],\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=1,\n batch_size=3,\n)\n\n\"\"\"\nYou can also use your own validation set\ninstead of splitting it from the training data with `validation_data`.\n\"\"\"\n\nsplit = 5\n\nimage_val = image_data[split:]\nnumerical_val = numerical_data[split:]\nregression_val = regression_target[split:]\nclassification_val = classification_target[split:]\n\nimage_data = image_data[:split]\nnumerical_data = numerical_data[:split]\nregression_target = regression_target[:split]\nclassification_target = classification_target[:split]\n\nmodel.fit(\n [image_data, numerical_data],\n [regression_target, classification_target],\n # Use your own validation set.\n validation_data=(\n [image_val, numerical_val],\n [regression_val, classification_val],\n ),\n epochs=1,\n batch_size=3,\n)\n\n\"\"\"\n## Customized Search Space\nYou can customize your search space.\nThe following figure shows the search space we want to define.\n\n
\ngraph LR\n id1(ImageInput) --> id2(Normalization)\n id2 --> id3(Image Augmentation)\n id3 --> id4(Convolutional)\n id3 --> id5(ResNet V2)\n id4 --> id6(Merge)\n id5 --> id6\n id7(Input) --> id9(DenseBlock)\n id6 --> id10(Merge)\n id9 --> id10\n id10 --> id11(Classification Head)\n id10 --> id12(Regression Head)\n
\n\"\"\"\n\ninput_node1 = ak.ImageInput()\noutput_node = ak.Normalization()(input_node1)\noutput_node = ak.ImageAugmentation()(output_node)\noutput_node1 = ak.ConvBlock()(output_node)\noutput_node2 = ak.ResNetBlock(version=\"v2\")(output_node)\noutput_node1 = ak.Merge()([output_node1, output_node2])\n\ninput_node2 = ak.Input()\noutput_node2 = ak.DenseBlock()(input_node2)\n\noutput_node = ak.Merge()([output_node1, output_node2])\noutput_node1 = ak.ClassificationHead()(output_node)\noutput_node2 = ak.RegressionHead()(output_node)\n\nauto_model = ak.AutoModel(\n inputs=[input_node1, input_node2],\n outputs=[output_node1, output_node2],\n overwrite=True,\n max_trials=2,\n)\n\nimage_data = np.random.rand(num_instances, 32, 32, 3).astype(np.float32)\nnumerical_data = np.random.rand(num_instances, 20).astype(np.float32)\nregression_target = np.random.rand(num_instances, 1).astype(np.float32)\nclassification_target = np.random.randint(5, size=num_instances)\n\nauto_model.fit(\n [image_data, numerical_data],\n [classification_target, regression_target],\n batch_size=3,\n epochs=1,\n)\n\n\"\"\"\n## Reference\n[AutoModel](/auto_model/#automodel-class),\n[ImageInput](/node/#imageinput-class),\n[Input](/node/#input-class),\n[DenseBlock](/block/#denseblock-class),\n[RegressionHead](/block/#regressionhead-class),\n[ClassificationHead](/block/#classificationhead-class),\n[CategoricalToNumerical](/block/#categoricaltonumerical-class).\n\"\"\"\n" }, { "path": "docs/py/structured_data_classification.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport keras\nimport pandas as pd\n\nimport autokeras as ak\n\n\"\"\"\n## A Simple Example\nThe first step is to prepare your data. Here we use the [Titanic\ndataset](https://www.kaggle.com/c/titanic) as an example.\n\"\"\"\n\n\nTRAIN_DATA_URL = \"https://storage.googleapis.com/tf-datasets/titanic/train.csv\"\nTEST_DATA_URL = \"https://storage.googleapis.com/tf-datasets/titanic/eval.csv\"\n\ntrain_file_path = keras.utils.get_file(\"train.csv\", TRAIN_DATA_URL)\ntest_file_path = keras.utils.get_file(\"eval.csv\", TEST_DATA_URL)\n\n# Load data into numpy arrays\ntrain_df = pd.read_csv(train_file_path)\ntest_df = pd.read_csv(test_file_path)\n\ny_train = train_df[\"survived\"].values\nx_train = train_df.drop(\"survived\", axis=1).values\n\ny_test = test_df[\"survived\"].values\nx_test = test_df.drop(\"survived\", axis=1).values\n\n\n\"\"\"\nThe second step is to run the\n[StructuredDataClassifier](/structured_data_classifier).\nAs a quick demo, we set epochs to 10.\nYou can also leave the epochs unspecified for an adaptive number of epochs.\n\"\"\"\n\n# Initialize the structured data classifier.\nclf = ak.StructuredDataClassifier(\n overwrite=True, max_trials=3\n) # It tries 3 different models.\n# Feed the structured data classifier with training data.\nclf.fit(\n x_train,\n y_train,\n epochs=10,\n)\n# Predict with the best model.\npredicted_y = clf.predict(x_test)\n# Evaluate the best model with testing data.\nprint(clf.evaluate(x_test, y_test))\n\n\n\"\"\"\nYou can also specify the column names and types for the data as follows. The\n`column_names` is optional if the training data already have the column names,\ne.g. pandas.DataFrame, CSV file. Any column, whose type is not specified will\nbe inferred from the training data.\n\"\"\"\n\n# Initialize the structured data classifier.\nclf = ak.StructuredDataClassifier(\n column_names=[\n \"sex\",\n \"age\",\n \"n_siblings_spouses\",\n \"parch\",\n \"fare\",\n \"class\",\n \"deck\",\n \"embark_town\",\n \"alone\",\n ],\n column_types={\"sex\": \"categorical\", \"fare\": \"numerical\"},\n max_trials=10, # It tries 10 different models.\n overwrite=True,\n)\n\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use `validation_split` to specify the\npercentage.\n\"\"\"\n\nclf.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=10,\n)\n\n\"\"\"\nYou can also use your own validation set\ninstead of splitting it from the training data with `validation_data`.\n\"\"\"\n\nsplit = 500\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nclf.fit(\n x_train,\n y_train,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n epochs=10,\n)\n\n\n\"\"\"\n## Reference\n[StructuredDataClassifier](/structured_data_classifier),\n[AutoModel](/auto_model/#automodel-class),\n[StructuredDataBlock](/block/#structureddatablock-class),\n[DenseBlock](/block/#denseblock-class),\n[StructuredDataInput](/node/#structureddatainput-class),\n[ClassificationHead](/block/#classificationhead-class),\n\"\"\"\n" }, { "path": "docs/py/structured_data_regression.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nfrom sklearn.datasets import fetch_california_housing\n\nimport autokeras as ak\n\n\"\"\"\n## A Simple Example\nThe first step is to prepare your data. Here we use the [California housing\ndataset](\nhttps://scikit-learn.org/stable/datasets/real_world.html#california-housing-dataset)\nas an example.\n\"\"\"\n\n\nhouse_dataset = fetch_california_housing()\ntrain_size = int(house_dataset.data.shape[0] * 0.9)\n\nx_train = house_dataset.data[:train_size]\ny_train = house_dataset.target[:train_size]\nx_test = house_dataset.data[train_size:]\ny_test = house_dataset.target[train_size:]\n\n\"\"\"\nThe second step is to run the\n[StructuredDataRegressor](/structured_data_regressor).\nAs a quick demo, we set epochs to 10.\nYou can also leave the epochs unspecified for an adaptive number of epochs.\n\"\"\"\n\n# Initialize the structured data regressor.\nreg = ak.StructuredDataRegressor(\n overwrite=True, max_trials=3\n) # It tries 3 different models.\n# Feed the structured data regressor with training data.\nreg.fit(\n x_train,\n y_train,\n epochs=10,\n)\n# Predict with the best model.\npredicted_y = reg.predict(x_test)\n# Evaluate the best model with testing data.\nprint(reg.evaluate(x_test, y_test))\n\n\"\"\"\nYou can also specify the column names and types for the data as follows. The\n`column_names` is optional if the training data already have the column names,\ne.g. pandas.DataFrame, CSV file. Any column, whose type is not specified will\nbe inferred from the training data.\n\"\"\"\n\n# Initialize the structured data regressor.\nreg = ak.StructuredDataRegressor(\n column_names=[\n \"MedInc\",\n \"HouseAge\",\n \"AveRooms\",\n \"AveBedrms\",\n \"Population\",\n \"AveOccup\",\n \"Latitude\",\n \"Longitude\",\n ],\n column_types={\"MedInc\": \"numerical\", \"Latitude\": \"numerical\"},\n max_trials=10, # It tries 10 different models.\n overwrite=True,\n)\n\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use `validation_split` to specify the\npercentage.\n\"\"\"\n\nreg.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=10,\n)\n\n\"\"\"\nYou can also use your own validation set\ninstead of splitting it from the training data with `validation_data`.\n\"\"\"\n\nsplit = 500\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nreg.fit(\n x_train,\n y_train,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n epochs=10,\n)\n\n\n\"\"\"\n## Reference\n[StructuredDataRegressor](/structured_data_regressor),\n[AutoModel](/auto_model/#automodel-class),\n[StructuredDataBlock](/block/#structureddatablock-class),\n[DenseBlock](/block/#denseblock-class),\n[StructuredDataInput](/node/#structureddatainput-class),\n[RegressionHead](/block/#regressionhead-class),\n\"\"\"\n" }, { "path": "docs/py/text_classification.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport os\n\nimport keras\nimport numpy as np\nfrom sklearn.datasets import load_files\n\nimport autokeras as ak\n\n\"\"\"\n## A Simple Example\nThe first step is to prepare your data. Here we use the [IMDB\ndataset](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification)\nas an example.\n\"\"\"\n\n\ndataset = keras.utils.get_file(\n fname=\"aclImdb.tar.gz\",\n origin=\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\",\n extract=True,\n)\n\n# set path to dataset\nIMDB_DATADIR = os.path.join(\n os.path.dirname(dataset), \"aclImdb_extracted\", \"aclImdb\"\n)\n\nclasses = [\"pos\", \"neg\"]\ntrain_data = load_files(\n os.path.join(IMDB_DATADIR, \"train\"), shuffle=True, categories=classes\n)\ntest_data = load_files(\n os.path.join(IMDB_DATADIR, \"test\"), shuffle=False, categories=classes\n)\n\nx_train = np.array(train_data.data)[:100]\ny_train = np.array(train_data.target)[:100]\nx_test = np.array(test_data.data)[:100]\ny_test = np.array(test_data.target)[:100]\n\nprint(x_train.shape) # (25000,)\nprint(y_train.shape) # (25000, 1)\nprint(x_train[0][:50]) # this film was just brilliant casting\n\n\"\"\"\nThe second step is to run the [TextClassifier](/text_classifier). As a quick\ndemo, we set epochs to 2. You can also leave the epochs unspecified for an\nadaptive number of epochs.\n\"\"\"\n\n\n# Initialize the text classifier.\nclf = ak.TextClassifier(\n overwrite=True, max_trials=1\n) # It only tries 1 model as a quick demo.\n# Feed the text classifier with training data.\nclf.fit(x_train, y_train, epochs=1, batch_size=2)\n# Predict with the best model.\npredicted_y = clf.predict(x_test)\n# Evaluate the best model with testing data.\nprint(clf.evaluate(x_test, y_test))\n\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use `validation_split` to specify the\npercentage.\n\"\"\"\n\nclf.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=1,\n batch_size=2,\n)\n\n\"\"\"\nYou can also use your own validation set instead of splitting it from the\ntraining data with `validation_data`.\n\"\"\"\n\nsplit = 5\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nclf.fit(\n x_train,\n y_train,\n epochs=1,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n batch_size=2,\n)\n\n\"\"\"\n## Customized Search Space\nFor advanced users, you may customize your search space by using\n[AutoModel](/auto_model/#automodel-class) instead of\n[TextClassifier](/text_classifier). You can configure the\n[TextBlock](/block/#textblock-class) for some high-level configurations. You can\nalso do not specify these arguments, which would leave the different choices to\nbe tuned automatically. See the following example for detail.\n\"\"\"\n\n\ninput_node = ak.TextInput()\noutput_node = ak.TextBlock()(input_node)\noutput_node = ak.ClassificationHead()(output_node)\nclf = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nclf.fit(x_train, y_train, epochs=1, batch_size=2)\n\n\"\"\"\n## Reference\n[TextClassifier](/text_classifier),\n[AutoModel](/auto_model/#automodel-class),\n[ConvBlock](/block/#convblock-class),\n[TextInput](/node/#textinput-class),\n[ClassificationHead](/block/#classificationhead-class).\n\"\"\"\n" }, { "path": "docs/py/text_regression.py", "content": "\"\"\"shell\nexport KERAS_BACKEND=\"torch\"\npip install autokeras\n\"\"\"\n\nimport os\n\nimport keras\nimport numpy as np\nfrom sklearn.datasets import load_files\n\nimport autokeras as ak\n\n\"\"\"\nTo make this tutorial easy to follow, we just treat IMDB dataset as a\nregression dataset. It means we will treat prediction targets of IMDB dataset,\nwhich are 0s and 1s as numerical values, so that they can be directly used as\nthe regression targets.\n\n## A Simple Example\nThe first step is to prepare your data. Here we use the [IMDB\ndataset](https://keras.io/datasets/#imdb-movie-reviews-sentiment-classification)\nas an example.\n\"\"\"\n\n\ndataset = keras.utils.get_file(\n fname=\"aclImdb.tar.gz\",\n origin=\"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\",\n extract=True,\n)\n\n# set path to dataset\nIMDB_DATADIR = os.path.join(\n os.path.dirname(dataset), \"aclImdb_extracted\", \"aclImdb\"\n)\n\nclasses = [\"pos\", \"neg\"]\ntrain_data = load_files(\n os.path.join(IMDB_DATADIR, \"train\"), shuffle=True, categories=classes\n)\ntest_data = load_files(\n os.path.join(IMDB_DATADIR, \"test\"), shuffle=False, categories=classes\n)\n\nx_train = np.array(train_data.data)[:100]\ny_train = np.array(train_data.target)[:100]\nx_test = np.array(test_data.data)[:100]\ny_test = np.array(test_data.target)[:100]\n\nprint(x_train.shape) # (25000,)\nprint(y_train.shape) # (25000, 1)\nprint(x_train[0][:50]) # this film was just brilliant casting \n\n\"\"\"\nThe second step is to run the [TextRegressor](/text_regressor). As a quick\ndemo, we set epochs to 2. You can also leave the epochs unspecified for an\nadaptive number of epochs.\n\"\"\"\n\n\n# Initialize the text regressor.\nreg = ak.TextRegressor(\n overwrite=True, max_trials=1 # It tries 10 different models.\n)\n# Feed the text regressor with training data.\nreg.fit(x_train, y_train, epochs=1, batch_size=2)\n# Predict with the best model.\npredicted_y = reg.predict(x_test)\n# Evaluate the best model with testing data.\nprint(reg.evaluate(x_test, y_test))\n\n\n\"\"\"\n## Validation Data\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use `validation_split` to specify the\npercentage.\n\"\"\"\n\nreg.fit(\n x_train,\n y_train,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n)\n\n\"\"\"\nYou can also use your own validation set instead of splitting it from the\ntraining data with `validation_data`.\n\"\"\"\n\nsplit = 5\nx_val = x_train[split:]\ny_val = y_train[split:]\nx_train = x_train[:split]\ny_train = y_train[:split]\nreg.fit(\n x_train,\n y_train,\n epochs=1,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n batch_size=2,\n)\n\n\"\"\"\n## Customized Search Space\nFor advanced users, you may customize your search space by using\n[AutoModel](/auto_model/#automodel-class) instead of\n[TextRegressor](/text_regressor). You can configure the\n[TextBlock](/block/#textblock-class) for some high-level configurations. You can\nalso do not specify these arguments, which would leave the different choices to\nbe tuned automatically. See the following example for detail.\n\"\"\"\n\n\ninput_node = ak.TextInput()\noutput_node = ak.TextBlock()(input_node)\noutput_node = ak.RegressionHead()(output_node)\nreg = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\nreg.fit(x_train, y_train, epochs=1, batch_size=2)\n\n\"\"\"\n## Reference\n[TextRegressor](/text_regressor),\n[AutoModel](/auto_model/#automodel-class),\n[TextBlock](/block/#textblock-class),\n[ConvBlock](/block/#convblock-class),\n[TextInput](/node/#textinput-class),\n[RegressionHead](/block/#regressionhead-class).\n\"\"\"\n" }, { "path": "docs/requirements.txt", "content": "mkdocs\nmkdocs-material\npygments\njupyter\npymdown-extensions\nSphinx\nmarkdown\nblack" }, { "path": "docs/run_py_files.sh", "content": "#!/bin/bash\n\nSUCCESS_FILE=\"success.txt\"\nFAILURE_FILE=\"failure.txt\"\n\n# Clear the files\n> $SUCCESS_FILE\n> $FAILURE_FILE\n\nfor file in py/*.py; do\n if python3 \"$file\" > /dev/null 2>&1; then\n echo \"$file\" >> $SUCCESS_FILE\n else\n echo \"$file\" >> $FAILURE_FILE\n fi\ndone" }, { "path": "docs/templates/about.md", "content": "This package is developed by [DATA LAB](http://faculty.cs.tamu.edu/xiahu/) at Texas A&M University,\ncollaborating with [keras-team](https://github.com/keras-team) for version 1.0 and above.\n\n## Core Team\n\n[**Haifeng Jin**](https://github.com/haifeng-jin):\nCreated, designed and implemented the AutoKeras system. \nMaintainer.\n\n[**François Chollet**](https://github.com/fchollet):\nThe API and system architecture design for AutoKeras 1.0.\nCode reviews for pull requests.\n\n[**Qingquan Song**](https://github.com/song3134):\nDesigned the neural architecture search algorithms.\nImplemented the tabular data classification and regression module.\n\n[**Xia \"Ben\" Hu**](http://faculty.cs.tamu.edu/xiahu/):\nProject lead and maintainer.\n\n" }, { "path": "docs/templates/index.md", "content": "#\n\"drawing\"\n\n##\n\n{{autogenerated}}\n" }, { "path": "docs/templates/install.md", "content": "## Requirements\n\n**Python 3**: Follow the TensorFlow install steps to install Python 3.\n\n**Pip**: Follow the TensorFlow install steps to install Pip.\n\n**PyTorch >= 2.3.0**: AutoKeras is based on Keras. We recommend using the\nPyTorch backend.\nPlease follow [this page](https://pytorch.org/get-started/locally/) to install\nPyTorch.\n\n## Install AutoKeras\nAutoKeras only support **Python 3**.\nIf you followed previous steps to use virtualenv to install tensorflow,\nyou can just activate the virtualenv and use the following command to install AutoKeras. \n```\npip install git+https://github.com/keras-team/keras-tuner.git\npip install autokeras\n```\n\nIf you did not use virtualenv, and you use `python3` command to execute your python program,\nplease use the following command to install AutoKeras.\n```\npython3 -m pip install git+https://github.com/keras-team/keras-tuner.git\npython3 -m pip install autokeras\n```\n\n" }, { "path": "docs/templates/stylesheets/extra.css", "content": ":root>* {\n --md-primary-fg-color: #d00000;\n --md-accent-fg-color: #d00000;\n}" }, { "path": "docs/templates/tutorial/faq.md", "content": "## How to resume a previously killed run?\nThis feature is controlled by the `overwrite` argument of `AutoModel` or any other task APIs.\nIt is set to `False` by default,\nwhich means it would not overwrite the contents of the directory.\nIn other words, it will continue the previous fit.\n\nYou can just run the same code again.\nIt will automatically resume the previously killed run.\n\n## How to customize metrics and loss?\nPlease see the code example below.\n\n```python\nimport autokeras as ak\n\n\nclf = ak.ImageClassifier(\n max_trials=3,\n metrics=['mse'],\n loss='mse',\n)\n```\n\n## How to use customized metrics to select the best model?\nBy default, AutoKeras use validation loss as the metric for selecting the best model.\nBelow is a code example of using customized metric for selecting models.\nPlease read the comments for the details.\n\n```python\n# Implement your customized metric according to the tutorial.\n# https://keras.io/api/metrics/#creating-custom-metrics\nimport autokeras as ak\n\n\ndef f1_score(y_true, y_pred):\n ...\n\nclf = ak.ImageClassifier(\n max_trials=3,\n\n # Wrap the function into a Keras Tuner Objective \n # and pass it to AutoKeras.\n\n # Direction can be 'min' or 'max'\n # meaning we want to minimize or maximize the metric.\n\n # 'val_f1_score' is just add a 'val_' prefix\n # to the function name or the metric name.\n\n objective=kerastuner.Objective('val_f1_score', direction='max'),\n # Include it as one of the metrics.\n metrics=[f1_score],\n)\n```" }, { "path": "docs/templates/tutorial/overview.md", "content": "# AutoKeras 1.0 Tutorial\n\n## Supported Tasks\n\nAutoKeras supports several tasks with an extremely simple interface.\nYou can click the links below to see the detailed tutorial for each task.\n\n**Supported Tasks**:\n\n[Image Classification](/tutorial/image_classification)\n\n[Image Regression](/tutorial/image_regression)\n\n[Text Classification](/tutorial/text_classification)\n\n[Text Regression](/tutorial/text_regression)\n\n[Structured Data Classification](/tutorial/structured_data_classification)\n\n[Structured Data Regression](/tutorial/structured_data_regression)\n\n## Multi-Task and Multi-Modal Data\n\nIf you are dealing with multi-task or multi-modal dataset, you can refer to this\n[tutorial](/tutorial/multi) for details.\n\n\n## Customized Model\n\nFollow this [tutorial](/tutorial/customized), to use AutoKeras building blocks to quickly construct your own model.\nWith these blocks, you only need to specify the high-level architecture of your model.\nAutoKeras would search for the best detailed configuration for you.\nMoreover, you can override the base classes to create your own block.\nThe following are the links to the documentation of the predefined input nodes and blocks in AutoKeras.\n\n**Nodes**:\n\n[ImageInput](/node/#imageinput-class)\n\n[Input](/node/#input-class)\n\n[TextInput](/node/#textinput-class)\n\n[StructuredDataInput](/node/#structureddatainput-class)\n\n**Blocks**:\n\n[ImageAugmentation](/block/#imageaugmentation-class)\n\n[Normalization](/block/#normalization-class)\n\n[ConvBlock](/block/#convblock-class)\n\n[DenseBlock](/block/#denseblock-class)\n\n[Embedding](/block/#embedding-class)\n\n[Merge](/block/#merge-class)\n\n[ResNetBlock](/block/#resnetblock-class)\n\n[RNNBlock](/block/#rnnblock-class)\n\n[SpatialReduction](/block/#spatialreduction-class)\n\n[TemporalReduction](/block/#temporalreduction-class)\n\n[XceptionBlock](/block/#xceptionblock-class)\n\n[ImageBlock](/block/#imageblock-class)\n\n[TextBlock](/block/#textblock-class)\n\n[StructuredDataBlock](/block/#structureddatablock-class)\n\n[ClassificationHead](/block/#classificationhead-class)\n\n[RegressionHead](/block/#regressionhead-class)\n\n## Export Model\nYou can follow this [tutorial](/tutorial/export) to export the best model.\n" }, { "path": "docs/tutobooks.py", "content": "\"\"\"Keras tutobooks implementation.\n\nA tutobook is a tutorial available simultaneously as a notebook,\nas a Python script, and as a nicely rendered webpage.\n\nIts source-of-truth (for manual edition and version control) is\nits Python script form, but you can also create one by starting\nfrom a notebook and converting it with the command `nb2py`.\n\nText cells are stored in markdown-formatted comment blocks.\nthe first line (starting with \" * 3) may optionally contain a special\nannotation, one of:\n\n- invisible: do not render this block.\n- shell: execute this block while prefixing each line with `!`.\n\nThe script form should start with a header with the following fields:\nTitle:\nAuthor: (could be `Authors`: as well, and may contain markdown links)\nDate created: (date in yyyy/mm/dd format)\nLast modified: (date in yyyy/mm/dd format)\nDescription: (one-line text description)\n\n## How to add a new code example to Keras.io\n\nYou would typically start from an existing notebook.\n\nSave it to disk (let's say as `path_to_your_nb.ipynb`).\n`cd` to the `keras-io/scripts/` directory.\n\nThen run:\n\n```\npython tutobooks nb2py path_to_your_nb.ipynb ../examples/your_example.py\n```\n\nThis will create the file `examples/your_example.py`. Open it,\nfill in the headers, and generally edit it so that it looks nice.\n\nNOTE THAT THE CONVERSION SCRIPT MAY MAKE MISTAKES IN ITS ATTEMPTS\nTO SHORTEN LINES. MAKE SURE TO PROOFREAD THE GENERATED .py IN FULL.\nOr alternatively, make sure to keep your lines reasonably-sized (<90 char)\nto start with, so that the script won't have to shorten them.\n\nYou can then preview what it looks like when converted back again\nto ipynb by running:\n\n```\npython tutobooks py2nb ../examples/your_example.py preview.ipynb\n```\n\nNOTE THAT THIS COMMAND WILL ERROR OUT IF ANY CELLS TAKES TOO LONG\nTO EXECUTE. In that case, make your code lighter/faster.\nRemember that examples are meant to demonstrate workflows, not\ntrain state-of-the-art models. They should\nstay very lightweight.\n\nOpen the generated `preview.ipynb` and make sure it looks like what\nyou expect. If not, keep editing `your_example.py` until it does.\n\nFinally, submit a PR adding `examples/your_example.py`.\n\"\"\"\n\nimport json\nimport os\nimport random\nimport shutil\nimport sys\nfrom pathlib import Path\n\nTIMEOUT = 60 * 60\nMAX_LOC = 300\n\n\ndef nb_to_py(nb_path, py_path):\n f = open(nb_path)\n content = f.read()\n f.close()\n nb = json.loads(content)\n py = '\"\"\"\\n'\n py += \"Title: FILLME\\n\"\n py += \"Author: FILLME\\n\"\n py += \"Date created: FILLME\\n\"\n py += \"Last modified: FILLME\\n\"\n py += \"Description: FILLME\\n\"\n py += '\"\"\"\\n'\n for cell in nb[\"cells\"]:\n if cell[\"cell_type\"] == \"code\":\n # Is it a shell cell?\n if (\n cell[\"source\"]\n and cell[\"source\"][0]\n and cell[\"source\"][0][0] == \"!\"\n ):\n # It's a shell cell\n py += '\"\"\"shell\\n'\n py += \"\".join(cell[\"source\"]) + \"\\n\"\n py += '\"\"\"\\n\\n'\n else:\n # It's a Python cell\n py += \"\".join(cell[\"source\"]) + \"\\n\\n\"\n elif cell[\"cell_type\"] == \"markdown\":\n py += '\"\"\"\\n'\n py += \"\".join(cell[\"source\"]) + \"\\n\"\n py += '\"\"\"\\n\\n'\n # Save file\n f = open(py_path, \"w\")\n f.write(py)\n f.close()\n # Format file with Black\n os.system(\"black \" + py_path)\n # Shorten lines\n py = open(py_path).read()\n try:\n py = _shorten_lines(py)\n finally:\n f = open(py_path, \"w\")\n f.write(py)\n f.close()\n\n\ndef py_to_nb(py_path, nb_path, fill_outputs=True):\n f = open(py_path)\n py = f.read()\n f.close()\n # validate(py)\n\n # header, _, py, tag = _get_next_script_element(py)\n # attributes = _parse_header(header)\n cells = []\n loc = 0\n # Write first header cell\n # header_cell = {\n # \"cell_type\": \"markdown\",\n # \"source\": [\n # \"# \" + attributes[\"title\"] + \"\\n\",\n # \"\\n\",\n # \"**\" + attributes[\"auth_field\"] + \":** \" + attributes[\"author\"] +\"
\\n\",\n # \"**Date created:** \" + attributes[\"date_created\"] + \"
\\n\",\n # \"**Last modified:** \" + attributes[\"last_modified\"] + \"
\\n\",\n # \"**Description:** \" + attributes[\"description\"],\n # ],\n # \"metadata\": {\"colab_type\": \"text\"},\n # }\n # cells.append(header_cell)\n while py:\n e, cell_type, py, tag = _get_next_script_element(py)\n lines = e.split(\"\\n\")\n\n if all(line == \"\" for line in lines):\n continue\n\n if lines and not lines[0]:\n lines = lines[1:]\n source = [line + \"\\n\" for line in lines]\n # Drop last newline char\n if source and not source[-1].strip():\n source = source[:-1]\n if tag == \"shell\":\n source = [\"!\" + line for line in source]\n cell_type = \"code\"\n if tag != \"invisible\" and source:\n cell = {\"cell_type\": cell_type, \"source\": source}\n if cell_type == \"code\":\n cell[\"outputs\"] = []\n cell[\"metadata\"] = {\"colab_type\": \"code\"}\n cell[\"execution_count\"] = 0\n loc += _count_locs(source)\n else:\n cell[\"metadata\"] = {\"colab_type\": \"text\"}\n cells.append(cell)\n notebook = {}\n for key in NB_BASE.keys():\n notebook[key] = NB_BASE[key]\n notebook[\"metadata\"][\"colab\"][\"name\"] = str(py_path).split(\"/\")[-1][:-3]\n notebook[\"cells\"] = cells\n if loc > MAX_LOC:\n raise ValueError(\n \"Found %d lines of code, but expected fewer than %d\"\n % (loc, MAX_LOC)\n )\n\n f = open(nb_path, \"w\")\n f.write(json.dumps(notebook, indent=1, sort_keys=True))\n f.close()\n if fill_outputs:\n print(\"Generating ipynb\")\n parent_dir = Path(nb_path).parent\n current_files = os.listdir(parent_dir)\n try:\n os.system(\n \"jupyter nbconvert --to notebook --execute --debug \"\n + str(nb_path)\n + \" --inplace\"\n + \" --ExecutePreprocessor.timeout=\"\n + str(TIMEOUT)\n )\n finally:\n new_files = os.listdir(parent_dir)\n for fname in new_files:\n if fname not in current_files:\n fpath = parent_dir / fname\n if os.path.isdir(fpath):\n print(\"Removing created folder:\", fname)\n shutil.rmtree(fpath)\n else:\n print(\"Removing created file:\", fname)\n os.remove(fpath)\n\n\ndef nb_to_md(nb_path, md_path, img_dir, working_dir=None):\n img_exts = (\"png\", \"jpg\", \"jpeg\")\n # Assumes an already populated notebook.\n assert str(md_path).endswith(\".md\")\n current_dir = os.getcwd()\n original_img_dir = str(img_dir)\n if original_img_dir.endswith(\"/\"):\n original_img_dir = original_img_dir[:-1]\n img_dir = os.path.abspath(img_dir)\n nb_path = os.path.abspath(nb_path)\n nb_fname = str(nb_path).split(\"/\")[-1]\n\n del_working_dir = False\n if working_dir is None:\n del_working_dir = True\n working_dir = \"tmp_\" + str(random.randint(1e6, 1e7))\n if not os.path.exists(working_dir):\n os.makedirs(working_dir)\n print(\"Using working_dir:\", working_dir)\n\n os.chdir(working_dir)\n shutil.copyfile(nb_path, nb_fname)\n\n md_name = str(md_path).split(\"/\")[-1][:-3]\n target_md = md_name + \".md\"\n img_dir = Path(img_dir) / md_name\n if not os.path.exists(img_dir):\n os.makedirs(img_dir)\n\n os.system(\n # \"jupyter nbconvert --to markdown --execute --debug \"\n \"jupyter nbconvert --to markdown \"\n + nb_fname\n + \" --output \"\n + target_md\n # + \" --ExecutePreprocessor.timeout=\"\n # + str(TIMEOUT)\n )\n tmp_img_dir = md_name + \"_files\"\n if os.path.exists(tmp_img_dir):\n for fname in os.listdir(tmp_img_dir):\n if fname.endswith(img_exts):\n src = Path(tmp_img_dir) / fname\n target = Path(img_dir) / fname\n print(\"copy\", src, \"to\", target)\n shutil.copyfile(src, target)\n os.chdir(current_dir)\n md_content = open(Path(working_dir) / (md_name + \".md\")).read()\n for ext in img_exts:\n md_content = md_content.replace(\n \"![\" + ext + \"](\" + md_name + \"_files\",\n \"![\" + ext + \"](\" + original_img_dir + \"/\" + md_name,\n )\n md_content = _make_output_code_blocks(md_content)\n open(md_path, \"w\").write(md_content)\n if del_working_dir:\n shutil.rmtree(working_dir)\n\n\ndef py_to_md(py_path, nb_path, md_path, img_dir, working_dir=None):\n py_to_nb(py_path, nb_path, fill_outputs=False)\n nb_to_md(nb_path, md_path, img_dir, working_dir=working_dir)\n\n\ndef validate(py):\n \"\"\"Validate the format of a tutobook script.\n\n Specifically:\n - validate headers\n - validate style with black\n \"\"\"\n lines = py.split(\"\\n\")\n if not lines[0].startswith('\"\"\"'):\n raise ValueError('Missing `\"\"\"`-fenced header at top of script.')\n if not lines[1].startswith(\"Title: \"):\n raise ValueError(\"Missing `Title:` field.\")\n if not lines[2].startswith(\"Author: \") and not lines[2].startswith(\n \"Authors: \"\n ):\n raise ValueError(\"Missing `Author:` field.\")\n if not lines[3].startswith(\"Date created: \"):\n raise ValueError(\"Missing `Date created:` field.\")\n if not lines[4].startswith(\"Last modified: \"):\n raise ValueError(\"Missing `Last modified:` field.\")\n if not lines[5].startswith(\"Description: \"):\n raise ValueError(\"Missing `Description:` field.\")\n description = lines[5][len(\"Description: \") :]\n if not description:\n raise ValueError(\"Missing `Description:` field content.\")\n if not description[0] == description[0].upper():\n raise ValueError(\"Description field content must be capitalized.\")\n if not description[-1] == \".\":\n raise ValueError(\"Description field content must end with a period.\")\n if len(description) > 100:\n raise ValueError(\n \"Description field content must be less than 100 chars.\"\n )\n for i, line in enumerate(lines):\n if line.startswith('\"\"\"') and line.endswith('\"\"\"') and len(line) > 3:\n raise ValueError(\n 'Do not use single line `\"\"\"`-fenced comments. '\n \"Encountered at line %d\" % (i,)\n )\n for i, line in enumerate(lines):\n if line.endswith(\" \"):\n raise ValueError(\n \"Found trailing space on line %d; line: `%s`\" % (i, line)\n )\n # Validate style with black\n fpath = \"/tmp/\" + str(random.randint(1e6, 1e7)) + \".py\"\n f = open(fpath, \"w\")\n pre_formatting = \"\\n\".join(lines)\n f.write(pre_formatting)\n f.close()\n os.system(\"black \" + fpath)\n f = open(fpath)\n formatted = f.read()\n f.close()\n os.remove(fpath)\n if formatted != pre_formatting:\n raise ValueError(\n \"You python file did not follow `black` conventions. \"\n \"Run `black your_file.py` to autoformat it.\"\n )\n\n\ndef _count_locs(lines):\n loc = 0\n string_open = False\n for line in lines:\n line = line.strip()\n if not line or line.startswith(\"#\"):\n continue\n if not string_open:\n if not line.startswith('\"\"\"'):\n loc += 1\n else:\n if not line.endswith('\"\"\"'):\n string_open = True\n else:\n if line.startswith('\"\"\"'):\n string_open = False\n return loc\n\n\ndef _shorten_lines(py):\n max_len = 90\n lines = []\n for line in py.split(\"\\n\"):\n if len(line) <= max_len:\n lines.append(line)\n continue\n i = 0\n while len(line) > max_len:\n line = line.lstrip()\n if \" \" not in line[1:]:\n lines.append(line)\n break\n else:\n short_line = line[:max_len]\n line = line[max_len:]\n if \" \" in short_line:\n reversed_short_line = short_line[::-1]\n index = reversed_short_line.find(\" \") + 1\n line = short_line[-index:] + line\n short_line = short_line[:-index]\n\n lines.append(short_line.lstrip())\n i += 1\n if i > 10:\n raise\n lines.append(line.lstrip())\n return \"\\n\".join(lines)\n\n\ndef _get_next_script_element(py):\n lines = py.split(\"\\n\")\n assert lines\n elines = []\n i = 0\n tag = None\n if lines[0].startswith('\"\"\"'):\n assert len(lines) >= 2\n etype = \"markdown\"\n if len(lines[0]) > 3:\n tag = lines[0][3:]\n if tag not in [\"shell\", \"invisible\"]:\n raise ValueError(\"Found unknown cell tag:\", tag)\n lines = lines[1:]\n else:\n etype = \"code\"\n\n for i, line in enumerate(lines):\n if line.startswith('\"\"\"'):\n break\n else:\n elines.append(line)\n\n if etype == \"markdown\":\n py = \"\\n\".join(lines[i + 1 :])\n else:\n py = \"\\n\".join(lines[i:])\n e = \"\\n\".join(elines)\n\n return e, etype, py, tag\n\n\ndef _parse_header(header):\n lines = header.split(\"\\n\")\n title = lines[0][len(\"Title: \") :]\n author_line = lines[1]\n if author_line.startswith(\"Authors\"):\n author = author_line[len(\"Authors: \") :]\n auth_field = \"Authors\"\n else:\n author = author_line[len(\"Author: \") :]\n auth_field = \"Author\"\n date_created = lines[2][len(\"Date created: \") :]\n last_modified = lines[3][len(\"Last modified: \") :]\n description = lines[4][len(\"Description: \") :]\n return {\n \"title\": title,\n \"author\": author,\n \"auth_field\": auth_field,\n \"date_created\": date_created,\n \"last_modified\": last_modified,\n \"description\": description,\n }\n\n\ndef _make_output_code_blocks(md):\n lines = md.split(\"\\n\")\n output_lines = []\n final_lines = []\n is_inside_backticks = False\n\n def is_output_line(line, prev_line, output_lines):\n if line.startswith(\" \") and len(line) >= 5:\n if output_lines or (lines[i - 1].strip() == \"\" and line.strip()):\n return True\n return False\n\n def flush(output_lines, final_lines):\n final_lines.append('
')\n final_lines.append(\"```\")\n if len(output_lines) == 1:\n line = output_lines[0]\n final_lines.append(line[4:])\n else:\n for line in output_lines:\n final_lines.append(line[4:])\n final_lines.append(\"```\")\n final_lines.append(\"
\")\n\n for i, line in enumerate(lines):\n if line.startswith(\"```\"):\n is_inside_backticks = not is_inside_backticks\n final_lines.append(line)\n continue\n\n if is_inside_backticks:\n final_lines.append(line)\n continue\n\n if i > 0 and is_output_line(line, lines[-1], output_lines):\n output_lines.append(line)\n elif not line:\n if output_lines:\n if output_lines[-1]:\n output_lines.append(line)\n else:\n final_lines.append(line)\n else:\n if output_lines:\n flush(output_lines, final_lines)\n output_lines = []\n final_lines.append(line)\n if output_lines:\n flush(output_lines, final_lines)\n return \"\\n\".join(final_lines)\n\n\nNB_BASE = {\n \"metadata\": {\n \"colab\": {\n \"collapsed_sections\": [],\n \"name\": \"\", # FILL ME\n \"private_outputs\": False,\n \"provenance\": [],\n \"toc_visible\": True,\n },\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\",\n },\n \"language_info\": {\n \"codemirror_mode\": {\"name\": \"ipython\", \"version\": 3},\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.7.0\",\n },\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 0,\n}\n\n\nif __name__ == \"__main__\":\n cmd = sys.argv[1]\n if cmd not in {\"nb2py\", \"py2nb\"}:\n raise ValueError(\n \"Specify a command: either \"\n \"`nb2py source_filename.ipynb target_filename.py` or \"\n \"`py2nb source_filename.py target_file name.ipynb\"\n )\n if len(sys.argv) < 4:\n raise ValueError(\"Specify a source filename and a target filename\")\n source = sys.argv[2]\n target = sys.argv[3]\n\n if cmd == \"py2nb\":\n if not source.endswith(\".py\"):\n raise ValueError(\n \"The source filename should be a Python file. Got:\", source\n )\n if not target.endswith(\".ipynb\"):\n raise ValueError(\n \"The target filename should be a notebook file. Got:\", target\n )\n py_to_nb(source, target)\n if cmd == \"nb2py\":\n if not source.endswith(\".ipynb\"):\n raise ValueError(\n \"The source filename should be a notebook file. Got:\", source\n )\n if not target.endswith(\".py\"):\n raise ValueError(\n \"The target filename should be a Python file. Got:\", target\n )\n nb_to_py(source, target)\n" }, { "path": "examples/automodel_with_cnn.py", "content": "# Library import\nimport keras\nimport numpy as np\n\nimport autokeras as ak\n\n# Prepare example Data - Shape 1D\nnum_instances = 100\nnum_features = 5\nx_train = np.random.rand(num_instances, num_features).astype(np.float32)\ny_train = np.zeros(num_instances).astype(np.float32)\ny_train[0 : int(num_instances / 2)] = 1\nx_test = np.random.rand(num_instances, num_features).astype(np.float32)\ny_test = np.zeros(num_instances).astype(np.float32)\ny_train[0 : int(num_instances / 2)] = 1\n\nx_train = np.expand_dims(\n x_train, axis=2\n) # This step it's very important an CNN will only accept this data shape\nprint(x_train.shape)\nprint(y_train.shape)\n\n\n# Prepare Automodel for search\ninput_node = ak.Input()\noutput_node = ak.ConvBlock()(input_node)\n# output_node = ak.DenseBlock()(output_node) #optional\n# output_node = ak.SpatialReduction()(output_node) #optional\noutput_node = ak.ClassificationHead(num_classes=2, multi_label=True)(\n output_node\n)\n\nauto_model = ak.AutoModel(\n inputs=input_node, outputs=output_node, overwrite=True, max_trials=1\n)\n\n\n# Search\nauto_model.fit(x_train, y_train, epochs=1)\nprint(auto_model.evaluate(x_test, y_test))\n\n\n# Export as a Keras Model\nmodel = auto_model.export_model()\nprint(type(model.summary()))\n\n# print model as image\nkeras.utils.plot_model(\n model, show_shapes=True, expand_treeed=True, to_file=\"name.png\"\n)\n" }, { "path": "examples/celeb_age.py", "content": "\"\"\"\nRegression tasks estimate a numeric variable, such as the price of a house or\nvoter turnout.\n\nThis example is adapted from a\n[notebook](https://gist.github.com/mapmeld/98d1e9839f2d1f9c4ee197953661ed07)\nwhich estimates a person's age from their image, trained on the\n[IMDB-WIKI](https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/) photographs\nof famous\npeople.\n\nFirst, prepare your image data in a numpy.ndarray.\nEach image must have the same shape, meaning each has the same width, height,\nand color channels as other images in the set.\n\"\"\"\n\nfrom datetime import datetime\nfrom datetime import timedelta\n\nimport numpy as np\nimport pandas as pd\nfrom google.colab import drive\nfrom PIL import Image\nfrom scipy.io import loadmat\n\nimport autokeras as ak\n\n\"\"\"\n### Connect your Google Drive for Data\n\"\"\"\n\n\ndrive.mount(\"/content/drive\")\n\n\"\"\"\n### Install AutoKeras\n\nDownload the master branch to your Google Drive for this tutorial. In general,\nyou can use *pip install autokeras* .\n\"\"\"\n\n\"\"\"shell\n!pip install -v \"/content/drive/My Drive/AutoKeras-dev/autokeras-master.zip\"\n!pip uninstall keras-tuner\n!pip install\ngit+git://github.com/keras-team/keras-tuner.git@d2d69cba21a0b482a85ce2a38893e2322e139c01\n\"\"\"\n\n\"\"\"shell\n!pip install torch\n\"\"\"\n\n\"\"\"\n###**Import IMDB Celeb images and metadata**\n\"\"\"\n\n\"\"\"shell\n!mkdir \"./drive/My Drive/mlin/celebs\"\n\"\"\"\n\n\"\"\"shell\n! wget -O \"./drive/My Drive/mlin/celebs/imdb_0.tar\"\nhttps://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/static/imdb_0.tar\n\"\"\"\n\n\"\"\"shell\n! cd \"./drive/My Drive/mlin/celebs\" && tar -xf imdb_0.tar\n! rm \"./drive/My Drive/mlin/celebs/imdb_0.tar\"\n\"\"\"\n\n\"\"\"\nUncomment and run the below cell if you need to re-run the cells again and\nabove don't need to install everything from the beginning.\n\"\"\"\n\n# ! cd ./drive/My\\ Drive/mlin/celebs.\n\n\"\"\"shell\n! ls \"./drive/My Drive/mlin/celebs/imdb/\"\n\"\"\"\n\n\"\"\"shell\n! wget https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/static/imdb_meta.tar\n! tar -xf imdb_meta.tar\n! rm imdb_meta.tar\n\"\"\"\n\n\"\"\"\n###**Converting from MATLAB date to actual Date-of-Birth**\n\"\"\"\n\n\ndef datenum_to_datetime(datenum):\n \"\"\"\n Convert Matlab datenum into Python datetime.\n \"\"\"\n days = datenum % 1\n hours = days % 1 * 24\n minutes = hours % 1 * 60\n seconds = minutes % 1 * 60\n try:\n return (\n datetime.fromordinal(int(datenum))\n + timedelta(days=int(days))\n + timedelta(hours=int(hours))\n + timedelta(minutes=int(minutes))\n + timedelta(seconds=round(seconds))\n - timedelta(days=366)\n )\n except Exception:\n return datenum_to_datetime(700000)\n\n\nprint(datenum_to_datetime(734963))\n\n\"\"\"\n### **Opening MatLab file to Pandas DataFrame**\n\"\"\"\n\n\nx = loadmat(\"imdb/imdb.mat\")\n\n\nmdata = x[\"imdb\"] # variable in mat file\nmdtype = mdata.dtype # dtypes of structures are \"unsized objects\"\nndata = {n: mdata[n][0, 0] for n in mdtype.names}\ncolumns = [n for n, v in ndata.items()]\n\nrows = []\nfor col in range(0, 10):\n values = list(ndata.items())[col]\n for num, val in enumerate(values[1][0], start=0):\n if col == 0:\n rows.append([])\n if num > 0:\n if columns[col] == \"dob\":\n rows[num].append(datenum_to_datetime(int(val)))\n elif columns[col] == \"photo_taken\":\n rows[num].append(datetime(year=int(val), month=6, day=30))\n else:\n rows[num].append(val)\n\ndt = map(lambda row: np.array(row), np.array(rows[1:]))\n\ndf = pd.DataFrame(data=dt, index=range(0, len(rows) - 1), columns=columns)\nprint(df.head())\n\nprint(columns)\nprint(df[\"full_path\"])\n\n\"\"\"\n### **Calculating age at time photo was taken**\n\"\"\"\n\ndf[\"age\"] = (df[\"photo_taken\"] - df[\"dob\"]).astype(\"int\") / 31558102e9\nprint(df[\"age\"])\n\n\"\"\"\n### **Creating dataset**\n\n\n* We sample 200 of the images which were included in this first download.\n* Images are resized to 128x128 to standardize shape and conserve memory\n* RGB images are converted to grayscale to standardize shape\n* Ages are converted to ints\n\n\n\"\"\"\n\n\ndef df2numpy(train_set):\n images = []\n for img_path in train_set[\"full_path\"]:\n img = (\n Image.open(\"./drive/My Drive/mlin/celebs/imdb/\" + img_path[0])\n .resize((128, 128))\n .convert(\"L\")\n )\n images.append(np.asarray(img, dtype=\"int32\"))\n\n image_inputs = np.array(images)\n\n ages = train_set[\"age\"].astype(\"int\").to_numpy()\n return image_inputs, ages\n\n\ntrain_set = df[df[\"full_path\"] < \"02\"].sample(200)\ntrain_imgs, train_ages = df2numpy(train_set)\n\ntest_set = df[df[\"full_path\"] < \"02\"].sample(100)\ntest_imgs, test_ages = df2numpy(test_set)\n\n\"\"\"\n### **Training using AutoKeras**\n\"\"\"\n\n\n# Initialize the image regressor\nreg = ak.ImageRegressor(max_trials=15) # AutoKeras tries 15 different models.\n\n# Find the best model for the given training data\nreg.fit(train_imgs, train_ages)\n\n# Predict with the chosen model:\n# predict_y = reg.predict(test_images) # Uncomment if required\n\n# Evaluate the chosen model with testing data\nprint(reg.evaluate(train_imgs, train_ages))\n\n\"\"\"\n### **Validation Data**\n\nBy default, AutoKeras use the last 20% of training data as validation data. As\nshown in the example below, you can use validation_split to specify the\npercentage.\n\"\"\"\n\nreg.fit(\n train_imgs,\n train_ages,\n # Split the training data and use the last 15% as validation data.\n validation_split=0.15,\n epochs=3,\n)\n\n\"\"\"\nYou can also use your own validation set instead of splitting it from the\ntraining data with validation_data.\n\"\"\"\n\nsplit = 460000\nx_val = train_imgs[split:]\ny_val = train_ages[split:]\nx_train = train_imgs[:split]\ny_train = train_ages[:split]\nreg.fit(\n x_train,\n y_train,\n # Use your own validation set.\n validation_data=(x_val, y_val),\n epochs=3,\n)\n\n\"\"\"\n### **Customized Search Space**\n\nFor advanced users, you may customize your search space by using AutoModel\ninstead of ImageRegressor. You can configure the ImageBlock for some high-level\nconfigurations, e.g., block_type for the type of neural network to search,\nnormalize for whether to do data normalization, augment for whether to do data\naugmentation. You can also choose not to specify these arguments, which would\nleave the different choices to be tuned automatically. See the following\nexample for detail.\n\"\"\"\n\n\ninput_node = ak.ImageInput()\noutput_node = ak.ImageBlock(\n # Only search ResNet architectures.\n block_type=\"resnet\",\n # Normalize the dataset.\n normalize=True,\n # Do not do data augmentation.\n augment=False,\n)(input_node)\noutput_node = ak.RegressionHead()(output_node)\nreg = ak.AutoModel(inputs=input_node, outputs=output_node, max_trials=10)\nreg.fit(x_train, y_train, epochs=3)\n\n\"\"\"\nThe usage of AutoModel is similar to the functional API of Keras. Basically, you\nare building a graph, whose edges are blocks and the nodes are intermediate\noutputs of blocks. To add an edge from input_node to output_node with\noutput_node = ak.some_block(input_node). You can even also use more fine\ngrained blocks to customize the search space even further. See the following\nexample.\n\"\"\"\n\n\ninput_node = ak.ImageInput()\noutput_node = ak.Normalization()(input_node)\noutput_node = ak.ImageAugmentation(translation_factor=0.3)(output_node)\noutput_node = ak.ResNetBlock(version=\"v2\")(output_node)\noutput_node = ak.RegressionHead()(output_node)\nclf = ak.AutoModel(inputs=input_node, outputs=output_node, max_trials=10)\nclf.fit(x_train, y_train, epochs=3)\n\n\"\"\"\n## References\n\n[Main Reference\nNotebook](https://gist.github.com/mapmeld/98d1e9839f2d1f9c4ee197953661ed07),\n[Dataset](https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/),\n[ImageRegressor](/image_regressor),\n[ResNetBlock](/block/#resnetblock-class),\n[ImageInput](/node/#imageinput-class),\n[AutoModel](/auto_model/#automodel-class),\n[ImageBlock](/block/#imageblock-class),\n[Normalization](/preprocessor/#normalization-class),\n[ImageAugmentation](/preprocessor/#image-augmentation-class),\n[RegressionHead](/head/#regressionhead-class).\n\n\"\"\"\n" }, { "path": "examples/cifar10.py", "content": "from keras.datasets import cifar10\n\nimport autokeras as ak\n\n# Prepare the dataset.\n(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n\n# Initialize the ImageClassifier.\nclf = ak.ImageClassifier(max_trials=3)\n# Search for the best model.\nclf.fit(x_train, y_train, epochs=5)\n# Evaluate on the testing data.\nprint(\"Accuracy: {accuracy}\".format(accuracy=clf.evaluate(x_test, y_test)[1]))\n" }, { "path": "examples/imdb.py", "content": "\"\"\"\nSearch for a good model for the\n[IMDB](\nhttps://keras.io/datasets/#imdb-movie-reviews-sentiment-classification) dataset.\n\"\"\"\n\nimport keras\nimport numpy as np\n\nimport autokeras as ak\n\n\ndef imdb_raw():\n max_features = 20000\n index_offset = 3 # word index offset\n\n (x_train, y_train), (x_test, y_test) = keras.datasets.imdb.load_data(\n num_words=max_features, index_from=index_offset\n )\n x_train = x_train\n y_train = y_train.reshape(-1, 1)\n x_test = x_test\n y_test = y_test.reshape(-1, 1)\n\n word_to_id = keras.datasets.imdb.get_word_index()\n word_to_id = {k: (v + index_offset) for k, v in word_to_id.items()}\n word_to_id[\"\"] = 0\n word_to_id[\"\"] = 1\n word_to_id[\"\"] = 2\n\n id_to_word = {value: key for key, value in word_to_id.items()}\n x_train = list(\n map(lambda sentence: \" \".join(id_to_word[i] for i in sentence), x_train)\n )\n x_test = list(\n map(lambda sentence: \" \".join(id_to_word[i] for i in sentence), x_test)\n )\n x_train = np.array(x_train, dtype=np.str)\n x_test = np.array(x_test, dtype=np.str)\n return (x_train, y_train), (x_test, y_test)\n\n\n# Prepare the data.\n(x_train, y_train), (x_test, y_test) = imdb_raw()\nprint(x_train.shape) # (25000,)\nprint(y_train.shape) # (25000, 1)\nprint(x_train[0][:50]) # this film was just brilliant casting \n\n# Initialize the TextClassifier\nclf = ak.TextClassifier(max_trials=3)\n# Search for the best model.\nclf.fit(x_train, y_train, epochs=2, batch_size=8)\n# Evaluate on the testing data.\nprint(\"Accuracy: {accuracy}\".format(accuracy=clf.evaluate(x_test, y_test)))\n" }, { "path": "examples/mnist.py", "content": "\"\"\"\nSearch for a good model for the\n[MNIST](https://keras.io/datasets/#mnist-database-of-handwritten-digits)\ndataset.\n\"\"\"\n\nfrom keras.datasets import mnist\n\nimport autokeras as ak\n\n# Prepare the dataset.\n(x_train, y_train), (x_test, y_test) = mnist.load_data()\nprint(x_train.shape) # (60000, 28, 28)\nprint(y_train.shape) # (60000,)\nprint(y_train[:3]) # array([7, 2, 1], dtype=uint8)\n\n# Initialize the ImageClassifier.\nclf = ak.ImageClassifier(max_trials=3)\n# Search for the best model.\nclf.fit(x_train, y_train, epochs=10)\n# Evaluate on the testing data.\nprint(\"Accuracy: {accuracy}\".format(accuracy=clf.evaluate(x_test, y_test)))\n" }, { "path": "examples/new_pop.py", "content": "\"\"\"shell\npip install autokeras\n\"\"\"\n\nimport pandas as pd\n\nimport autokeras as ak\n\n\"\"\"\n## Social Media Articles Example\n\nRegression tasks estimate a numeric variable, such as the price of a house\nor a person's age.\n\nThis example estimates the view counts for an article on social media platforms,\ntrained on a\n[News Popularity](\nhttps://archive.ics.uci.edu/ml/datasets/\nNews+Popularity+in+Multiple+Social+Media+Platforms)\ndataset collected from 2015-2016.\n\nFirst, prepare your text data in a `numpy.ndarray` format.\n\"\"\"\n\n\n# converting from other formats (such as pandas) to numpy\ndf = pd.read_csv(\"./News_Final.csv\")\n\ntext_inputs = df.Title.to_numpy(dtype=\"str\")\nmedia_success_outputs = df.Facebook.to_numpy(dtype=\"int\")\n\n\"\"\"\nNext, initialize and train the [TextRegressor](/text_regressor).\n\"\"\"\n\n\n# Initialize the text regressor\nreg = ak.TextRegressor(max_trials=15) # AutoKeras tries 15 different models.\n\n# Find the best model for the given training data\nreg.fit(text_inputs, media_success_outputs)\n\n# Predict with the chosen model:\npredict_y = reg.predict(text_inputs)\n" }, { "path": "examples/reuters.py", "content": "\"\"\"shell\n!pip install -q -U pip\n!pip install -q -U autokeras==1.0.8\n!pip install -q git+https://github.com/keras-team/keras-tuner.git@1.0.2rc1\n\"\"\"\n\nimport keras\nimport numpy as np\nfrom keras.datasets import reuters\n\nimport autokeras as ak\n\n\"\"\"\nSearch for a good model for the\n[Reuters](https://keras.io/ja/datasets/#_5) dataset.\n\"\"\"\n\n\n# Prepare the dataset.\ndef reuters_raw(max_features=20000):\n index_offset = 3 # word index offset\n\n (x_train, y_train), (x_test, y_test) = reuters.load_data(\n num_words=max_features, index_from=index_offset\n )\n x_train = x_train\n y_train = y_train.reshape(-1, 1)\n x_test = x_test\n y_test = y_test.reshape(-1, 1)\n\n word_to_id = reuters.get_word_index()\n word_to_id = {k: (v + index_offset) for k, v in word_to_id.items()}\n word_to_id[\"\"] = 0\n word_to_id[\"\"] = 1\n word_to_id[\"\"] = 2\n\n id_to_word = {value: key for key, value in word_to_id.items()}\n x_train = list(\n map(lambda sentence: \" \".join(id_to_word[i] for i in sentence), x_train)\n )\n x_test = list(\n map(lambda sentence: \" \".join(id_to_word[i] for i in sentence), x_test)\n )\n x_train = np.array(x_train, dtype=np.str)\n x_test = np.array(x_test, dtype=np.str)\n return (x_train, y_train), (x_test, y_test)\n\n\n# Prepare the data.\n(x_train, y_train), (x_test, y_test) = reuters_raw()\nprint(x_train.shape) # (8982,)\nprint(y_train.shape) # (8982, 1)\nprint(x_train[0][:50]) # said as a result of its decemb\n\n# Initialize the TextClassifier\nclf = ak.TextClassifier(\n max_trials=5,\n overwrite=True,\n)\n\n# Callback to avoid overfitting with the EarlyStopping.\ncbs = [\n keras.callbacks.EarlyStopping(patience=3),\n]\n\n# Search for the best model.\nclf.fit(x_train, y_train, epochs=10, callback=cbs)\n\n# Evaluate on the testing data.\nprint(\"Accuracy: {accuracy}\".format(accuracy=clf.evaluate(x_test, y_test)))\n" }, { "path": "pyproject.toml", "content": "[build-system]\nrequires = [\"setuptools>=61.0\"]\nbuild-backend = \"setuptools.build_meta\"\n\n[project]\nname = \"autokeras\"\nauthors = [\n {name = \"Keras team\", email = \"keras-users@googlegroups.com\"},\n]\ndescription = \"AutoML for deep learning\"\nreadme = \"README.md\"\nrequires-python = \">=3.8\"\nlicense = {text = \"Apache License 2.0\"}\ndynamic = [\"version\"]\nclassifiers = [\n \"Intended Audience :: Developers\",\n \"Intended Audience :: Education\",\n \"Intended Audience :: Science/Research\",\n \"License :: OSI Approved :: Apache Software License\",\n \"Programming Language :: Python :: 3.8\",\n \"Programming Language :: Python :: 3.9\",\n \"Programming Language :: Python :: 3.10\",\n \"Programming Language :: Python :: 3.11\",\n \"Topic :: Scientific/Engineering :: Mathematics\",\n \"Topic :: Software Development :: Libraries :: Python Modules\",\n \"Topic :: Software Development :: Libraries\",\n]\ndependencies = [\n \"packaging\",\n \"keras-tuner>=1.4.0\",\n \"keras>=3.0.0\",\n \"dm-tree\",\n]\n\n[project.optional-dependencies]\ntests = [\n \"pandas\",\n \"pytest>=4.4.0\",\n \"flake8\",\n \"black[jupyter]\",\n \"isort\",\n \"pytest-xdist\",\n \"pytest-cov\",\n \"coverage\",\n \"typedapi>=0.2,<0.3\",\n \"scikit-learn\",\n]\n\n[project.urls]\nHome = \"https://autokeras.com/\"\nRepository = \"https://github.com/keras-team/autokeras\"\n\n[tool.setuptools.dynamic]\nversion = {attr = \"autokeras.__init__.__version__\"}\n\n[tool.setuptools.packages.find]\ninclude = [\"autokeras\", \"autokeras.*\"]\n\n[tool.black]\nline-length = 80\ntarget-version = []\n\n[tool.isort]\nprofile = \"black\"\nknown_first_party = [\"autokeras\", \"tests\"]\ndefault_section = \"THIRDPARTY\"\nline_length = 80\nforce_single_line = \"True\"" }, { "path": "setup.cfg", "content": "[tool:pytest]\naddopts=-v\n -p no:warnings\n --durations=10\n --log-cli-level=CRITICAL\n\n# Do not run tests in the build folder\nnorecursedirs= build\n\n[coverage:report]\nexclude_lines =\n pragma: no cover\n @abstract\n raise NotImplementedError\nomit = \n *test*\n autokeras/prototype/*\n\n[flake8]\n\n# imported but unused in __init__.py, that's ok.\nper-file-ignores = **/__init__.py:F401\nignore = E203, W503\nmax-line-length = 80\n" }, { "path": "shell/contributors.py", "content": "import base64\nimport json\nimport os\nimport sys\nfrom io import BytesIO\n\nimport requests\nfrom PIL import Image\n\n\ndef main(directory):\n contributors = []\n for contributor in json.load(open(\"contributors.json\")):\n if contributor[\"type\"] != \"User\":\n continue\n if contributor[\"login\"] == \"codacy-badger\":\n continue\n contributors.append(contributor)\n\n size = 36\n gap = 3\n elem_per_line = 22\n width = elem_per_line * (size + gap) + gap\n height = ((len(contributors) - 1) // elem_per_line + 1) * (size + gap) + gap\n\n html = '\"\n defs += ''\n defs += ' '\n defs += \"\"\n\n html += defs + \"\\n\"\n\n for index, contributor in enumerate(contributors):\n file_name = os.path.join(directory, str(index) + \".jpeg\")\n response = requests.get(contributor[\"avatar_url\"])\n file = open(file_name, \"wb\")\n file.write(response.content)\n file.close()\n image = Image.open(file_name)\n image = image.resize((size, size))\n # image.convert('RGB').save(file_name)\n buffered = BytesIO()\n image.save(buffered, format=\"PNG\")\n img_str = base64.b64encode(buffered.getvalue()).decode(\"UTF-8\")\n\n xi = index % elem_per_line\n yi = index // elem_per_line\n x = xi * (size + gap) + gap\n y = yi * (size + gap) + gap\n temp = (\n ' '\n + ''\n )\n temp = temp.format(\n html_url=contributor[\"html_url\"],\n x=x,\n y=y,\n img_str=img_str,\n login=contributor[\"login\"],\n )\n html += temp + \"\\n\"\n\n html += \"\"\n print(html)\n\n\nif __name__ == \"__main__\":\n main(sys.argv[1])\n" }, { "path": "shell/contributors.sh", "content": "# node shell/generate_json.js\ngh api -H \"Accept: application/vnd.github+json\" /repos/keras-team/autokeras/contributors --paginate > response.json\nsed \"s/\\]\\[/,/g\" response.json > contributors.json\nrm response.json\nmkdir avatars\npython shell/contributors.py avatars > docs/templates/img/contributors.svg\nrm contributors.json\nrm -rf avatars\n" }, { "path": "shell/copyright.txt", "content": "# Copyright 2020 The AutoKeras Authors.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n" }, { "path": "shell/cov.sh", "content": "pytest tests --cov-report html --cov-report xml:cov.xml --cov=autokeras\n" }, { "path": "shell/coverage.sh", "content": "pytest --cov-report xml:cov.xml --cov autokeras $1" }, { "path": "shell/docs.sh", "content": "#!/usr/bin/env bash\ncd docs\npython autogen.py\nmkdocs build\ncd ..\nsh shell/format.sh\necho \"autokeras.com\" > docs/site/CNAME\ngit checkout -b gh-pages-temp\ngit add -f docs/site\ngit commit -m \"gh-pages update\"\ngit subtree split --prefix docs/site -b gh-pages\ngit push -f origin gh-pages:gh-pages\ngit branch -D gh-pages\ngit checkout master\ngit branch -D gh-pages-temp\n" }, { "path": "shell/format.sh", "content": "isort .\nblack .\n\nfor i in $(find autokeras benchmark -name '*.py')\ndo\n if ! grep -q Copyright $i\n then\n echo $i\n cat shell/copyright.txt $i >$i.new && mv $i.new $i\n fi\ndone\n\nflake8 .\n" }, { "path": "shell/generate_json.js", "content": "const { Octokit } = require(\"@octokit/rest\");\nconst fs = require('fs')\nconst octokit = new Octokit({});\n\noctokit.paginate(octokit.repos.listContributors,{\n owner: 'keras-team',\n repo: 'autokeras',\n}).then((contributors) => {\n fs.writeFileSync('contributors.json', JSON.stringify(contributors))\n});\n" }, { "path": "shell/lint.sh", "content": "isort -c .\nif ! [ $? -eq 0 ]\nthen\n echo \"Please run \\\"sh shell/format.sh\\\" to format the code.\"\n exit 1\nfi\nflake8 .\nif ! [ $? -eq 0 ]\nthen\n echo \"Please fix the code style issue.\"\n exit 1\nfi\nblack --check .\nif ! [ $? -eq 0 ]\nthen\n echo \"Please run \\\"sh shell/format.sh\\\" to format the code.\"\n exit 1\nfi\nfor i in $(find autokeras benchmark -name '*.py') # or whatever other pattern...\ndo\n if ! grep -q Copyright $i\n then\n echo \"Please run \\\"sh shell/format.sh\\\" to format the code.\"\n exit 1\n fi\ndone\n" }, { "path": "shell/perf.sh", "content": "pytest tests/performance.py | tee perf_output.txt\n" }, { "path": "shell/pre-commit.sh", "content": "#!/usr/bin/env bash\n\nset -e\n\nexport DOCKER_BUILDKIT=1\ndocker build -t autokeras_formatting -f docker/pre-commit.Dockerfile .\ndocker run --rm -t -v \"$(pwd -P):/autokeras\" autokeras_formatting\n" }, { "path": "shell/pypi.sh", "content": "#!/usr/bin/env bash\nrm dist/*\npython -m build\ntwine upload --repository-url https://upload.pypi.org/legacy/ dist/*\n" } ]