Repository: Kaggle/kaggle-cli Branch: main Commit: 121f04ad4192 Files: 45 Total size: 225.7 KB Directory structure: gitextract_ds3iv5y5/ ├── .gcloudignore ├── .github/ │ └── workflows/ │ └── no-response.yaml ├── .gitignore ├── CHANGELOG.md ├── CONTRIBUTING.md ├── Dockerfile ├── GEMINI.md ├── LICENSE.txt ├── README.md ├── SECURITY.md ├── docker-hatch ├── docs/ │ ├── Makefile │ ├── README.md │ ├── competitions.md │ ├── conf.py │ ├── configuration.md │ ├── datasets.md │ ├── datasets_metadata.md │ ├── index.rst │ ├── kernels.md │ ├── kernels_metadata.md │ ├── make.bat │ ├── model_variations.md │ ├── model_variations_versions.md │ ├── models.md │ ├── models_metadata.md │ └── tutorials.md ├── integration_tests/ │ ├── __init__.py │ └── test_models.py ├── pyproject.toml ├── tests/ │ ├── __init__.py │ ├── dataset/ │ │ └── data.csv │ ├── kernel/ │ │ └── testing-x.ipynb │ ├── model/ │ │ └── instance/ │ │ ├── data.csv │ │ └── version/ │ │ └── metadata.json │ ├── sample_submission.csv │ ├── test_commands.sh │ └── unit_tests.py └── tools/ ├── cicd/ │ └── integration-tests.yaml ├── releases/ │ ├── Dockerfile │ ├── cloudbuild.yaml │ ├── requirements.in │ └── requirements.txt ├── use-localhost.sh └── use-prod.sh ================================================ FILE CONTENTS ================================================ ================================================ FILE: .gcloudignore ================================================ python ================================================ FILE: .github/workflows/no-response.yaml ================================================ name: No Response # Both `issue_comment` and `scheduled` event types are required for this Action # to work properly. on: issue_comment: types: [created] schedule: # Schedule for five minutes after midnight, every day - cron: '5 0 * * *' # By specifying the access of one of the scopes, all of those that are not # specified are set to 'none'. permissions: issues: write jobs: noResponse: runs-on: ubuntu-latest steps: - uses: lee-dohm/no-response@9bb0a4b5e6a45046f00353d5de7d90fb8bd773bb with: token: ${{ github.token }} # Comment to post when closing an Issue for lack of response. Set to `false` to disable closeComment: > Without additional information we're not able to resolve this issue, so it will be closed at this time. You're still free to add more info and respond to any questions above, though. We'll reopen the case if you do. Thanks for your contribution! # Number of days of inactivity before an issue is closed for lack of response. daysUntilClose: 14 # Label requiring a response. responseRequiredLabel: "waiting for response" ================================================ FILE: .gitignore ================================================ # Byte-compiled / optimized / DLL files __pycache__/ *.py[cod] *$py.class # C extensions *.so # Distribution / packaging .Python env/ build/ develop-eggs/ dist/ downloads/ eggs/ .eggs/ kaggle kagglesdk !*/kaggle/ !*/kagglesdk/ lib/ lib64/ parts/ sdist/ var/ *.egg-info/ .installed.cfg *.egg # PyInstaller # Usually these files are written by a python script from a template # before PyInstaller builds the exe, so as to inject date/other infos into it. *.manifest *.spec # Installer logs pip-log.txt pip-delete-this-directory.txt # Unit test / coverage reports htmlcov/ .tox/ .coverage .coverage.* .cache nosetests.xml coverage.xml *,cover .hypothesis/ venv/ .python-version # Translations *.mo *.pot # Django stuff: *.log # Sphinx documentation docs/_build/ # PyBuilder target/ #Ipython Notebook .ipynb_checkpoints # Rider/IntelliJ .idea/ # Gemini .gemini/ # Claude Code .claude/ ================================================ FILE: CHANGELOG.md ================================================ Changelog ==== ### Next * Add `--sandbox` flag to `kaggle competitions submit` for sandbox submissions (competition hosts/admins only) ### 2.0.0 * General Availability release * Change more "instance" to "variation" * Update link for the integration test auth instructions (#926) * Fix string formatting in upgrade nudge message (#928) Thanks PythonicVarun! ### 1.8.4 * Rename `kaggle-api` to `kaggle-cli` * Allow auth to happen multiple times (#922) * Add --acc to set accelerator for: kaggle kernels push ... (#907) * Add automatic retry and resume to download_file (#905) Thanks katoue! * Restore model validation check (#902) * Add file pattern matching in output download (#901) Thanks piotr-ginal! ### 1.8.3 * Add packaging dep (#883) * Add version checking against server known-version (#880) * Fix edit error (#876) * Use kagglesdk from pypi (#875) * Fix Kaggle access token auth KeyError when KAGGLE_API_TOKEN is unset (#874) ### 1.8.2 * Changes to build script ### 1.8.1 * Fix memory exhaustion when downloading large files (#869) * Add python-dateutil to pyproject.toml dependencies (#866) ### 1.8.0 * Fix resumable download error (#865) * Fix dataset version spec (#862) * Add machine_shape to the metadata of kaggle kernels pull (#856) * Add pagination options to models * Add pagination options for submissions (#832) * Add pagination options to list commands (#815) * Add canonical aliases for push/pull (#787) * Add parquet as a filter option (#786) * Add variations as alt for instances (#784) * Enable (and rename) synonyms i and v (#782) ### 1.7.5.0 (not released) * Require Python 3.11. * Add KernelExecutionType (#775) * Output docker_image as part of the pull metadata (#773) * Allow user to specify docker_image during kernel push (#774) * Add kernel version type to save request (#771) * Add tests for delete and de-flake (#769) * Rename "yes" params and make confirmation consistent (#765) * Fix bug that caused double serialization (#764) * Add kaggle kernels delete (#762) * Add test for dataset_delete() and make script more robust (#760) * Check dataset status before uploading (#759) * Add kaggle datasets delete (#755) * Fix calls to download_file() (#752) * Add type annotations for mypy (#746) * Use Optional[...] in cases where the proto file does (#744) * Improve some type hints and fix a bug (#741) * Reformat everything with black (#737) * Add more type hints (#736) * Add type annotations to main file (#735) * Bulk reformat docstrings (#732) * Merge envars before sending a request (#729) * Use PROD if no environment is specified. (#726) * Add a no response action to auto-close issues (#723) ### 1.7.4.2 * Fix a problem in downloading kernel output files. ### 1.7.4.1 * Fix a dataset download problem. Datasets that had a license were failing to download. * Update the documentation to include code competition submit. ### 1.7.4 Version 1.7.3 was never released. There were errors in versioning on test.pypi.org. For consistency, we decided to jump several version numbers. This is the first release since 1.6.17. The actual changes are described in 1.7.3. ### 1.7.3 There was an error in versioning. We went from 1.6.17 to 1.7.3. * Added the ability to submit to a code competition. Some required arguments have been made optional. * Added a `--timeout` option to `kaggle kernels push` to limit the run-time to the specified number of seconds. * Removed Swagger. Projects that use `kaggle/api/kaggle_api.py` may be affected. That file is deprecated and will be removed. Most of its functions still work, but those that involve uploading files no longer work. The command-line tool uses a higher-level abstraction for uploading, and client code needs to be converted to use that. ### 1.7.3b2 * Added the ability to submit to a code competition. Some required arguments have been made optional. * Added a `--timeout` option to `kaggle kernels push` to limit the run-time to the specified number of seconds. ### 1.7.3b1 * Fix escaped-quote issue in HTTP requests. ### 1.7.3b0 * Remove Swagger. No user-visible changes to the command-line tool. However, projects that use `kaggle/api/kaggle_api.py` may be affected. That file is deprecated and will be removed. Most of its functions still work, but those that involve uploading files no longer work. The command-line tool uses a higher-level abstraction for uploading and client code needs to be converted to use that. ### 1.6.17 * No changes; release 1.6.16 did not complete. ### 1.6.16 * No changes; release 1.6.15 isn't usable. We're working on process updates to prevent this from happening again. ### 1.6.15 * Support XDG base directory specification on Linux * Disable out-of-date API version warning with -W * Allow an array of strings in "source" when uploading .ipynb files (thanks to GitHub user mgallifrey for the contribution!) * Add triton framework for models * Update model licenses ### 1.6.14 * No changes; release 1.6.13 isn't usable. ### 1.6.13 * Add --page-size and --page-token CLI options to all commands that display lists of files. ### 1.6.12 * Re-release 1.6.11 without the `src` directory included in the package. ### 1.6.11 * Allow unauthenticated usage of "datasets download", "datasets files". * This will only work after April 8th, 2024. More more details, see: * Allow "help" and "version" to be used for all commands, unauthenticated. * Fix: "dataset download -f" can accept a specific dataset version. ### 1.6.10 Repackage of 1.6.8 as a new release, to fix the problematic 1.6.9 release. ### 1.6.9 * Do not use. Problematic release that causes an error: `ModuleNotFoundError: No module named 'kaggle.api'` ### 1.6.8 * Add "gguf" ### 1.6.7 * Add "TensorRtLlm" model framework. ### 1.6.6 * Add "GemmaCpp" and "GGML" model frameworks. ### 1.6.5 * Add "MaxText" model framework. ### 1.6.4 * Add "Transformers" model framework. ### 1.6.3 Release date: 01/11/24 * Add "Flax" and "Pax" model frameworks. ### 1.6.2 Release date: 01/09/24 * Add "Other" model framework. ### 1.6.1 Release date: 01/08/24 * Fix dataset/model upload. ### 1.6.0 Release date: 01/04/24 * Release the pre-release branch with models endpoints. #### 1.6.0a7 Release date: 11/22/23 * Add model_instance_type and base_model_instance_id to ModelInstance #### 1.6.0a6 Release date: 9/19/23 * Include version_number and version_id in the model-instance-metadata.json file #### 1.6.0a5 Release date: 8/02/23 * Add Keras model framework. #### 1.5.16 Release date: 7/17/23 * Fix dataset download bug with locale * Resumable uploads * Retry some failed requests #### 1.6.0a4 Release date: 7/07/23 * Resumable uploads * Retry some failed requests * Flag `-y` to delete model/instance/version without confirmation #### 1.6.0a3 Release date: 7/06/23 * Confirmation for deleting a model, instance or version * Merge changes from 1.5.14 and 1.5.15 #### 1.5.15 Release date: 6/30/23 * Add missing licenses for datasets * Re-add option to pass dataset with `-d` * Download / list files for a specific version of a dataset * Documentation improvements #### 1.5.14 Release date: 6/29/23 * Show the full error message from the API * Improve and fix documentation * Fix kernel's data sources bug, and add the model data source to push/pull * Implement resumable downloads * Fix unreachable code bug * Make some arguments required * Add enable_tpu to kernel's push/pull #### 1.6.0a2 Release date: 6/12/23 * Add endpoint to get a modelInstance * Simplify the modelInstanceVersion creation * Fix Model files zipping #### 1.6.0a0 Release date: 6/07/23 * Add Models endpoints #### 1.5.13 Release date: 2/27/23 * Add ability to add a model to a kernel ### 1.5.12 Release date: 03/12/21 * No changes ### 1.5.11 Release date: 03/12/21 * Add support for non-ASCII characters for kernels. ### 1.5.10 Release date: 11/30/20 * Remove dependency on slugify. ### 1.5.9 Release date: 10/21/20 * Drop version restriction on urllib3 in setup.py. ### 1.5.8 Release date: 09/03/20 * No user-facing changes #### 1.5.7 Release date: 8/31/20 * Add ability to specify the kernel docker image pinning type * Kernels have internet enabled by default * Various competitions fixes #### 1.5.6 Release date: 9/19/19 * Downloading all files for a competition downloads a zip instead of individual files #### 1.5.5 Release date: 8/30/19 * Add vote count and usability rating to datasets listing * Add min and max dataset size filters to datasets listing * Add additional information to dataset metadata API * Allow updating dataset metdata #### 1.5.4 Release date: 5/28/19 * Make kernels init more friendly * Make directories if needed for kernels output #### 1.5.3 Release date: 2/20/19 * Bump urllib3 version #### 1.5.2 Release date: 1/28/19 * Don't error on encoding errors when printing tables * Exit with error code when an exception is caught #### 1.5.1.1 Release date: 12/5/18 * Add missing cli option for dataset subfolders #### 1.5.1 Release date: 12/5/18 * Allow custom ca_cert files * Support uplodaing datasets with subfolders * Fix kaggle.json permissions warning #### 1.5.0 Release date: 10/19/18 * Update API to work with new competitions submissions backend. This change will force old API clients to update. * Update error message when config file is not found. #### 1.4.7.1 Release date: 8/28/18 * Fix host #### 1.4.7 Release date: 8/28/18 * Make dataset version `-p` argument actually optinal * Don't require the `resources` field when updating a dataset * Don't automatically unzip datasets * Add an unzip option for dataset downloads * Add validation for kernel title and slug length * Give a warning if kernel title does not resolve to the specified slug * Show kernel version number after pushing * Respect `code_file` value in kernel metadata when pulling kernels #### 1.4.6 Release date: 8/7/18 * Allow setting config values through environmental variables #### 1.4.5 Release date: 8/1/18 * Add error if dataset metadata repeats files #### 1.4.4 Release date: 7/30/18 * Fix issue with reading kernel metadata #### 1.4.3 Release date: 7/30/18 * Add more competitions list options * Add more datasets list options * Add a couple more fields to kernels list display * Add support for kernel and dataset ID's * Allow generating metadata for an existing dataset * Fix issue with downloading from datasets whose titles don't match their slugs * Use kernel slug as filename for kernel output * Make upload and download directory default to current working directory * Use a default username on downloading kernel or dataset data if none is specified * Support extended data types on datasets * Stop requiring `-c`, `-d`, and `-k` arguments * Don't require `resources` field in dataset metadata #### 1.4.2 Release date: 7/20/18 * Validate dataset slug and title length before uploading * Fix issue with dataset metadata file detection * Cleaned up KeyboardInterrupt errors * Validate all specified files in a dataset exist prior to uploading * Make ApiExceptions (slightly) less ugly #### 1.4.1 Release date: 7/20/18 * Add python 3.7 compatibility #### 1.4.0 Release date: 7/19/18 * Add kernels support ** List and search kernels ** Push kernels code ** Pull kernels code ** Download kernel output ** Get latest kernel run status #### 1.3.12 Release date: 6/25/18 * Allow setting a `'KAGGLE_CONFIG_DIR'` environmental token * Return metadata file after creating * Alert users that dataset creation takes time #### 1.3.11.1 * Fix other invalid tags check #### 1.3.11 Release date: 6/12/18 * Improve version check * Fix invalid tags check #### 1.3.10 Release date: 6/10/18 * Restrict urllib3's version due to requests dependency problem #### 1.3.9.1 Release date: 6/9/18 * Fix bug with competitions submissions. #### 1.3.9 Release date: 6/8/18 * Improve error message for closed competitions * Remove stacktrace on errors * Print any invalid tags * Warn if there are no competition files to download * Implement resumable uploads * Add subtitle metadata to dataset uploads * Add progress bars for uploads and downloads * Add command for downloading competitions leaderboard * Add command for viewing the top of the leaderboard #### 1.3.8 Release date: 5/18/18 * Add option to delete all previous dataset versions #### 1.3.7 Release date: 5/18/18 * Add aliases for subcommands (ex. `kaggle c` is the same thing as `kaggle competitions`) * Add version command * Show full download path for files * Remove file size limitation from uploads #### 1.3.6 Release date: 5/7/18 * Give the option to add tags to datasets. * Known limitiation - you cannot delete tags through the API. Those changes must be done through the website. #### 1.3.5 Release date: 5/4/18 * Fix schema declaration in dataset resources #### 1.3.4 Release date: 4/30/18 * Rename `columns` to `fields` #### 1.3.3 Release date: 4/26/18 * Fix UnicodeEncodeError for certain datasets * Include Swagger yaml and config files #### 1.3.2.1 Release date: 4/24/18 * Fix bug with column metadata #### 1.3.2 Release date: 4/24/18 * Give the option to specify a schema for uploaded datasets * Give the option to set the dataset description during updates #### 1.3.1 Release date: 4/19/18 * Give the option to set dataset file descriptions * Give the option to not convert tabular datasets to csv #### 1.3.0 Release date: 4/18/18 * Give the option to set the dataset description during creation #### 1.2.1 Release date: 4/17/18 * [Issue #5](https://github.com/Kaggle/kaggle-api/issues/5) - Reformat code for consistency and to align with [Google's python coding style](https://google.github.io/styleguide/pyguide.html). Most of the changes are cosmetic, but most cases of `camelCasing` other than class names have been changed to `snake_case`. This is a breaking change for anyone directly using the python code rather than simply using the command line. ================================================ FILE: CONTRIBUTING.md ================================================ # How to Contribute We'd love to accept your patches and contributions to this project. There are just a few small guidelines you need to follow. ## Contributor License Agreement Contributions to this project must be accompanied by a Contributor License Agreement. You (or your employer) retain the copyright to your contribution; this simply gives us permission to use and redistribute your contributions as part of the project. Head over to to see your current agreements on file or to sign a new one. You generally only need to submit a CLA once, so if you've already submitted one (even if it was for a different project), you probably don't need to do it again. ## Code reviews All submissions, including submissions by project members, require review. We use GitHub pull requests for this purpose. Consult [GitHub Help](https://help.github.com/articles/about-pull-requests/) for more information on using pull requests. ## Community Guidelines This project follows [Google's Open Source Community Guidelines](https://opensource.google.com/conduct/). ================================================ FILE: Dockerfile ================================================ ARG PYTHON_VERSION FROM python:${PYTHON_VERSION} RUN pip install hatch==1.14.0 # Add only the minimal files required to be able to pre-create the hatch environments. # If any of these files changes, a new Docker build is necessary. This is why we need # to only include the minimal set of files. ADD pyproject.toml /working/pyproject.toml ADD LICENSE.txt /working/LICENSE.txt ADD README.md /working/README.md ADD src/kaggle/__init__.py /working/src/kaggle/__init__.py WORKDIR /working # Pre-create the hatch environments. # This drastically cut the time to run commands with the `docker-hatch` wrapper # since the creation of the environments (including syncing dependencies) is # only done once when building this image and is skipped later. RUN hatch env create default RUN hatch env create lint ENTRYPOINT ["hatch"] ================================================ FILE: GEMINI.md ================================================ # Coworker Relationship - We are a team. Your success is my success. - I'm your boss, but we're informal. - We both have valuable, complementary experience. - It's okay to admit when we don't know something. - Push back with evidence. # Coding Standards - Use simple, clean, and maintainable solutions. - Make the smallest reasonable changes. Ask for permission before rewriting. - Match the existing code style. - Stay on task. Create issues for unrelated fixes. - Don't remove comments unless they are false. - Use evergreen comments. - No mock implementations. - Do not rewrite code to fix a bug without permission. - Use evergreen naming conventions. # Documentation - Store documentation in the `documentation` directory. - Use Markdown and create an index named `intro.md` with links. - Document all commands, sub-commands, and options with examples. # Anlyzing Python Code - When analyzing Python code, use the `api` module to parse it, UNLESS instructed otherwise. - Use `api.get_docstring()` to locate a docstring for an item. - To find type hints, walk the AST using `api.walk_tree()` looking for type parameters with `ast.TypeVar()`, `ast.ParamSpec()`, and `ast.TypeVarTuple()`. # Getting Help - Ask for clarification. - Ask for help when needed. # Testing - Tests must cover the implemented functionality. - Pay attention to logs and test output. - Test output must be pristine. - Test for expected errors. - Practice TDD: 1. Write a failing test. 2. Write the minimum code to pass the test. 3. Refactor. 4. Repeat. # Specific Technologies - @~/.gemini/docs/python.md ================================================ FILE: LICENSE.txt ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2018 Kaggle Inc Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: README.md ================================================ # Kaggle CLI The official CLI to interact with [Kaggle](https://www.kaggle.com). --- [User documentation](docs/README.md) --- ## Key Features Some of the key features are: * List competitions, download competition data, submit to a competition. * List, create, update, download or delete datasets. * List, create, update, download or delete models & model variations. * List, update & run, download code & output or delete kernels (notebooks). ## Installation Install the `kaggle` package with [pip](https://pypi.org/project/pip/): ```sh pip install kaggle ``` Additional installation instructions can be found [here](docs/README.md#installation). ## Quick start Explore the available commands by running: ```sh kaggle --help ``` See the [User documentation](docs/README.md) for more examples & tutorials. ## Development ### Prerequisites We use [hatch](https://hatch.pypa.io) to manage this project. Follow these [instructions](https://hatch.pypa.io/latest/install/) to install it. ### Run `kaggle` from source #### Option 1: Execute a one-liner of code from the command line ```sh hatch run kaggle datasets list ``` #### Option 2: Run many commands in a shell ```sh hatch shell # Inside the shell, you can run many commands kaggle datasets list kaggle competitions list ... ``` ### Lint / Format ```sh # Lint check hatch run lint:style hatch run lint:typing hatch run lint:all # for both # Format hatch run lint:fmt ``` ### Tests Note: These tests are not true unit tests and are calling the Kaggle web server. ```sh # Run against kaggle.com hatch run test:prod # Run against a local web server (Kaggle engineers only) hatch run test:local ``` ### Integration Tests To run integration tests on your local machine, you need to set up your Kaggle credentials. You can do this by following the [authentication instructions](docs/README.md#authentication). After setting up your credentials, you can run the integration tests as follows: ```sh hatch run test:integration ``` ### Running `hatch` commands inside Docker This is useful to run in a consistent environment and easily switch between Python versions. The following shows how to run `hatch run lint:all` but this also works for any other hatch commands: ``` # Use default Python version ./docker-hatch run lint:all ``` ## Changelog See [CHANGELOG](CHANGELOG.md). ## Contributing See [CONTRIBUTING.md](CONTRIBUTING.md). ## License The Kaggle CLI is released under the [Apache 2.0 license](LICENSE.txt). ================================================ FILE: SECURITY.md ================================================ # Security Policy ## Supported Versions Security updates are applied only to the latest release. ## Reporting a Vulnerability If you have discovered a security vulnerability in this project, please report it privately. **Do not disclose it as a public issue.** This gives us time to work with you to fix the issue before public exposure, reducing the chance that the exploit will be used before a patch is released. Please disclose it at [security advisory](https://github.com/Kaggle/kaggle-cli/security/advisories/new). The vulnerabilities will be addressed as soon as possible, with a maximum of 90 days before a public exposure. ================================================ FILE: docker-hatch ================================================ #!/bin/bash set -e PYTHON_VERSION='3.11.6' # Default Python version to use. CACHE_FLAG='' usage() { cat << EOF Usage: $0 [OPTIONS] HATCH-OPTIONS-COMMAND-ARGS Run the given HATCH-OPTIONS-COMMAND-ARGS in a container. Options: -h, --help Show documentation on how to use $0 -v, --python-version PYTHON-VERSION Run in a container using this specfic Python version. Valid versions are tags of the \`python\` Docker image: https://hub.docker.com/_/python -f, --force-rebuild-docker-image Force re-build the Docker image. EOF } while :; do case "$1" in -h|--help) usage exit ;; -v|--python-version) if [[ -z $2 ]]; then usage printf 'ERROR: No PYTHON-VERSION specified after the %s flag.\n' "$1" >&2 exit fi PYTHON_VERSION=$2 shift # skip the flag value ;; -f|--force-rebuild-docker-image) CACHE_FLAG='--no-cache' ;; # Use this marker to indicate that the rest of the command is for hatch --) shift # skip this one to pass the rest to hatch break ;; -?*) usage printf 'ERROR: Unknown option: %s\n' "$1" >&2 exit ;; *) break esac shift done DOCKER_IMAGE="kaggle-cli-docker-hatch:${PYTHON_VERSION}" readonly PYTHON_VERSION readonly DOCKER_IMAGE readonly CACHE_FLAG echo "Running 'hatch $@' inside a Docker container using Python ${PYTHON_VERSION}" docker build -f Dockerfile \ -t ${DOCKER_IMAGE} \ ${CACHE_FLAG} \ --cache-from ${DOCKER_IMAGE} \ --build-arg PYTHON_VERSION=${PYTHON_VERSION} \ --quiet \ . set -x docker run -it --rm -v $PWD:/working -v ~/.kaggle:/root/.kaggle -w /working ${DOCKER_IMAGE} "$@" ================================================ FILE: docs/Makefile ================================================ # Minimal makefile for Sphinx documentation # # You can set these variables from the command line, and also # from the environment for the first two. SPHINXOPTS ?= SPHINXBUILD ?= sphinx-build SOURCEDIR = . BUILDDIR = _build SPHINX_APIDOC_OPTIONS=members,show-inheritance # Put it first so that "make" without argument is like "make help". help: @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) .PHONY: help Makefile # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) cleaner: Makefile rm -rf _build source ================================================ FILE: docs/README.md ================================================ # Kaggle CLI Documentation Welcome to the Kaggle CLI documentation. This guide provides detailed information on how to use the Kaggle command-line interface to interact with Kaggle's platform. ## Installation Note: Ensure you have Python 3.11+ and the package manager `pip` installed. Install the `kaggle` package with [pip](https://pypi.org/project/pip/): ```sh pip install kaggle ``` If you run into a `Command kaggle not found` error, ensure that your Python executable scripts are in your $PATH. For a local user install on Linux, the default location is `~/.local/bin`. On Windows, the default location is `$PYTHON_HOME/Scripts`. ## Authentication First, you will need a Kaggle account. You can sign up [here](https://www.kaggle.com/account/login). After login, you can download your Kaggle API credentials at https://www.kaggle.com/settings by clicking on the "Generate New Token" button under the "API" section. ### Option 1: Environment variable ```sh export KAGGLE_API_TOKEN=xxxxxxxxxxxxxx # Copied from the settings UI ``` ### Option 2: API token file Store your Kaggle API token obtained from your [Kaggle account settings page](https://www.kaggle.com/settings) in a file at `~/.kaggle/access_token`. ### Option 3: Legacy API credentials file From your [Kaggle account settings page](https://www.kaggle.com/settings), under "Legacy API Credentials", click on the "Create Legacy API Key" button to generate a `kaggle.json` file and store it at `~/.kaggle/kaggle.json`. ## CLI Usage Run the following command to list the available commands: ```sh kaggle --help ``` The Kaggle CLI is organized into several command groups: * [Competitions](./competitions.md): Manage and participate in Kaggle competitions. * [Datasets](./datasets.md): Search, download, and manage Kaggle datasets. * [Kernels](./kernels.md): Interact with Kaggle Kernels (notebooks and scripts). * [Models](./models.md): Manage your Kaggle Models. * [Model Variations](./model_variations.md): Manage variations of your Kaggle Models. * [Model Variation Versions](./model_variations_versions.md): Manage versions of your Kaggle Model Variations. * [Configuration](./configuration.md): Configure the Kaggle CLI. ## Tutorials Explore these tutorials to learn how to perform common tasks: * [Tutorials](./tutorials.md) * [Create a Dataset](./tutorials.md#tutorial-create-a-dataset) * [Find and Download a Dataset](./tutorials.md#tutorial-find-and-download-a-dataset) * [Create a Model](./tutorials.md#tutorial-create-a-model) * [Create a Model Variation](./tutorials.md#tutorial-create-a-model-variation) * [Create a Model Variation Version](./tutorials.md#tutorial-create-a-model-variation-version) * [How to Submit to a Competition](./tutorials.md#tutorial-how-to-submit-to-a-competition) * [How to Submit to a Code Competition](./tutorials.md#tutorial-how-to-submit-to-a-code-competition) ================================================ FILE: docs/competitions.md ================================================ # Competitions Commands Commands for interacting with Kaggle competitions. For tutorials on how to submit to competitions : * [How to Submit to a Competition](./tutorials.md#tutorial-how-to-submit-to-a-competition) * [How to Submit to a Code Competition](./tutorials.md#tutorial-how-to-submit-to-a-code-competition) ## `kaggle competitions list` Lists available competitions. **Usage:** ```bash kaggle competitions list [options] ``` **Options:** * `--group `: Filter by competition group. Valid options: `general`, `entered`, `inClass`. * `--category `: Filter by competition category. Valid options: `all`, `featured`, `research`, `recruitment`, `gettingStarted`, `masters`, `playground`. * `--sort-by `: Sort results. Valid options: `grouped`, `prize`, `earliestDeadline`, `latestDeadline`, `numberOfTeams`, `recentlyCreated` (default: `latestDeadline`). * `-p, --page `: Page number for results (default: 1). * `-s, --search `: Search term. * `-v, --csv`: Print results in CSV format. **Example:** List featured competitions in the general group, sorted by prize: ```bash kaggle competitions list --group general --category featured --sort-by prize ``` **Purpose:** This command helps you discover new competitions or find specific ones based on various criteria. ## `kaggle competitions files` Lists files for a specific competition. **Usage:** ```bash kaggle competitions files [options] ``` **Arguments:** * ``: Competition URL suffix (e.g., `titanic`). **Options:** * `-v, --csv`: Print results in CSV format. * `-q, --quiet`: Suppress verbose output. * `--page-token `: Page token for results paging. * `--page-size `: Number of items to show on a page (default: 20, max: 200). **Example:** List the first 3 files for the "titanic" competition in CSV format, quietly: ```bash kaggle competitions files titanic --page-size=3 -v -q ``` **Purpose:** Use this command to see the data files available for a competition before downloading them. ## `kaggle competitions download` Downloads competition files. **Usage:** ```bash kaggle competitions download [options] ``` **Arguments:** * ``: Competition URL suffix (e.g., `titanic`). **Options:** * `-f, --file `: Specific file to download (downloads all if not specified). * `-p, --path `: Folder to download files to (defaults to current directory). * `-w, --wp`: Download files to the current working path (equivalent to `-p .`). * `-o, --force`: Force download, overwriting existing files. * `-q, --quiet`: Suppress verbose output. **Examples:** 1. Download all files for the "titanic" competition to the current directory, overwriting existing files, quietly: ```bash kaggle competitions download titanic -w -o -q ``` 2. Download the `test.csv` file from the "titanic" competition to a folder named `tost`: ```bash kaggle competitions download titanic -f test.csv -p tost ``` **Purpose:** This command allows you to get the necessary data files for a competition onto your local machine. ## `kaggle competitions submit` Makes a new submission to a competition. **Usage:** ```bash kaggle competitions submit -f -m [options] ``` **Arguments:** * ``: Competition URL suffix (e.g., `house-prices-advanced-regression-techniques`). * `-f, --file `: The submission file. * `-m, --message `: The submission message. **Options:** * `-k, --kernel `: Name of the kernel (notebook) to submit (for code competitions). * `-v, --version `: Version of the kernel to submit (e.g. `2`). * `-q, --quiet`: Suppress verbose output. * `--sandbox`: Mark submission as a sandbox submission (competition hosts/admins only). **Example: Standard (not code) competition:** Submit `sample_submission.csv` to the "house-prices-advanced-regression-techniques" competition with the message "Test message": ```bash kaggle competitions submit house-prices-advanced-regression-techniques -f sample_submission.csv -m "Test message" ``` **Example: Code competition:** Submit the `submission.csv` produced by version `3` of your `/rsna-submission` for the `rsna-2024-lumbar-spine-degenerative-classification` competition: ```bash kaggle competitions submit rsna-2024-lumbar-spine-degenerative-classification -f submission.csv -k /rsna-submission -v 3 -m "Test message" ``` **Purpose:** Use this command to upload your predictions or code to a competition for scoring. ## `kaggle competitions submissions` Shows your past submissions for a competition. **Usage:** ```bash kaggle competitions submissions [options] ``` **Arguments:** * ``: Competition URL suffix (e.g., `house-prices-advanced-regression-techniques`). **Options:** * `-v, --csv`: Print results in CSV format. * `-q, --quiet`: Suppress verbose output. **Example:** Show submissions for "house-prices-advanced-regression-techniques" in CSV format, quietly: ```bash kaggle competitions submissions house-prices-advanced-regression-techniques -v -q ``` **Purpose:** This command allows you to review your previous submission attempts and their scores. ## `kaggle competitions leaderboard` Gets competition leaderboard information. **Usage:** ```bash kaggle competitions leaderboard [options] ``` **Arguments:** * ``: Competition URL suffix (e.g., `titanic`). **Options:** * `-s, --show`: Show the top of the leaderboard in the console. * `-d, --download`: Download the entire leaderboard to a CSV file. * `-p, --path `: Folder to download the leaderboard to (if `-d` is used). * `-v, --csv`: Print results in CSV format (used with `-s`). * `-q, --quiet`: Suppress verbose output. **Examples:** 1. Download the "titanic" leaderboard to a folder named `leaders`, quietly: ```bash kaggle competitions leaderboard titanic -d -p leaders -q ``` 2. Download the leaderboard and save it to `leaderboard.txt`: ```bash kaggle competitions leaderboard titanic > leaderboard.txt ``` **Purpose:** This command lets you view your ranking and the scores of other participants in a competition. ================================================ FILE: docs/conf.py ================================================ # Configuration file for the Sphinx documentation builder. # # For the full list of built-in configuration values, see the documentation: # https://www.sphinx-doc.org/en/master/usage/configuration.html # -- Project information ----------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information import sys from pathlib import Path sys.path.insert(0, str(Path("..").resolve())) def autodoc_skip_member(app, what, name, obj, skip, options): # 'what' is the type of the object (e.g., 'function', 'class') # 'name' is the name of the object (e.g., 'myfunction') # 'obj' is the Python object itself # 'skip' is a boolean indicating whether it's already marked to be skipped # 'options' are the autodoc options # Skip all CLI methods if name.endswith("_cli"): # Param what should be 'method' but is 'class'; obj is the function object. # https://github.com/sphinx-doc/sphinx/issues/6808 return True return skip # Return the original skip value for other members def setup(app): app.connect("autodoc-skip-member", autodoc_skip_member) project = "kaggle-cli" copyright = "2025, Kaggle" author = "Kaggle" # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration extensions = [ "sphinx.ext.apidoc", "sphinx.ext.autodoc", "sphinx.ext.autosectionlabel", "sphinx.ext.autosummary", "sphinx.ext.napoleon", "myst_parser", ] templates_path = ["_templates"] exclude_patterns = ["_build", "Thumbs.db", ".DS_Store", "**/kaggle/models/*.py"] source_suffix = { ".rst": "restructuredtext", ".txt": "restructuredtext", ".md": "markdown", } # -- Options for HTML output ------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output html_theme = "alabaster" autoclass_content = "both" myst_heading_anchors = 2 suppress_warnings = ["ref.unknown"] # -- Options for sphinx.ext.apidoc ------------------------------------------- apidoc_module_dir = "../src/kaggle" apidoc_output_dir = "source" apidoc_excluded_paths = ["api/kaggle_api.py"] apidoc_separate_modules = True ================================================ FILE: docs/configuration.md ================================================ # Kaggle CLI Configuration The Kaggle CLI uses a configuration file to store settings such as your API credentials and default values for commands. ## Configuration Commands ### `config view` Displays the current configuration values. **Usage:** ```bash kaggle config view ``` **Purpose:** This command allows you to inspect the current settings of your Kaggle CLI, such as the configured API endpoint, proxy settings, and default competition. ### `config set` Sets a specific configuration value. **Usage:** ```bash kaggle config set -n -v ``` **Arguments:** * `-n, --name `: The name of the configuration parameter to set. Valid options are `competition`, `path`, and `proxy`. * `-v, --value `: The value to set for the configuration parameter. * For `competition`: The competition URL suffix (e.g., `titanic`). * For `path`: The default folder where files will be downloaded. * For `proxy`: The proxy server URL. **Example:** Set the default competition to "titanic": ```bash kaggle config set -n competition -v titanic ``` **Purpose:** Use this command to customize the behavior of the Kaggle CLI, such as setting a default competition to avoid specifying it in every command, defining a default download path, or configuring a proxy server. ### `config unset` Clears a specific configuration value, reverting it to its default. **Usage:** ```bash kaggle config unset -n ``` **Arguments:** * `-n, --name `: The name of the configuration parameter to clear. Valid options are `competition`, `path`, and `proxy`. **Example:** Clear the default competition: ```bash kaggle config unset -n competition ``` **Purpose:** This command removes a previously set configuration value, allowing the CLI to use its default behavior or prompt for the value if required. ## Configuration File Location The Kaggle CLI configuration is typically stored in a file named `kaggle.json` located in the `~/.kaggle/` directory on Linux and macOS, or `C:\Users\\.kaggle\` on Windows. This file contains your API username and key: ```json {"username":"YOUR_USERNAME","key":"YOUR_API_KEY"} ``` You can download this file from your Kaggle account page (`https://www.kaggle.com//account`) and place it in the correct directory. Alternatively, you can set the `KAGGLE_USERNAME` and `KAGGLE_KEY` environment variables. ================================================ FILE: docs/datasets.md ================================================ # Datasets Commands Commands for interacting with Kaggle datasets. ## `kaggle datasets list` Lists available datasets. **Usage:** ```bash kaggle datasets list [options] ``` **Options:** * `--sort-by `: Sort results. Valid options: `hottest`, `votes`, `updated`, `active` (default: `hottest`). * `--size `: DEPRECATED. Use `--min-size` and `--max-size`. * `--file-type `: Filter by file type. Valid options: `all`, `csv`, `sqlite`, `json`, `bigQuery`. * `--license `: Filter by license. Valid options: `all`, `cc`, `gpl`, `odb`, `other`. * `--tags `: Filter by tags (comma-separated tag IDs). * `-s, --search `: Search term. * `-m, --mine`: Display only your datasets. * `--user `: Filter by a specific user or organization. * `-p, --page `: Page number for results (default: 1). * `-v, --csv`: Print results in CSV format. * `--max-size `: Maximum dataset size in bytes. * `--min-size `: Minimum dataset size in bytes. **Examples:** 1. List your own datasets: ```bash kaggle datasets list -m ``` 2. List CSV datasets, page 2, sorted by last updated, containing "student" in their title, with size between 13000 and 15000 bytes: ```bash kaggle datasets list --file-type csv --page 2 --sort-by updated -s student --min-size 13000 --max-size 15000 ``` 3. List datasets with an ODB license, tagged with "internet", and matching the search term "telco": ```bash kaggle datasets list --license odb --tags internet --search telco ``` **Purpose:** This command helps you find datasets on Kaggle based on various criteria like owner, file type, tags, and size. ## `kaggle datasets files` Lists files for a specific dataset. **Usage:** ```bash kaggle datasets files [options] ``` **Arguments:** * ``: Dataset URL suffix in the format `owner/dataset-name` (e.g., `kerneler/brazilian-bird-observation-metadata-from-wikiaves`). **Options:** * `-v, --csv`: Print results in CSV format. * `--page-token `: Page token for results paging. * `--page-size `: Number of items to show on a page (default: 20, max: 200). **Example:** List the first 7 files for the dataset `kerneler/brazilian-bird-observation-metadata-from-wikiaves`: ```bash kaggle datasets files kerneler/brazilian-bird-observation-metadata-from-wikiaves --page-size=7 ``` **Purpose:** Use this command to see the individual files within a dataset before downloading. ## `kaggle datasets download` Downloads dataset files. **Usage:** ```bash kaggle datasets download [options] ``` **Arguments:** * ``: Dataset URL suffix (e.g., `willianoliveiragibin/pixar-films`). **Options:** * `-f, --file `: Specific file to download (downloads all if not specified). * `-p, --path `: Folder to download files to (defaults to current directory). * `-w, --wp`: Download files to the current working path. * `--unzip`: Unzip the downloaded file (deletes the .zip file afterwards). * `-o, --force`: Force download, overwriting existing files. * `-q, --quiet`: Suppress verbose output. **Examples:** 1. Download all files for the dataset `willianoliveiragibin/pixar-films`: ```bash kaggle datasets download -d willianoliveiragibin/pixar-films ``` 2. Download the dataset `goefft/public-datasets-with-file-types-and-columns`, unzip it into the `tmp` folder, overwriting if necessary, and suppress output: ```bash kaggle datasets download goefft/public-datasets-with-file-types-and-columns -p tmp --unzip -o -q ``` 3. Download the specific file `dataset_results.csv` from `goefft/public-datasets-with-file-types-and-columns` to the current working directory, quietly, and force overwrite: ```bash kaggle datasets download goefft/public-datasets-with-file-types-and-columns -f dataset_results.csv -w -q -o ``` **Purpose:** This command allows you to retrieve dataset files for local use. ## `kaggle datasets init` Initializes a metadata file (`dataset-metadata.json`) for creating a new dataset. See [metadata file format](./datasets_metadata.md). **Usage:** ```bash kaggle datasets init -p ``` **Options:** * `-p, --path `: The path to the folder where the `dataset-metadata.json` file will be created (defaults to the current directory). **Example:** Initialize a dataset metadata file in the `tests/dataset` folder: ```bash kaggle datasets init -p tests/dataset ``` **Purpose:** This command creates a template `dataset-metadata.json` file that you need to edit before creating a new dataset on Kaggle. This file contains information like the dataset title, ID (slug), and licenses. ## `kaggle datasets create` Creates a new dataset on Kaggle. **Usage:** ```bash kaggle datasets create -p [options] ``` **Options:** * `-p, --path `: Path to the folder containing the data files and the `dataset-metadata.json` file (defaults to the current directory). * `-u, --public`: Make the dataset public (default is private). * `-q, --quiet`: Suppress verbose output. * `-t, --keep-tabular`: Do not convert tabular files to CSV (default is to convert). * `-r, --dir-mode `: How to handle directories: `skip` (ignore), `zip` (compressed upload), `tar` (uncompressed upload) (default: `skip`). **Example:** Create a new public dataset from the files in `tests/dataset`, quietly, without converting tabular files, and skipping subdirectories. (Assumes `dataset-metadata.json` in `tests/dataset` has been properly edited with title and slug): ```bash # Example: Edit dataset-metadata.json first # sed -i 's/INSERT_TITLE_HERE/My Dataset Title/' tests/dataset/dataset-metadata.json # sed -i 's/INSERT_SLUG_HERE/my-dataset-slug/' tests/dataset/dataset-metadata.json kaggle datasets create -p tests/dataset --public -q -t -r skip ``` **Purpose:** This command uploads your local data files and the associated metadata to create a new dataset on Kaggle. ## `kaggle datasets version` Creates a new version of an existing dataset. **Usage:** ```bash kaggle datasets version -p -m [options] ``` **Options:** * `-p, --path `: Path to the folder containing the updated data files and `dataset-metadata.json` (defaults to current directory). * `-m, --message `: (Required) Message describing the new version. * `-q, --quiet`: Suppress verbose output. * `-t, --keep-tabular`: Do not convert tabular files to CSV. * `-r, --dir-mode `: Directory handling mode (`skip`, `zip`, `tar`). * `-d, --delete-old-versions`: Delete old versions of this dataset. **Example:** Create a new version of a dataset using files from `tests/dataset` with version notes "Updated data", quietly, keeping tabular formats, skipping directories, and deleting old versions: ```bash kaggle datasets version -m "Updated data" -p tests/dataset -q -t -r skip -d ``` **Purpose:** Use this command to update an existing dataset with new files or metadata changes. ## `kaggle datasets metadata` Downloads metadata for a dataset or updates existing local metadata. **Usage:** ```bash kaggle datasets metadata [options] ``` **Arguments:** * ``: Dataset URL suffix (e.g., `goefft/public-datasets-with-file-types-and-columns`). **Options:** * `-p, --path `: Directory to download/update metadata file (`dataset-metadata.json`). Defaults to current working directory. * `--update`: Update existing local metadata file instead of downloading anew. **Example:** Download metadata for the dataset `goefft/public-datasets-with-file-types-and-columns` into the `tests/dataset` folder: ```bash kaggle datasets metadata goefft/public-datasets-with-file-types-and-columns -p tests/dataset ``` **Purpose:** This command allows you to fetch the `dataset-metadata.json` file for an existing dataset, which can be useful for inspection or as a template for creating a new version. ## `kaggle datasets status` Gets the creation status of a dataset. **Usage:** ```bash kaggle datasets status ``` **Arguments:** * ``: Dataset URL suffix (e.g., `goefft/public-datasets-with-file-types-and-columns`). **Example:** Get the status of the dataset `goefft/public-datasets-with-file-types-and-columns`: ```bash kaggle datasets status goefft/public-datasets-with-file-types-and-columns ``` **Purpose:** After creating or updating a dataset, this command helps you check if the process was successful or if there were any issues. ## `kaggle datasets delete` Deletes a dataset from Kaggle. **Usage:** ```bash kaggle datasets delete [options] ``` **Arguments:** * ``: Dataset URL suffix (e.g., `username/dataset-slug`). **Options:** * `-y, --yes`: Automatically confirm deletion without prompting. **Example:** Delete the dataset `username/dataset-slug` and automatically confirm: ```bash kaggle datasets delete username/dataset-slug --yes ``` **Purpose:** This command permanently removes one of your datasets from Kaggle. Use with caution. ================================================ FILE: docs/datasets_metadata.md ================================================ The Kaggle API follows the [Data Package specification](https://frictionlessdata.io/specs/data-package/) for specifying metadata when creating new Datasets and Dataset versions. Next to your files, you have to put a special `dataset-metadata.json` file in your upload folder alongside the files for each new Dataset (version). Here's a basic example for `dataset-metadata.json`: ``` { "title": "My Awesome Dataset", "id": "timoboz/my-awesome-dataset", "licenses": [{"name": "CC0-1.0"}] } ``` You can also use the API command `kaggle datasets init -p /path/to/dataset` to have the API create this file for you. Here's an example containing file metadata: ``` { "title": "My Awesome Dataset", "subtitle": "My awesomer subtitle", "description": "My awesomest description", "id": "timoboz/my-awesome-dataset", "id_no": 12345, "licenses": [{"name": "CC0-1.0"}], "resources": [ { "path": "my-awesome-data.csv", "description": "This is my awesome data!", "schema": { "fields": [ { "name": "StringField", "description": "String field description", "type": "string" }, { "name": "NumberField", "description": "Number field description", "type": "number" }, { "name": "DateTimeField", "description": "Date time field description", "type": "datetime" } ] } }, { "path": "my-awesome-extra-file.txt", "description": "This is my awesome extra file!" } ], "keywords": [ "beginner", "tutorial" ] } ``` ## Contents The following metadata is currently supported: * `kaggle datasets create` (create a new Dataset): * `title`: Title of the dataset, must be between 6 and 50 characters in length. * `subtitle`: Subtitle of the dataset, must be between 20 and 80 characters in length. * `description`: Description of the dataset. * `id`: The URL slug of your new dataset, a combination of: 1. Your username or organization slug (if you are a member of an organization). 2. A unique Dataset slug, must be between 3 and 50 characters in length. * `licenses`: Must have exactly one entry that specifies the license. Only `name` is evaluated, all other information is ignored. See below for options. * `resources`: Contains an array of files that are being uploaded. (Note - this is not required, nor if included, does it need to include all of the files to be uploaded.): * `path`: File path. * `description`: File description. * `schema`: File schema (definition below): * `fields`: Array of fields in the dataset. Please note that this needs to include ALL of the fields in the data in order or they will not be matched up correctly. A later version of the API will fix this bug. * `name`: Field name * `title`: Field description * `type`: Field type. A best-effort list of types will be kept at the bottom of this page, but new types may be added that are not documented here. * `keywords`: Contains an array of strings that correspond to an existing tag on Kaggle. If a specified tag doesn't exist, the upload will continue, but that specific tag won't be added. * `kaggle datasets version` (create a new version for an existing Dataset): * `subtitle`: Subtitle of the dataset, must be between 20 and 80 characters in length. * `description`: Description of the dataset. * `id`: The URL slug of the dataset you want to update (see above). You must be the owner or otherwise have edit rights for this dataset. One of `id` or `id_no` must be specified. If both are, `id_no` will be preferred. * `id_no`: The ID of the dataset. One of `id` or `id_no` must be specified. You must be the owner or otherwise have edit rights for this dataset. If both are, `id_no` will be preferred. * `resources`: Contains an array of files that are being uploaded. (Note - this is not required, nor if included, does it need to include all of the files to be uploaded.): * `path`: File path. * `description`: File description. * `schema`: File schema (definition below): * `fields`: Array of fields in the dataset. Please note that this needs to include ALL of the fields in the data in order or they will not be matched up correctly. A later version of the API will fix this bug. * `name`: Field name * `title`: Field description * `type`: Field type. A best-effort list of types will be kept at the bottom of this page, but new types may be added that are not documented here. * `keywords`: Contains an array of strings that correspond to an existing tag on Kaggle. If a specified tag doesn't exist, the upload will continue, but that specific tag won't be added. We will add further metadata processing in upcoming versions of the API. ## Licenses You can specify the following licenses for your datasets: * `CC0-1.0`: [CC0: Public Domain](https://creativecommons.org/publicdomain/zero/1.0/) * `CC-BY-SA-3.0`: [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) * `CC-BY-SA-4.0`: [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/) * `CC-BY-NC-SA-4.0`: [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) * `GPL-2.0`: [GPL 2](http://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html) * `ODbL-1.0`: Database: [Open Database](http://opendatacommons.org/licenses/odbl/1.0/), Contents: © Original Authors * `DbCL-1.0`: Database: [Open Database](http://opendatacommons.org/licenses/odbl/1.0/), Contents: [Database Contents](http://opendatacommons.org/licenses/dbcl/1.0/) * `copyright-authors`: Data files © Original Authors * `other`: Other (specified in description) * `unknown`: Unknown * `CC-BY-4.0`: https://creativecommons.org/licenses/by/4.0/ * `CC-BY-NC-4.0`: https://creativecommons.org/licenses/by-nc/4.0/ * `PDDL`: https://opendatacommons.org/licenses/pddl/1.0/ * `CC-BY-3.0`: https://creativecommons.org/licenses/by/3.0/ * `CC-BY-3.0-IGO`: https://creativecommons.org/licenses/by/3.0/igo/ * `US-Government-Works`: https://www.usa.gov/government-works/ * `CC-BY-NC-SA-3.0-IGO`: https://creativecommons.org/licenses/by-nc-sa/3.0/igo/ * `CDLA-Permissive-1.0`: https://cdla.io/permissive-1-0/ * `CDLA-Sharing-1.0`: https://cdla.io/sharing-1-0/ * `CC-BY-ND-4.0`: https://creativecommons.org/licenses/by-nd/4.0/ * `CC-BY-NC-ND-4.0`: https://creativecommons.org/licenses/by-nc-nd/4.0/ * `ODC-BY-1.0`: https://opendatacommons.org/licenses/by/1-0/index.html * `LGPL-3.0`: http://www.gnu.org/licenses/lgpl-3.0.html * `AGPL-3.0`: http://www.gnu.org/licenses/agpl-3.0.html * `FDL-1.3`: http://www.gnu.org/licenses/fdl-1.3.html * `EU-ODP-Legal-Notice`: https://ec.europa.eu/info/legal-notice_en * `apache-2.0`: https://www.apache.org/licenses/LICENSE-2.0 * `GPL-3.0`: [GPL 2](https://www.gnu.org/licenses/gpl-3.0.html) ## Data types You can specify the following data types * `string` * `boolean` * `numeric` * `datetime` * `id` * `uuid` * `latitude` * `longitude` * `coordinates` * `country` * `province` (these are states in the US) * `postalcode` * `address` * `email` * `url` * `integer` * `decimal` * `city` ================================================ FILE: docs/index.rst ================================================ .. _kaggle: .. kaggle-cli documentation master file, created by sphinx-quickstart on Thu Jun 26 22:53:21 2025. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. kaggle-cli documentation ======================== .. toctree:: :maxdepth: 2 :caption: Contents: intro configuration competitions datasets kernels models model_instances model_instances_versions tutorials ================================================ FILE: docs/kernels.md ================================================ # Kernels Commands Commands for interacting with Kaggle Kernels (notebooks and scripts). ## `kaggle kernels list` Lists available kernels. **Usage:** ```bash kaggle kernels list [options] ``` **Options:** * `-m, --mine`: Display only your kernels. * `-p, --page `: Page number for results (default: 1). * `--page-size `: Number of items per page (default: 20). * `-s, --search `: Search term. * `-v, --csv`: Print results in CSV format. * `--parent `: Filter by parent kernel (format: `owner/kernel-slug`). * `--competition `: Filter by competition. * `--dataset `: Filter by dataset (format: `owner/dataset-slug`). * `--user `: Filter by a specific user. * `--language `: Filter by language (`all`, `python`, `r`, `sqlite`, `julia`). * `--kernel-type `: Filter by kernel type (`all`, `script`, `notebook`). * `--output-type `: Filter by output type (`all`, `visualizations`, `data`). * `--sort-by `: Sort results (`hotness`, `commentCount`, `dateCreated`, `dateRun`, `relevance`, `scoreAscending`, `scoreDescending`, `viewCount`, `voteCount`). Default: `hotness`. **Examples:** 1. List your own kernels containing "Exercise" in the title, page 2, 5 items per page, in CSV format, sorted by run date: ```bash kaggle kernels list -m -s Exercise --page-size 5 -p 2 -v --sort-by dateRun ``` 2. List kernels that are children of `$KAGGLE_DEVELOPER/exercise-lists` (replace `$KAGGLE_DEVELOPER` with your username): ```bash kaggle kernels list --parent $KAGGLE_DEVELOPER/exercise-lists ``` 3. List the first 5 kernels for the "house-prices-advanced-regression-techniques" competition: ```bash kaggle kernels list --competition house-prices-advanced-regression-techniques --page-size 5 ``` 4. List the first 5 kernels associated with the dataset `dansbecker/home-data-for-ml-course`: ```bash kaggle kernels list --dataset dansbecker/home-data-for-ml-course --page-size 5 ``` 5. List Python notebooks by user `$KAGGLE_DEVELOPER` that output data: ```bash kaggle kernels list --user $KAGGLE_DEVELOPER --language python --kernel-type notebook --output-type data ``` **Purpose:** This command allows you to find kernels based on various filters like ownership, associated competition/dataset, language, or type. ## `kaggle kernels files` Lists output files for a specific kernel. **Usage:** ```bash kaggle kernels files [options] ``` **Arguments:** * ``: Kernel URL suffix (format: `owner/kernel-slug`, e.g., `kerneler/sqlite-global-default`). **Options:** * `-v, --csv`: Print results in CSV format. * `--page-token `: Page token for results paging. * `--page-size `: Number of items to show on a page (default: 20, max: 200). **Example:** List the first output file for the kernel `kerneler/sqlite-global-default` in CSV format: ```bash kaggle kernels files kerneler/sqlite-global-default -v --page-size=1 ``` **Purpose:** Use this command to view the files generated by a kernel run. ## `kaggle kernels init` Initializes a metadata file (`kernel-metadata.json`) for a new or existing kernel. See [metadata file format](./kernels_metadata.md). **Usage:** ```bash kaggle kernels init -p ``` **Options:** * `-p, --path `: The path to the folder where the `kernel-metadata.json` file will be created (defaults to the current directory). **Example:** Initialize a kernel metadata file in the `tests/kernel` folder: ```bash kaggle kernels init -p tests/kernel ``` **Purpose:** This command creates a template `kernel-metadata.json` file. You need to edit this file with details like the kernel's title, ID (slug), language, kernel type, and data sources before pushing it to Kaggle. ## `kaggle kernels push` Pushes new code/notebook and metadata to a kernel, then runs the kernel. **Usage:** ```bash kaggle kernels push -p [options] ``` **Options:** * `--accelerator `: ID name of the accelerator to use during the run. E.g. "NvidiaTeslaP100" (aka default GPU), "NvidiaTeslaT4", "TpuV6E8". * `-p, --path `: Path to the folder containing the kernel file (e.g., `.ipynb`, `.Rmd`, `.py`) and the `kernel-metadata.json` file (defaults to the current directory). * `-t, --timeout `: Maximum run time in seconds. **Example:** Push the kernel from the `tests/kernel` folder (assuming it contains the kernel file and `kernel-metadata.json`): ```bash kaggle kernels push -p tests/kernel ``` **Purpose:** This command uploads your local kernel file and its metadata to Kaggle. If the kernel specified in the metadata exists under your account, it will be updated. Otherwise, a new kernel will be created. After uploading, Kaggle will attempt to run the kernel. Accelerators available as of Feb 2026: * NvidiaTeslaP100 * TpuV38 * NvidiaTeslaT4 * NvidiaTeslaT4Highmem * Tpu1VmV38 * NvidiaTeslaA100 * NvidiaL4 * TpuV5E8 * NvidiaL4X1 * TpuV6E8 * NvidiaH100 * NvidiaRtxPro6000 Some of these are only available to participants of specific competitions, and some are only available to Kaggle admins. ## `kaggle kernels pull` Pulls down the code/notebook and metadata for a kernel. **Usage:** ```bash kaggle kernels pull [options] ``` **Arguments:** * ``: Kernel URL suffix (format: `owner/kernel-slug`, e.g., `$KAGGLE_DEVELOPER/exercise-as-with`). **Options:** * `-p, --path `: Folder to download files to (defaults to current directory). * `-w, --wp`: Download files to the current working path. * `-m, --metadata`: Generate a `kernel-metadata.json` file along with the kernel code. **Examples:** 1. Pull the kernel `$KAGGLE_DEVELOPER/exercise-as-with` and its metadata into the `tests/kernel` folder: ```bash kaggle kernels pull -p tests/kernel $KAGGLE_DEVELOPER/exercise-as-with -m ``` 2. Pull the kernel `$KAGGLE_DEVELOPER/exercise-as-with` into the current working directory: ```bash kaggle kernels pull --wp $KAGGLE_DEVELOPER/exercise-as-with ``` **Purpose:** This command allows you to download the source code and optionally the metadata of a kernel from Kaggle to your local machine. ## `kaggle kernels output` Gets the data output from the latest run of a kernel. **Usage:** ```bash kaggle kernels output [options] ``` **Arguments:** * ``: Kernel URL suffix (e.g., `kerneler/using-google-bird-vocalization-model`). **Options:** * `-p, --path `: Folder to download output files to (defaults to current directory). * `-w, --wp`: Download files to the current working path. * `-o, --force`: Force download, overwriting existing files. * `-q, --quiet`: Suppress verbose output. * `--file-pattern `: Regex pattern to match against filenames. Only files matching the pattern will be downloaded. **Example:** Download the output of the kernel `kerneler/using-google-bird-vocalization-model`, forcing overwrite: ```bash kaggle kernels output kerneler/sqlite-global-default -o ``` Download PNG files only: ```bash kaggle kernels output --file-pattern ".*\.png$" # Only PNG files ``` **Purpose:** Use this command to retrieve the files generated by a kernel run, such as submission files, processed data, or visualizations. ## `kaggle kernels status` Displays the status of the latest run of a kernel. **Usage:** ```bash kaggle kernels status ``` **Arguments:** * ``: Kernel URL suffix (e.g., `kerneler/sqlite-global-default`). **Example:** Get the status of the kernel `kerneler/sqlite-global-default`: ```bash kaggle kernels status kerneler/sqlite-global-default ``` **Purpose:** This command tells you whether the latest run of your kernel is still running, completed successfully, or failed. ## `kaggle kernels delete` Deletes a kernel from Kaggle. **Usage:** ```bash kaggle kernels delete [options] ``` **Arguments:** * ``: Kernel URL suffix (format: `owner/kernel-slug`, e.g., `$KAGGLE_DEVELOPER/exercise-delete`). **Options:** * `-y, --yes`: Automatically confirm deletion without prompting. **Example:** Delete the kernel `$KAGGLE_DEVELOPER/exercise-delete` and automatically confirm: ```bash kaggle kernels delete $KAGGLE_DEVELOPER/exercise-delete --yes ``` **Purpose:** This command permanently removes one of your kernels from Kaggle. Use with caution. ================================================ FILE: docs/kernels_metadata.md ================================================ To upload and run a kernel, a special `kernel-metadata.json` file must be specified. Here's a basic example for `kernel-metadata.json`: ``` { "id": "timoboz/my-awesome-kernel", "id_no": 12345, "title": "My Awesome Kernel", "code_file": "my-awesome-kernel.ipynb", "language": "python", "kernel_type": "notebook", "is_private": "false", "enable_gpu": "false", "enable_internet": "false", "dataset_sources": ["timoboz/my-awesome-dataset"], "competition_sources": [], "kernel_sources": [], "model_sources": [] } ``` You can also use the API command `kaggle kernels init -p /path/to/kernel` to have the API create this file for you for a new kernel. If you wish to get the metadata for an existing kernel, you can use `kaggle kernels pull -p /path/to/download -k username/kernel-slug -m`. ## Contents We currently support the following metadata fields for kernels. * `id`: The URL slug of your kernel. One of `id` or `id_no` must be specified. If both are, `id_no` will be preferred. 1. Your username slug 2. A unique kernel slug * `id_no`: The kernel's numeric ID. One of `id` or `id_no` must be specified. If both are, `id_no` will be preferred. * `title`: The title of the kernel. Required for new kernels - optional for existing ones. Please be aware that kernel titles and slugs are linked to each other. A kernel slug is always the title lowercased with dashes (`-`) replacing spaces. * If you wish to rename your kernel, you may change the title within the metadata. However, you will need to update the `id` as well AFTER the rename is complete. * `code_file`: The path to your kernel source code. Required. If not an absolute path, it should be relative to the location of `kernel-metadata.json`. * `language`: The language your kernel is written in. Valid options are `python`, `r`, and `rmarkdown`. Required. * `kernel_type`: The type of kernel. Valid options are `script` and `notebook`. Required. * `is_private`: Whether or not the kernel should be private. If not specified, will be `true`. * `enable_gpu`: Whether or not the kernel should run on a GPU. If not specified, will be `false`. * `enable_internet`: Whether or not the kernel should be able to access the internet. If not specified, will be `false`. * `dataset_sources`: A list of dataset sources, specified as `"username/dataset-slug"` * `competition_sources`: A list of competition sources, specified as `"competition-slug"` * `kernel_sources`: A list of kernel sources, specified as `"username/kernel-slug"` * `model_sources`: A list of model sources, specified as `"username/model-slug/framework/variation-slug/version-number"` We will add further metadata processing in upcoming versions of the API. ================================================ FILE: docs/make.bat ================================================ @ECHO OFF pushd %~dp0 REM Command file for Sphinx documentation if "%SPHINXBUILD%" == "" ( set SPHINXBUILD=sphinx-build ) set SOURCEDIR=. set BUILDDIR=_build %SPHINXBUILD% >NUL 2>NUL if errorlevel 9009 ( echo. echo.The 'sphinx-build' command was not found. Make sure you have Sphinx echo.installed, then set the SPHINXBUILD environment variable to point echo.to the full path of the 'sphinx-build' executable. Alternatively you echo.may add the Sphinx directory to PATH. echo. echo.If you don't have Sphinx installed, grab it from echo.https://www.sphinx-doc.org/ exit /b 1 ) if "%1" == "" goto help %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% goto end :help %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O% :end popd ================================================ FILE: docs/model_variations.md ================================================ # Model Variation Commands Commands for interacting with variations of Kaggle Models. A model variation typically represents a specific framework of a parent model. ## `kaggle models variations init` Initializes a metadata file (`model-instance-metadata.json`) for creating a new model variation. Note that the name of the file reflects the old name for a variation, which was "instance". **Usage:** ```bash kaggle models variations init -p ``` **Options:** * `-p, --path `: The path to the folder where the `model-instance-metadata.json` file will be created (defaults to the current directory). **Example:** Initialize a model variation metadata file in the `tmp` folder: ```bash kaggle models variations init -p tmp ``` **Purpose:** This command creates a template `model-instance-metadata.json` file. You must edit this file with details such as the owner slug, the parent model slug, the variation (or instance) slug (URL-friendly name for this variations), and the framework (e.g., `tensorflow`, `pytorch`, `jax`, `sklearn`) before creating the variation. ## `kaggle models variations create` Creates a new model variation under an existing model on Kaggle. **Usage:** ```bash kaggle models variations create -p [options] ``` **Options:** * `-p, --path `: Path to the folder containing the model variation files and the `model-instance-metadata.json` file (defaults to the current directory). * `-q, --quiet`: Suppress verbose output. * `-r, --dir-mode `: How to handle directories within the upload: `skip` (ignore), `zip` (compressed upload), `tar` (uncompressed upload) (default: `skip`). **Example:** Create a new model variation using the metadata and files in the `tmp` folder, quietly, skipping subdirectories. (Assumes `model-instance-metadata.json` in `tmp` has been properly edited): ```bash # Example: Edit model-instance-metadata.json first # sed -i 's/INSERT_OWNER_SLUG_HERE/your-username/' tmp/model-instance-metadata.json # sed -i 's/INSERT_EXISTING_MODEL_SLUG_HERE/parent-model-slug/' tmp/model-instance-metadata.json # sed -i 's/INSERT_INSTANCE_SLUG_HERE/my-variation-slug/' tmp/model-instance-metadata.json # sed -i 's/INSERT_FRAMEWORK_HERE/jax/' tmp/model-instance-metadata.json # echo "a,b,c,d" > tmp/data.csv # Example model file kaggle models variations create -p tmp -q -r skip ``` **Purpose:** This command uploads your local model files (e.g., weights, architecture definition) and the associated variation metadata to create a new variation under a specified parent model on Kaggle. This effectively creates the first version of this model variation. ## `kaggle models variations get` Downloads the `model-instance-metadata.json` file for an existing model variation. **Usage:** ```bash kaggle models variations get -p ``` **Arguments:** * ``: Model variation URL suffix in the format `owner/model-slug/framework/variation-slug` (e.g., `$KAGGLE_DEVELOPER/test-model/jax/main`). **Options:** * `-p, --path `: Folder to download the `model-instance-metadata.json` file to. **Example:** Download the metadata for model variation `$KAGGLE_DEVELOPER/test-model/jax/main` into the `tmp` folder: ```bash kaggle models variations get $KAGGLE_DEVELOPER/test-model/jax/main -p tmp ``` **Purpose:** This command retrieves the metadata file for an existing model variation. This can be useful for inspection or as a basis for an update. ## `kaggle models variations files` Lists files for the current version of a model variation. **Usage:** ```bash kaggle models variations files [options] ``` **Arguments:** * ``: Model variation URL suffix (e.g., `$KAGGLE_DEVELOPER/test-model/jax/main`). **Options:** * `-v, --csv`: Print results in CSV format. * `--page-size `: Number of items per page (default: 20). * `--page-token `: Page token for results paging. **Example:** List the first 5 files for the model variation `$KAGGLE_DEVELOPER/test-model/jax/main` in CSV format: ```bash kaggle models variations files $KAGGLE_DEVELOPER/test-model/jax/main -v --page-size 5 ``` **Purpose:** Use this command to see the files associated with the latest version of a specific model variation. ## `kaggle models variations update` Updates an existing model variation on Kaggle using a local `model-instance-metadata.json` file. **Usage:** ```bash kaggle models variations update -p ``` **Options:** * `-p, --path `: Path to the folder containing the `model-instance-metadata.json` file with the updated information (defaults to the current directory). Note: This command only updates the metadata of the variation, not the files. To update files, create a new version. **Example:** Update the model variation whose details are in `tmp/model-instance-metadata.json` (ensure the slugs and owner in the JSON match an existing model variation): ```bash kaggle models variations update -p tmp ``` **Purpose:** Use this command to change the metadata of an existing model variation, such as its description or other fields defined in the `model-instance-metadata.json` file. This does not upload new files or create a new version. ## `kaggle models variations delete` Deletes a model variation from Kaggle. **Usage:** ```bash kaggle models variations delete [options] ``` **Arguments:** * ``: Model variation URL suffix in the format `owner/model-slug/framework/variation-slug` (e.g., `$KAGGLE_DEVELOPER/test-model/jax/main`). **Options:** * `-y, --yes`: Automatically confirm deletion without prompting. **Example:** Delete the model variation `$KAGGLE_DEVELOPER/test-model/jax/main` and automatically confirm: ```bash kaggle models variations delete $KAGGLE_DEVELOPER/test-model/jax/main -y ``` **Purpose:** This command permanently removes one of your model variations (and all its versions) from Kaggle. Use with caution. ================================================ FILE: docs/model_variations_versions.md ================================================ # Model Variation Versions Commands Commands for managing versions of a specific Kaggle Model Variation. Each version represents a snapshot of the model variation files at a point in time. ## `kaggle models variations versions create` Creates a new version of an existing model variation. **Usage:** ```bash kaggle models variations versions create -p [options] ``` **Arguments:** * ``: The target model variation URL suffix for the new version (format: `owner/model-slug/framework/variation-slug`, e.g., `$KAGGLE_DEVELOPER/test-model/jax/main`). **Options:** * `-p, --path `: Path to the folder containing the files for this new version (defaults to the current directory). * `-n, --version-notes `: Notes describing this version. * `-q, --quiet`: Suppress verbose output. * `-r, --dir-mode `: How to handle directories within the upload: `skip` (ignore), `zip` (compressed upload), `tar` (uncompressed upload) (default: `skip`). **Example:** Create a new version for the model variation `$KAGGLE_DEVELOPER/test-model/jax/main` using files from the `tmp` folder, with version notes "Updated model files", quietly, and skipping subdirectories: ```bash # Ensure tmp folder contains the new files for the version, e.g., data_v2.csv # echo "e,f,g,h" > tmp/data_v2.csv kaggle models variations versions create $KAGGLE_DEVELOPER/test-model/jax/main -p tmp -n "Updated model files" -q -r skip ``` **Purpose:** This command uploads a new set of files to an existing model variation, creating a new, numbered version. This allows you to track changes and revert to previous versions of your model variation files. ## `kaggle models variations versions download` Downloads files for a specific version of a model variation. **Usage:** ```bash kaggle models variations versions download [options] ``` **Arguments:** * ``: Model variation version URL suffix in the format `owner/model-slug/framework/variation-slug/version-number` (e.g., `$KAGGLE_DEVELOPER/test-model/jax/main/1`). **Options:** * `-p, --path `: Folder to download files to (defaults to current directory). * `--untar`: Untar the downloaded file if it's a `.tar` archive (deletes the `.tar` file afterwards). * `--unzip`: Unzip the downloaded file if it's a `.zip` archive (deletes the `.zip` file afterwards). * `-f, --force`: Force download, overwriting existing files. * `-q, --quiet`: Suppress verbose output. **Example:** Download version 1 of the model variation `$KAGGLE_DEVELOPER/test-model/jax/main` into the `tmp` folder, untar if applicable, force overwrite, and do it quietly: ```bash kaggle models variations versions download $KAGGLE_DEVELOPER/test-model/jax/main/1 -p tmp -q -f --untar ``` **Purpose:** This command allows you to retrieve the specific files associated with a particular version of a model variation. ## `kaggle models variations versions files` Lists files for a specific version of a model variation. **Usage:** ```bash kaggle models variations versions files [options] ``` **Arguments:** * ``: Model variation version URL suffix (e.g., `google/gemma/pytorch/7b/2`). **Options:** * `-v, --csv`: Print results in CSV format. * `--page-size `: Number of items per page (default: 20). * `--page-token `: Page token for results paging. **Example:** List the first 3 files for version 2 of the model variation `google/gemma/pytorch/7b` in CSV format: ```bash kaggle models variations versions files google/gemma/pytorch/7b/2 -v --page-size=3 ``` **Purpose:** Use this command to see the individual files that constitute a specific version of a model variation before downloading. ## `kaggle models variations versions delete` Deletes a specific version of a model variation from Kaggle. **Usage:** ```bash kaggle models variations versions delete [options] ``` **Arguments:** * ``: Model variation version URL suffix in the format `owner/model-slug/framework/variation-slug/version-number` (e.g., `$KAGGLE_DEVELOPER/test-model/jax/main/1`). **Options:** * `-y, --yes`: Automatically confirm deletion without prompting. **Example:** Delete version 1 of the model variation `$KAGGLE_DEVELOPER/test-model/jax/main` and automatically confirm: ```bash kaggle models variations versions delete $KAGGLE_DEVELOPER/test-model/jax/main/1 -y ``` **Purpose:** This command permanently removes a specific version of your model variation from Kaggle. Use with caution. If it's the only version, this may lead to the deletion of the model variation itself if no other versions exist. ================================================ FILE: docs/models.md ================================================ # Models Commands Commands for interacting with Kaggle Models. ## `kaggle models list` Lists available models. **Usage:** ```bash kaggle models list [options] ``` **Options:** * `--owner `: Filter by a specific user or organization. * `--sort-by `: Sort results. Valid options: `hotness`, `downloadCount`, `voteCount`, `notebookCount`, `createTime` (default: `hotness`). * `-s, --search `: Search term. * `--page-size `: Number of items per page (default: 20). * `--page-token `: Page token for results paging. * `-v, --csv`: Print results in CSV format. **Examples:** 1. List models owned by `$KAGGLE_DEVELOPER` (replace with your username), sorted by creation time, in CSV format: ```bash kaggle models list --owner $KAGGLE_DEVELOPER --sort-by createTime -v ``` 2. List the first 5 models matching the search term "gemini": ```bash kaggle models list -s gemini --page-size 5 ``` **Purpose:** This command helps you find models on Kaggle, filtering by owner or searching by keywords, and sorting by various criteria. ## `kaggle models init` Initializes a metadata file (`model-metadata.json`) for creating a new model. See [metadata file format](./models_metadata.md). **Usage:** ```bash kaggle models init -p ``` **Options:** * `-p, --path `: The path to the folder where the `model-metadata.json` file will be created (defaults to the current directory). **Example:** Initialize a model metadata file in a new temporary folder `tmp`: ```bash mkdir tmp kaggle models init -p tmp ``` **Purpose:** This command creates a template `model-metadata.json` file. You must edit this file with your model's details, such as owner slug, title, model slug (URL-friendly version of the title), and a description, before creating the model on Kaggle. ## `kaggle models create` Creates a new model on Kaggle. **Usage:** ```bash kaggle models create -p ``` **Options:** * `-p, --path `: Path to the folder containing the `model-metadata.json` file (defaults to the current directory). This folder should also contain your model files that you intend to upload as part of the first model variation. **Example:** Create a new model using the metadata in `tmp/model-metadata.json`. (Assumes the metadata file has been edited with owner, title, and slug): ```bash # Example: Edit model-metadata.json first # sed -i 's/INSERT_OWNER_SLUG_HERE/your-username/' tmp/model-metadata.json # sed -i 's/INSERT_TITLE_HERE/My Awesome Model/' tmp/model-metadata.json # sed -i 's/INSERT_SLUG_HERE/my-awesome-model/' tmp/model-metadata.json kaggle models create -p tmp ``` **Purpose:** This command registers a new model on Kaggle using the provided metadata. After this, you will typically create model variations and versions. ## `kaggle models get` Downloads the `model-metadata.json` file for an existing model. **Usage:** ```bash kaggle models get -p ``` **Arguments:** * ``: Model URL suffix in the format `owner/model-slug` (e.g., `$KAGGLE_DEVELOPER/test-model`). **Options:** * `-p, --path `: Folder to download the `model-metadata.json` file to. **Example:** Download the metadata for model `$KAGGLE_DEVELOPER/test-model` into the `tmp` folder: ```bash kaggle models get -p tmp $KAGGLE_DEVELOPER/test-model ``` **Purpose:** This command retrieves the metadata file for an existing model, which can be useful for inspection or as a basis for an update. ## `kaggle models update` Updates an existing model on Kaggle using a local `model-metadata.json` file. **Usage:** ```bash kaggle models update -p ``` **Options:** * `-p, --path `: Path to the folder containing the `model-metadata.json` file with the updated information (defaults to the current directory). **Example:** Update the model whose details are in `tmp/model-metadata.json` (ensure the slug and owner in the JSON match an existing model): ```bash kaggle models update -p tmp ``` **Purpose:** Use this command to change the metadata of an existing model, such as its title, description, or other fields defined in the `model-metadata.json` file. ## `kaggle models delete` Deletes a model from Kaggle. **Usage:** ```bash kaggle models delete [options] ``` **Arguments:** * ``: Model URL suffix in the format `owner/model-slug` (e.g., `$KAGGLE_DEVELOPER/test-model`). **Options:** * `-y, --yes`: Automatically confirm deletion without prompting. **Example:** Delete the model `$KAGGLE_DEVELOPER/test-model` and automatically confirm: ```bash kaggle models delete $KAGGLE_DEVELOPER/test-model -y ``` **Purpose:** This command permanently removes one of your models (and all its variations and versions) from Kaggle. Use with caution. ================================================ FILE: docs/models_metadata.md ================================================ A full model is composed of 3 types of entities: 1. The model 2. The variations 3. The variation versions Let's take the example of [efficientnet](https://www.kaggle.com/models/tensorflow/efficientnet) to explain these entities. A model like `efficientnet` contains multiple variations. A variation is a specific variation of the model (e.g. B0, B1, ...) with a certain framework (e.g. TensorFlow2). ## Model To create a model, a special `model-metadata.json` file must be specified. Here's a basic example for `model-metadata.json`: ``` { "ownerSlug": "INSERT_OWNER_SLUG_HERE", "title": "INSERT_TITLE_HERE", "slug": "INSERT_SLUG_HERE", "subtitle": "", "isPrivate": true, "description": "Model Card Markdown, see below", "publishTime": "", "provenanceSources": "" } ``` You can also use the API command `kaggle models init -p /path/to/model` to have the API create this file for you for a new model. If you wish to get the metadata for an existing model, you can use `kaggle models get username/model-slug`. ### Contents We currently support the following metadata fields for models. * `ownerSlug`: the slug of the user or organization * `title`: the model's title * `slug`: the model's slug (unique per owner) * `licenseName`: the name of the license (see the list below) * `subtitle`: the model's subtitle * `isPrivate`: whether or not the model should be private (only visible by the owners). If not specified, will be `true` * `description`: the model's card in markdown syntax (see the template below) * `publishTime`: the original publishing time of the model * `provenanceSources`: the provenance of the model ## Model Variation To create a model variation, a special `model-instance-metadata.json` file must be specified. Here's a basic example for `model-instance-metadata.json`: ``` { "ownerSlug": "INSERT_OWNER_SLUG_HERE", "modelSlug": "INSERT_EXISTING_MODEL_SLUG_HERE", "instanceSlug": "INSERT_INSTANCE_SLUG_HERE", "framework": "INSERT_FRAMEWORK_HERE", "overview": "", "usage": "Usage Markdown, see below", "licenseName": "Apache 2.0", "fineTunable": False, "trainingData": [], "modelInstanceType": "Unspecified", "baseModelInstance": "", "externalBaseModelUrl": "" } ``` You can also use the API command `kaggle models variations init -p /path/to/model-variation` to have the API create this file for you for a new model variation. ### Contents We currently support the following metadata fields for model variations. * `ownerSlug`: the slug of the user or organization of the model * `modelSlug`: the existing model's slug * `instanceSlug`: the slug of the variation * `framework`: the variation's framework (possible options: `tensorFlow1`,`tensorFlow2`,`tfLite`,`tfJs`,`pyTorch`,`jax`,`coral`, ...) * `overview`: a short overview of the variation * `usage`: the variation's usage in markdown syntax (see the template below) * `fineTunable`: whether the variation is fine tunable * `trainingData`: a list of training data in the form of strings, URLs, Kaggle Datasets, etc... * `modelInstanceType`: whether the model variation is a base model, external variant, internal variant, or unspecified * `baseModelInstance`: if this is an internal variant, the `{owner-slug}/{model-slug}/{framework}/{variation-slug}` of the base model variation * `externalBaseModelUrl`: if this is an external variant, a URL to the base model ### Licenses Here is a list of the available licenses for models: - Apache 2.0 - Attribution 3.0 IGO (CC BY 3.0 IGO) - Attribution 3.0 Unported (CC BY 3.0) - Attribution 4.0 International (CC BY 4.0) - Attribution-NoDerivatives 4.0 International (CC BY-ND 4.0) - Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) - Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) - Attribution-NonCommercial-ShareAlike 3.0 IGO (CC BY-NC-SA 3.0 IGO) - BSD-3-Clause - CC BY-NC-SA 4.0 - CC BY-SA 3.0 - CC BY-SA 4.0 - CC0: Public Domain - Community Data License Agreement - Permissive - Version 1.0 - Community Data License Agreement - Sharing - Version 1.0 - GNU Affero General Public License 3.0 - GNU Free Documentation License 1.3 - GNU Lesser General Public License 3.0 - GPL 2 - MIT - ODC Attribution License (ODC-By) - ODC Public Domain Dedication and Licence (PDDL) - GPL 3 ### Usage The following template variables can be used in this markdown: - `${VERSION_NUMBER}` is replaced by the version number when rendered - `${VARIATION_SLUG}` is replaced by the variation slug when rendered - `${FRAMEWORK}` is replaced by the framework name - `${PATH}` is replaced by `/kaggle/input////`. - `${FILEPATH}` is replaced by `/kaggle/input/////`. This value is only defined if the databundle contain a single file - `${URL}` is replaced by the absolute URL of the model ================================================ FILE: docs/tutorials.md ================================================ # Kaggle CLI Tutorials These tutorials illustrate how to use a sequence of Kaggle CLI commands to accomplish common tasks. ## Introduction Before starting these tutorials, please make sure you have: 1. Installed the Kaggle CLI, following the instructions [here](./README.md#installation). 2. Set up your API credentials, following the instructions [here](./README.md#authentication) 3. Logged in to Kaggle in a web browser. This will allow you to verify the results of the CLI commands in the [`Your Work`](https://www.kaggle.com/work) section of your Kaggle profile. ## Tutorial: Create a Dataset This tutorial walks you through creating a new dataset on Kaggle. 1. **Start from an empty directory.** Create a new directory for your dataset files and navigate into it. ```bash mkdir my-new-dataset cd my-new-dataset ``` 2. **Create a sample data file.** For this example, create a CSV file named `sample_data.csv` with an index column and three random data columns, and a few rows of data. ```bash echo "id,col_a,col_b,col_c" > sample_data.csv echo "1,0.5,0.2,0.8" >> sample_data.csv echo "2,0.1,0.7,0.3" >> sample_data.csv echo "3,0.9,0.4,0.6" >> sample_data.csv ``` 3. **Initialize dataset metadata.** This creates a `dataset-metadata.json` file in your current directory. ```bash kaggle datasets init ``` 4. **Edit the metadata file.** Open `dataset-metadata.json` in a text editor and make the following changes: * Replace `"INSERT_TITLE_HERE"` with your desired dataset title, e.g., `"My Sample Dataset"`. * Replace `"INSERT_SLUG_HERE"` with a URL-friendly version of your title, e.g., `"my-sample-dataset"`. The URL-friendly version is made by converting the title to lower-case and changing spaces to dashes. * You can also add licenses, descriptions, and other relevant information. 5. **Create the dataset.** This command uploads your `sample_data.csv` and `dataset-metadata.json` to Kaggle. ```bash kaggle datasets create -p . ``` You can add `--public` to make it public immediately. 6. **Verify on Kaggle.com.** Refresh the [`Datasets` tab in `Your Work`](https://www.kaggle.com/work/datasets). You should see "My Sample Dataset". ## Tutorial: Find and Download a Dataset This tutorial explains how to find and download using the CLI. 1. **Search for a Dataset (Optional).** * If you know the dataset you want, you can skip this step. Otherwise, you can search for datasets. For example, to search for datasets related to "iris": ```bash kaggle datasets list -s iris ``` * This command will list datasets matching your search query. Note the dataset's "id" (e.g., `uciml/iris`) which you'll use for downloading. 2. **Choose a Dataset and Create a Directory.** * For this tutorial, we'll use the classic "Iris" dataset, which has the id `uciml/iris`. * Create a new directory for your dataset and navigate into it: ```bash mkdir iris-dataset-analysis cd iris-dataset-analysis ``` 3. **Download the Dataset.** * Use the `kaggle datasets download` command with the dataset's id. ```bash kaggle datasets download -d uciml/iris ``` * This will download the dataset files, typically as a ZIP archive (e.g., `iris.zip`), into your current directory (`iris-dataset-analysis`). 4. **Unzip the Dataset.** * Note: you could skip this step by using the `--unzip` flag on the previous command. * Most datasets are downloaded as ZIP files. You'll need to unzip the archive to access the data files (e.g., CSV files). ```bash # Make sure you have unzip installed, or use your OS's GUI to extract # The actual zip file name might vary based on the dataset. # For uciml/iris, it's iris.zip unzip iris.zip ``` 5. **Verify the results.** * After unzipping, you should see the data files (e.g., `Iris.csv`, `database.sqlite`). ## Tutorial: Update a Kernel (Notebook) This tutorial shows how to download an existing kernel, modify it, and push the changes back to Kaggle. 1. **Create or identify a kernel on Kaggle.com.** * Log in to kaggle.com. * Find an existing notebook you own (or create one). For this tutorial, let's assume its title is "My CLI Test Kernel". * Note the kernel slug from the browser's address bar. It will be something like `YOUR_USERNAME/my-cli-test-kernel`. 2. **Create a new local directory for your kernel.** ```bash mkdir my-kernel-project cd my-kernel-project ``` 3. **Pull the kernel.** Use the `kaggle kernels pull` command with your username and the kernel slug. The `-m` flag includes the `kernel-metadata.json` file, which is required for pushing updates. ```bash # Replace YOUR_USERNAME with your actual Kaggle username kaggle kernels pull YOUR_USERNAME/my-cli-test-kernel -m ``` This will download `my-cli-test-kernel.ipynb` (or `.py`/`.Rmd`) and `kernel-metadata.json`. 4. **Edit the kernel or metadata.** * Open the downloaded notebook file (e.g., `my-cli-test-kernel.ipynb`) and make some changes to the code or content. * Open `kernel-metadata.json`. Let's add "benchmark" to the keywords. Find the `"keywords": []` line and change it to `"keywords": ["benchmark"]`. * *Note: While you can edit keywords here, it's often best to manage them on kaggle.com, as there is a restricted list of allowed keywords.* 5. **Push the kernel.** This uploads your changes and the updated metadata, then runs the kernel on Kaggle. ```bash kaggle kernels push -p . ``` 6. **Verify on Kaggle.com.** Refresh the [`Code` tab in `Your Work`](https://www.kaggle.com/work/code). You should see your code changes and the "benchmark" tag added to the kernel settings. ## Tutorial: Create a Model This tutorial guides you through creating a new model on Kaggle. 1. **Start from an empty directory.** Create a new directory for your model files and navigate into it. ```bash mkdir my-new-model cd my-new-model ``` 2. **Copy your model definition files (optional for this step).** If you have files that define your model (e.g., Python scripts, model weights), copy them into this directory. For the `kaggle models create` step, only the metadata is strictly required, but you'll need files when you create a model variation. 3. **Initialize model metadata.** This creates a `model-metadata.json` file. ```bash kaggle models init ``` 4. **Edit the metadata file.** Open `model-metadata.json` and make the following changes: * Replace `"INSERT_OWNER_SLUG_HERE"` with your Kaggle username (e.g., `"YOUR_USERNAME"`). * Replace `"INSERT_TITLE_HERE"` with your model's title (e.g., `"My Awesome AI Model"`). * Replace `"INSERT_SLUG_HERE"` with a URL-friendly version of the title (e.g., `"my-awesome-ai-model"`). * Fill out the `"description"` field and other relevant sections like `"licenses"`. 5. **Create the model.** ```bash kaggle models create -p . ``` 6. **Verify on Kaggle.com.** Refresh the [`Models` tab in `Your Work`](https://www.kaggle.com/work/models). You should see "My Awesome AI Model". ## Tutorial: Create a Model Variation This tutorial shows how to create a variation under an existing model. A model variation usually represents the model implemented in a specific framework (like TensorFlow, PyTorch, JAX, etc.) and includes the actual model files. 1. **Ensure you have a parent model.** Follow the "Create a Model" tutorial if you haven't already. Let's assume your model slug is `my-awesome-ai-model` and your username is `YOUR_USERNAME`. 2. **Prepare your model variation files.** In your model directory (e.g., `my-new-model`), create or place the files for this specific variation. For example, a JAX model might have a `flax_model.params` file. ```bash # In the my-new-model directory echo "This is a placeholder for JAX model parameters" > flax_model.params ``` 3. **Initialize model variation metadata.** This creates `model-instance-metadata.json`. ```bash # Still in the my-new-model directory kaggle models variations init ``` 4. **Edit the variation metadata file.** Open `model-instance-metadata.json` and make changes: * Replace `"INSERT_OWNER_SLUG_HERE"` with your Kaggle username (e.g., `"YOUR_USERNAME"`). * Replace `"INSERT_EXISTING_MODEL_SLUG_HERE"` with your parent model's slug (e.g., `"my-awesome-ai-model"`). * Replace `"INSERT_INSTANCE_SLUG_HERE"` with a slug for this variation (e.g., `"jax-implementation"`). * Replace `"INSERT_FRAMEWORK_HERE"` with the model framework (e.g., `"jax"`, `"tensorflow"`, `"pytorch"`, `"sklearn"`). * Update the `"instance_size_bytes"` if known, and add a `"description"`. 5. **Create the model variation.** This uploads the files in the current directory (e.g., `flax_model.params`) along with the variation metadata. ```bash kaggle models variations create -p . ``` 6. **Verify on Kaggle.com.** Go to your model's page on Kaggle by clicking on the model under in the [`Models` tab on `Your Work`](https://www.kaggle.com/work/models). You should see a new "jax-implementation" variation listed, and it will have one version containing `flax_model.params`. ## Tutorial: Create a Model Variation Version This tutorial explains how to add a new version to an existing model variation, for example, when you have updated model weights or files. 1. **Ensure you have a model variation.** Follow the "Create a Model Variation" tutorial. Let's assume your variation is `YOUR_USERNAME/my-awesome-ai-model/jax/jax-implementation`. 2. **Prepare your updated files.** In your model variation directory (e.g., `my-new-model`), update or add new files for this version. For example, create `flax_model_v2.params`. ```bash # In the my-new-model directory echo "Updated JAX model parameters for V2" > flax_model_v2.params # You might also remove or update flax_model.params if it's being replaced ``` 3. **Create the new model variation version.** You need to specify the parent model variation and provide version notes. The files from the `-p` path will form the contents of this new version. ```bash # Replace YOUR_USERNAME and the slugs for model and variation accordingly kaggle models variations versions create YOUR_USERNAME/my-awesome-ai-model/jax/jax-implementation -p . -n "Second version with updated parameters" ``` *Note: The `-p .` means all files in the current directory will be uploaded as part of this new version. If you only want to upload `flax_model_v2.params`, ensure only it (and any other V2 files) are in a directory and point `-p` to that directory, or manage your files carefully.* 4. **Verify on Kaggle.com.** Go to your model variation page on Kaggle (e.g., `YOUR_USERNAME/my-awesome-ai-model/jax/jax-implementation`) by clicking on the [`Models` tab on `Your Work`](https://www.kaggle.com/work/models). You should see a new version (e.g., version 2) listed with your notes and the new files. ## Tutorial: How to Submit to a Competition This tutorial walks you through the process of making a submission to a Kaggle competition using the CLI. 1. **Find a Competition and Accept Rules.** * First, you need to find a competition. You can list active competitions using `kaggle competitions list`. * For this tutorial, we'll use the "titanic" competition, which is a common starting point. You can find it at [`https://www.kaggle.com/c/titanic`](https://www.kaggle.com/c/titanic). * **Important**: Before you can download data or submit, you *must* join the competition and accept the competition's rules on the Kaggle website. Navigate to the competition on kaggle.com to do this. 2. **Create a Directory and Download Competition Files.** * Create a new directory for your competition files and navigate into it. ```bash mkdir titanic-competition cd titanic-competition ``` * Download the competition files. This usually includes training data, test data, and a sample submission file. ```bash kaggle competitions download -c titanic ``` * This will download `titanic.zip`. You'll need to unzip it to see the files (e.g., `train.csv`, `test.csv`, `gender_submission.csv`). ```bash # Make sure you have unzip installed, or use your OS's GUI to extract # The actual zip file name might vary based on the competition. unzip titanic.zip ``` 3. **Create Your Submission File.** * The required format for the submission file is specific to each competition. You can find this information on the competition's "Evaluation" page or by examining the sample submission file (e.g., `gender_submission.csv` for the Titanic competition). * For the Titanic competition, the submission file needs two columns: `PassengerId` and `Survived`. The `Survived` column should contain your predictions (0 for deceased, 1 for survived). * Let's create a very simple submission file based on the `gender_submission.csv` (which predicts survival based on gender). For this tutorial, we'll just copy it and use it as our submission. In a real scenario, you would generate this file from your model's predictions on the `test.csv` data. ```bash cp gender_submission.csv my_submission.csv ``` * Your `my_submission.csv` should look something like this: ``` PassengerId,Survived 892,0 893,1 894,0 ... ``` 4. **Submit to the Competition.** * Use the `kaggle competitions submit` command. You need to specify: * The competition ID (`titanic`). * The path to your submission file (`-f my_submission.csv`). * A message describing your submission (`-m "My first submission via CLI"`). ```bash kaggle competitions submit titanic -f my_submission.csv -m "My first submission via CLI" ``` 5. **Check Your Submission Status.** * After submitting, you'll get a message indicating success or failure. * You can check your submission's score and status on the "My Submissions" tab of the competition page on Kaggle.com (e.g., [`https://www.kaggle.com/c/titanic/submissions`](https://www.kaggle.com/c/titanic/submissions)). * You can also list your recent submissions and their scores via the CLI: ```bash kaggle competitions submissions -c titanic ``` * This command will show your submission, its status (e.g., `complete`, `error`), and your public/private scores if available. ## Tutorial: How to Submit to a Code Competition This tutorial walks you through the process of submitting to a code competition on Kaggle. 1. **Find a Code Competition.** * First, you need to find a code competition to participate in. You can browse the available competitions on the [Kaggle competitions page](https://www.kaggle.com/competitions). Many Featured Competitions are code competitions. 2. **Download the Dataset.** * Once you have chosen a competition, you need to download the dataset. You can do this using the `kaggle competitions download` command: ```bash kaggle competitions download -c ``` * Replace `` with the name of the competition you want to participate in. 3. **Create a Notebook.** * Next, you need to create a Kaggle Notebook to work on your submission. A Kaggle Notebook contains the code and environment settings for Kaggle to run and evaluate your submission. Follow the tutorial on [Creating / Updating Notebooks](#tutorial-update-a-kernel-notebook) if you're not sure how to do this. 4. **Write Your Code.** * Now it's time to write your code! You can use any programming language or framework that is supported by Kaggle. The goal is to create a model that can make predictions on the test set. 5. **Submit Your Prediction.** * Once you are happy with your model, you can submit your prediction to the competition. You can do this using the `kaggle competitions submit` command: ```bash kaggle competitions submit -k / -f -v -m ``` * Replace: * `` with the name of the competition * `/` with the identifier of your notebook * `` with the name of the submission file produced by your notebook (e.g. `submission.csv`). * `` with the version to submit (e.g. `3` to submit the 3rd version of your notebook). * `` with a brief description of your submission. 6. **Check Your Score.** * After you have submitted your prediction, you can check your score on the competition leaderboard. The leaderboard shows the scores of all the participants in the competition. You can download the leaderboard using the `kaggle competitions leaderboard` command: ```bash kaggle competitions leaderboard ``` ================================================ FILE: integration_tests/__init__.py ================================================ ================================================ FILE: integration_tests/test_models.py ================================================ import unittest from kaggle.api.kaggle_api_extended import KaggleApi MODEL_HANDLE = "keras/bert" MODEL_ID = 2819 # TODO(messick) Add a test that creates a dataset w/o specifying privacy that is created private. class TestModels(unittest.TestCase): def setUp(self): self.api = KaggleApi() self.api.authenticate() def test_list_models(self) -> None: models = self.api.model_list() self.assertGreater(len(models), 0) def test_get_model(self) -> None: model = self.api.model_get(MODEL_HANDLE) self.assertEqual(MODEL_ID, model.id) ================================================ FILE: pyproject.toml ================================================ [build-system] requires = [ "hatchling", ] build-backend = "hatchling.build" [project] name = "kaggle" dynamic = [ "version", ] description = "Access Kaggle resources anywhere" authors = [ { name = "Kaggle", email = "support@kaggle.com" }, ] license = { file = "LICENSE.txt" } readme = "README.md" classifiers = [ "Programming Language :: Python :: 3", "License :: OSI Approved :: Apache Software License", "Operating System :: OS Independent", ] keywords = ["Kaggle", "API"] requires-python = ">= 3.11" dependencies = [ "bleach", "kagglesdk >= 0.1.16, < 1.0", # sync with kagglehub "python-slugify", "requests", "python-dateutil", "tqdm", "urllib3 >= 1.15.1", "packaging", "protobuf", ] [project.scripts] kaggle = "kaggle.cli:main" [project.urls] Homepage = "https://github.com/Kaggle/kaggle-cli" Issues = "https://github.com/Kaggle/kaggle-cli/issues" [tool.hatch.version] path = "src/kaggle/__init__.py" [tool.hatch.build.targets.wheel] packages = ["src/kaggle"] [tool.hatch.envs.test] dependencies = [ "pytest" ] [tool.hatch.envs.test.scripts] local = "source tools/use-localhost.sh; cd tests; python unit_tests.py --failfast" prod = "source tools/use-prod.sh; cd tests; python unit_tests.py --failfast" integration = "pytest integration_tests" [tool.hatch.envs.lint] detached = true dependencies = [ "black >= 24.10.0", "mypy >= 1.15.0", # Pre-install `types-*` packages to speed up lint:typing command. "types-requests", "types-tqdm", ] [tool.hatch.envs.lint.scripts] typing = "mypy --install-types --non-interactive {args:src/kagglehub tests}" style = [ "black --check --diff {args:.}", ] fmt = [ "black {args:.}", "style", ] all = [ "style", "typing", ] [tool.docformatter] recursive = true [tool.black] target-version = ["py311"] line-length = 120 [tool.mypy] strict = false show_error_codes = true follow_imports = 'silent' exclude = '''(?x)( /src/ )''' python_version = 3.11 # Start off with these warn_unused_configs = true warn_redundant_casts = true warn_unused_ignores = true # Getting these passing should be easy strict_equality = true extra_checks = true # Strongly recommend enabling this one as soon as you can check_untyped_defs = true # These shouldn't be too much additional work, but may be tricky to # get passing if you use a lot of untyped libraries disallow_subclassing_any = true disallow_untyped_decorators = true disallow_any_generics = true # These next few are various gradations of forcing use of type annotations disallow_incomplete_defs = true # TODO Enable these after GA. #disallow_untyped_calls = true # 167 errors reported a/o April 30, 2025 #disallow_untyped_defs = true # 132 # This one isn't too hard to get passing, but return on investment is lower no_implicit_reexport = true # 50 # This one can be tricky to get passing if you use a lot of untyped libraries warn_return_any = true ================================================ FILE: tests/__init__.py ================================================ ================================================ FILE: tests/dataset/data.csv ================================================ id, fruit 1, apple 2, banana 3, citrus 4, apple 5, durian ================================================ FILE: tests/kernel/testing-x.ipynb ================================================ {"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[],"dockerImageVersionId":30646,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2024-06-11T17:18:41.771461Z","iopub.execute_input":"2024-06-11T17:18:41.771892Z","iopub.status.idle":"2024-06-11T17:18:43.04124Z","shell.execute_reply.started":"2024-06-11T17:18:41.771858Z","shell.execute_reply":"2024-06-11T17:18:43.03999Z"},"trusted":true},"execution_count":null,"outputs":[]}]} ================================================ FILE: tests/model/instance/data.csv ================================================ id, fruit 1, apple 2, banana 3, citrus 4, apple 5, durian ================================================ FILE: tests/model/instance/version/metadata.json ================================================ {} ================================================ FILE: tests/sample_submission.csv ================================================ Id,SalePrice 1461,169277.0524984 1462,187758.393988768 1463,183583.683569555 1464,179317.47751083 1465,150730.079976501 1466,177150.989247307 1467,172070.659229164 1468,175110.956519547 1469,162011.698831665 1470,160726.247831419 1471,157933.279456005 1472,145291.245020389 1473,159672.017631819 1474,164167.518301885 1475,150891.638244053 1476,179460.96518734 1477,185034.62891405 1478,182352.192644656 1479,183053.458213802 1480,187823.339254278 1481,186544.114327568 1482,158230.77520516 1483,190552.829321091 1484,147183.67487199 1485,185855.300905493 1486,174350.470676986 1487,201740.620690863 1488,162986.378895754 1489,162330.199085679 1490,165845.938616539 1491,180929.622876974 1492,163481.501519718 1493,187798.076714233 1494,198822.198942566 1495,194868.409899858 1496,152605.298564403 1497,147797.702836811 1498,150521.96899297 1499,146991.630153739 1500,150306.307814534 1501,151164.372534604 1502,151133.706960953 1503,156214.042540726 1504,171992.760735142 1505,173214.912549738 1506,192429.187345783 1507,190878.69508543 1508,194542.544135519 1509,191849.439072822 1510,176363.773907793 1511,176954.185412429 1512,176521.216975696 1513,179436.704810176 1514,220079.756777048 1515,175502.918109444 1516,188321.073833569 1517,163276.324450004 1518,185911.366293097 1519,171392.830997252 1520,174418.207020775 1521,179682.709603774 1522,179423.751581665 1523,171756.918091777 1524,166849.638174419 1525,181122.168676666 1526,170934.462746566 1527,159738.292580329 1528,174445.759557658 1529,174706.363659627 1530,164507.672539365 1531,163602.512172832 1532,154126.270249525 1533,171104.853481351 1534,167735.39270528 1535,183003.613338104 1536,172580.381161499 1537,165407.889104689 1538,176363.773907793 1539,175182.950898522 1540,190757.177789246 1541,167186.995771991 1542,167839.376779276 1543,173912.421165137 1544,154034.917445551 1545,156002.955794336 1546,168173.94329857 1547,168882.437104132 1548,168173.94329857 1549,157580.177551642 1550,181922.15256011 1551,155134.227842592 1552,188885.573319552 1553,183963.193012381 1554,161298.762306335 1555,188613.66763056 1556,175080.111822945 1557,174744.400305232 1558,168175.911336919 1559,182333.472575006 1560,158307.206742274 1561,193053.055502348 1562,175031.089987177 1563,160713.294602908 1564,173186.215014436 1565,191736.7598055 1566,170401.630997116 1567,164626.577880222 1568,205469.409444832 1569,209561.784211885 1570,182271.503072356 1571,178081.549427793 1572,178425.956138831 1573,162015.318511503 1574,181722.420373045 1575,156705.730169433 1576,182902.420342386 1577,157574.595395085 1578,184380.739100813 1579,169364.469225677 1580,175846.179822063 1581,189673.295302136 1582,174401.317715566 1583,179021.448718583 1584,189196.845337149 1585,139647.095720655 1586,161468.198288911 1587,171557.32317862 1588,179447.36804185 1589,169611.619017694 1590,172088.872655744 1591,171190.624128768 1592,154850.508361878 1593,158617.655719941 1594,209258.33693701 1595,177939.027626751 1596,194631.100299584 1597,213618.871562568 1598,198342.504228533 1599,138607.971472497 1600,150778.958976731 1601,146966.230339786 1602,162182.59620952 1603,176825.940961269 1604,152799.812402444 1605,180322.322067129 1606,177508.027228367 1607,208029.642652019 1608,181987.282510201 1609,160172.72797397 1610,176761.317654248 1611,176515.497545231 1612,176270.453065471 1613,183050.846258475 1614,150011.102062216 1615,159270.537808667 1616,163419.663729346 1617,163399.983345859 1618,173364.161505756 1619,169556.835902417 1620,183690.595995738 1621,176980.914909382 1622,204773.36222471 1623,174728.655998442 1624,181873.458244461 1625,177322.000823979 1626,193927.939041863 1627,181715.622732304 1628,199270.841200324 1629,177109.589956218 1630,153909.578271486 1631,162931.203336223 1632,166386.7567182 1633,173719.30379824 1634,179757.925656704 1635,179007.601964376 1636,180370.808623106 1637,185102.616730563 1638,198825.563452058 1639,184294.576009142 1640,200443.7920562 1641,181294.784484153 1642,174354.336267919 1643,172023.677781517 1644,181666.922855025 1645,179024.491269586 1646,178324.191575907 1647,184534.676687694 1648,159397.250378784 1649,178430.966728182 1650,177743.799385967 1651,179395.305519087 1652,151713.38474815 1653,151713.38474815 1654,168434.977996215 1655,153999.100311019 1656,164096.097354123 1657,166335.403036551 1658,163020.725375757 1659,155862.510668829 1660,182760.651095509 1661,201912.270622883 1662,185988.233987516 1663,183778.44888032 1664,170935.85921771 1665,184468.908382254 1666,191569.089663229 1667,232991.025583822 1668,180980.721388278 1669,164279.13048219 1670,183859.460411109 1671,185922.465682076 1672,191742.778119363 1673,199954.072465842 1674,180690.274752587 1675,163099.3096358 1676,140791.922472443 1677,166481.86647592 1678,172080.434496773 1679,191719.161659178 1680,160741.098612515 1681,157829.546854733 1682,196896.748596341 1683,159675.423990355 1684,182084.790901946 1685,179233.926374487 1686,155774.270901623 1687,181354.326716058 1688,179605.563663918 1689,181609.34866147 1690,178221.531623281 1691,175559.920735795 1692,200328.822792041 1693,178630.060559899 1694,177174.535221728 1695,172515.687368714 1696,204032.992922943 1697,176023.232787689 1698,202202.073341595 1699,181734.480075862 1700,183982.158993126 1701,188007.94241481 1702,185922.966763517 1703,183978.544874918 1704,177199.618638821 1705,181878.647956764 1706,173622.088728263 1707,180728.168562655 1708,176477.026606328 1709,184282.266697609 1710,162062.47538448 1711,182550.070992189 1712,180987.949624695 1713,178173.79762147 1714,179980.635948606 1715,173257.637826205 1716,177271.291059307 1717,175338.355442312 1718,177548.140549508 1719,175969.91662932 1720,175011.481953462 1721,185199.372568143 1722,188514.050228937 1723,185080.145268797 1724,157304.402574096 1725,194260.859481297 1726,181262.329995106 1727,157003.292706732 1728,182924.499359899 1729,181902.586375439 1730,188985.371708134 1731,185290.904495068 1732,177304.425752748 1733,166274.900490809 1734,177807.420530107 1735,180330.624816201 1736,179069.112234629 1737,175943.371816948 1738,185199.050609653 1739,167350.910824524 1740,149315.311876449 1741,139010.847766793 1742,155412.151845447 1743,171308.313985441 1744,176220.543265638 1745,177643.434991809 1746,187222.653264601 1747,185635.132083154 1748,206492.534215854 1749,181681.021081956 1750,180500.198072685 1751,206486.17086841 1752,161334.301195429 1753,176156.558313965 1754,191642.223478994 1755,191945.808027777 1756,164146.306037354 1757,179883.057071096 1758,178071.137668844 1759,188241.637896875 1760,174559.656173171 1761,182347.363042264 1762,191507.251872857 1763,199751.865597358 1764,162106.416145131 1765,164575.982314367 1766,179176.352180931 1767,177327.403857584 1768,177818.083761781 1769,186965.204048443 1770,178762.742169197 1771,183322.866146283 1772,178903.295931891 1773,186570.129421778 1774,199144.242829024 1775,172154.713310956 1776,177444.019201603 1777,166200.938073485 1778,158995.770555632 1779,168273.282454755 1780,189680.453052788 1781,181681.021081956 1782,160277.142643643 1783,197318.54715833 1784,162228.935604196 1785,187340.455456083 1786,181065.347037275 1787,190233.609102705 1788,157929.594852031 1789,168557.001935469 1790,160805.584645628 1791,221648.391978216 1792,180539.88079815 1793,182105.616283853 1794,166380.852603154 1795,178942.155617426 1796,162804.747800461 1797,183077.684392615 1798,171728.4720292 1799,164786.741540638 1800,177427.267170302 1801,197318.54715833 1802,178658.114178223 1803,185437.320523764 1804,169759.652489529 1805,173986.635055186 1806,168607.664289468 1807,194138.519145183 1808,192502.440921994 1809,176746.969818601 1810,177604.891703134 1811,193283.746584832 1812,181627.061006609 1813,169071.62025834 1814,167398.006470987 1815,150106.505141704 1816,159650.304285848 1817,179471.23597476 1818,177109.589956218 1819,166558.113328453 1820,153796.714319583 1821,174520.152570658 1822,196297.95829524 1823,169100.681601175 1824,176911.319164431 1825,169234.6454828 1826,172386.297919134 1827,156031.904802362 1828,168202.892306596 1829,166505.984017547 1830,176507.37022149 1831,180116.752553161 1832,183072.740591406 1833,189595.964677698 1834,167523.919076265 1835,210817.775863413 1836,172942.930813351 1837,145286.278144089 1838,176468.653371492 1839,159040.069562187 1840,178518.204332507 1841,169163.980786825 1842,189786.685274579 1843,181246.728523853 1844,176349.927153587 1845,205266.631009142 1846,187397.993362224 1847,208943.427726113 1848,165014.532907657 1849,182492.037566236 1850,161718.71259042 1851,180084.118941162 1852,178534.950802179 1853,151217.259961305 1854,156342.717587562 1855,188511.443835239 1856,183570.337896789 1857,225810.160292177 1858,214217.401131694 1859,187665.64101603 1860,161157.177744039 1861,187643.992594193 1862,228156.372839158 1863,220449.534665317 1864,220522.352084222 1865,156647.763531624 1866,187388.833374873 1867,178640.723791573 1868,180847.216739049 1869,159505.170529478 1870,164305.538020654 1871,180181.19673723 1872,184602.734989972 1873,193440.372174434 1874,184199.788209911 1875,196241.892907637 1876,175588.618271096 1877,179503.046546829 1878,183658.076582555 1879,193700.976276404 1880,165399.62450704 1881,186847.944787446 1882,198127.73287817 1883,183320.898107934 1884,181613.606696657 1885,178298.791761954 1886,185733.534000593 1887,180008.188485489 1888,175127.59621604 1889,183467.176862723 1890,182705.546021743 1891,152324.943593181 1892,169878.515981342 1893,183735.975076576 1894,224118.280105941 1895,169355.202465146 1896,180054.276407441 1897,174081.601977368 1898,168494.985022146 1899,181871.598843299 1900,173554.489658383 1901,169805.382165577 1902,176192.990728755 1903,204264.39284654 1904,169630.906956928 1905,185724.838807268 1906,195699.036281861 1907,189494.276162169 1908,149607.905673439 1909,154650.199045978 1910,151579.558140433 1911,185147.380531144 1912,196314.53120359 1913,210802.395364155 1914,166271.2863726 1915,154865.359142973 1916,173575.5052865 1917,179399.563554274 1918,164280.776562049 1919,171247.48948121 1920,166878.587182445 1921,188129.459710994 1922,183517.34369691 1923,175522.026925727 1924,190060.105331152 1925,174179.824771856 1926,171059.523675194 1927,183004.186769318 1928,183601.647387418 1929,163539.327185998 1930,164677.676391525 1931,162395.073865424 1932,182207.6323195 1933,192223.939790304 1934,176391.829390125 1935,181913.179121348 1936,179136.097888261 1937,196595.568243212 1938,194822.365690957 1939,148356.669440918 1940,160387.604263899 1941,181276.500571809 1942,192474.817899346 1943,157699.907796437 1944,215785.540813051 1945,181824.300998793 1946,221813.00948166 1947,165281.292597397 1948,255629.49047034 1949,173154.590990955 1950,183884.65246539 1951,200210.353608489 1952,186599.221265342 1953,192718.532696106 1954,178628.665952764 1955,180650.342418406 1956,206003.107947263 1957,166457.67844853 1958,202916.221653487 1959,192463.969983091 1960,171775.497189898 1961,175249.222149411 1962,147086.59893993 1963,149709.672100371 1964,171411.404533743 1965,178188.964799425 1966,156491.711373235 1967,180953.241201168 1968,203909.759061135 1969,175470.149087545 1970,205578.333622415 1971,199428.857699441 1972,187599.163869476 1973,192265.198109864 1974,196666.554897677 1975,155537.862252682 1976,169543.240620935 1977,202487.010170501 1978,208232.716273485 1979,173621.195202569 1980,172414.608571812 1981,164400.75641556 1982,160480.424024781 1983,156060.853810389 1984,157437.192820581 1985,158163.720929772 1986,154849.043268978 1987,152186.609341561 1988,180340.215399228 1989,178344.62451356 1990,190170.382266827 1991,168092.975480832 1992,178757.912566805 1993,174518.256882082 1994,198168.490116289 1995,176882.693978902 1996,183801.672896251 1997,196400.046680661 1998,172281.605004025 1999,196380.366297173 2000,198228.354306682 2001,195556.581268962 2002,186453.264469043 2003,181869.381196234 2004,175610.840124147 2005,183438.730800145 2006,179584.488673295 2007,182386.152242034 2008,160750.367237054 2009,182477.505046008 2010,187720.359207171 2011,187201.942081511 2012,176385.102235149 2013,175901.787841278 2014,182584.280198283 2015,195664.686104237 2016,181420.346494222 2017,176676.04995228 2018,181594.678867334 2019,178521.747964951 2020,175895.883726231 2021,168468.005916477 2022,200973.129447888 2023,197030.641992202 2024,192867.417844592 2025,196449.247639381 2026,141684.196398607 2027,153353.334123901 2028,151143.549016705 2029,163753.087114229 2030,158682.460013921 2031,144959.835250915 2032,160144.390548579 2033,156286.534303521 2034,165726.707619571 2035,182427.481047359 2036,173310.56154032 2037,173310.56154032 2038,151556.01403002 2039,158908.146068683 2040,209834.383092536 2041,192410.516550815 2042,174026.247294886 2043,195499.830115336 2044,200918.018812493 2045,207243.616023976 2046,196149.783851876 2047,192097.914850217 2048,178570.948923671 2049,228617.968325428 2050,199929.884438451 2051,160206.365612859 2052,179854.431885567 2053,185987.340461822 2054,161122.505607926 2055,175949.342720138 2056,183683.590595324 2057,176401.34762338 2058,205832.532527897 2059,177799.799849436 2060,167565.362080406 2061,186348.958436557 2062,179782.759465081 2063,169837.623333323 2064,178817.275675758 2065,174444.479149339 2066,192834.968917174 2067,196564.717984981 2068,206977.567039357 2069,157054.253944128 2070,175142.948078577 2071,159932.1643654 2072,182801.408333628 2073,181510.375176825 2074,181613.035129451 2075,186920.512597635 2076,157950.170625222 2077,176115.159022876 2078,182744.514344465 2079,180660.683691591 2080,160775.629777099 2081,186711.715848082 2082,223581.758190888 2083,172330.943236652 2084,163474.633393212 2085,175308.263299874 2086,187462.725306432 2087,180655.101535034 2088,152121.98603454 2089,159856.233909727 2090,186559.854936737 2091,183962.550959411 2092,162107.168699296 2093,162582.288981283 2094,154407.701597409 2095,181625.666399474 2096,164810.609473548 2097,176429.401241704 2098,179188.089925259 2099,145997.635377703 2100,218676.768270367 2101,188323.861214226 2102,168690.0722914 2103,165088.746797705 2104,191435.007885166 2105,168864.404664512 2106,176041.882371574 2107,215911.674390325 2108,167388.238629016 2109,163854.786753017 2110,163299.477980171 2111,178298.214633119 2112,176376.586164775 2113,170211.043976522 2114,170818.344786366 2115,174388.867432503 2116,161112.987374671 2117,172179.082325307 2118,157798.309713876 2119,169106.151422924 2120,170129.531364292 2121,157680.227412949 2122,162690.209131977 2123,146968.379365095 2124,181507.721372455 2125,191215.589752983 2126,189432.689844522 2127,207271.484957719 2128,170030.807488363 2129,148409.806476335 2130,193850.613979055 2131,193808.319298263 2132,166300.235380627 2133,163474.633393212 2134,177473.606564978 2135,157443.925537187 2136,180681.007992057 2137,183463.17030026 2138,182481.763081195 2139,193717.15117887 2140,182782.55099007 2141,175530.651633287 2142,177804.057884623 2143,159448.670848577 2144,181338.976717529 2145,178553.558537021 2146,162820.928264556 2147,188832.479997186 2148,164682.185899437 2149,181549.735943801 2150,199158.097008868 2151,152889.520990566 2152,181150.551679116 2153,181416.732376013 2154,164391.238182305 2155,185421.046498812 2156,193981.327550004 2157,178824.324789223 2158,209270.051606246 2159,177801.266806344 2160,179053.762236101 2161,178762.170601992 2162,184655.300458183 2163,191284.655779772 2164,179598.085818785 2165,167517.628078595 2166,182873.903794044 2167,177484.91371363 2168,188444.597319524 2169,179184.153848562 2170,184365.175780982 2171,184479.322005212 2172,182927.863869391 2173,178611.639373646 2174,181943.343613558 2175,175080.614768394 2176,190720.794649138 2177,198422.868144723 2178,184482.11308349 2179,139214.952187861 2180,169233.113601757 2181,180664.118686848 2182,178818.742632666 2183,180422.049969947 2184,178601.93645581 2185,183083.159775993 2186,173163.101499699 2187,185968.161159774 2188,171226.050683054 2189,281643.976116786 2190,160031.711281258 2191,162775.979779394 2192,160735.445970193 2193,166646.109048572 2194,188384.548444549 2195,165830.697255197 2196,182138.358533039 2197,171595.397975647 2198,160337.079183809 2199,191215.088671543 2200,166956.093232213 2201,186581.830878692 2202,176450.548582099 2203,193743.194909801 2204,198882.566078408 2205,176385.102235149 2206,162447.639333636 2207,193782.555676777 2208,183653.890897141 2209,210578.623546866 2210,158527.164107319 2211,163081.025723456 2212,174388.867432503 2213,191905.870131966 2214,174388.867432503 2215,161642.711648983 2216,186939.507215101 2217,172482.165792649 2218,159695.999763546 2219,157230.369671007 2220,179188.089925259 2221,157972.82120994 2222,156804.951429181 2223,211491.972463654 2224,186537.246201062 2225,200468.161070551 2226,182241.340444154 2227,157342.225898399 2228,182022.387105998 2229,181244.510876788 2230,178556.671573788 2231,189547.199876284 2232,187948.65165563 2233,194107.287565956 2234,183521.710369283 2235,183682.123638416 2236,178483.353073443 2237,184003.879764736 2238,171318.59033449 2239,162039.754313997 2240,154846.252190699 2241,194822.365690957 2242,169788.738771463 2243,178891.554489941 2244,152084.772428865 2245,139169.86642879 2246,192439.536044606 2247,161067.859766557 2248,158762.648504781 2249,175569.690441774 2250,183659.795012187 2251,280618.132617258 2252,180051.809151659 2253,176519.18031559 2254,179028.429210291 2255,177161.583857224 2256,180081.508849842 2257,205895.254584712 2258,183389.78131415 2259,178543.647859512 2260,194798.320499104 2261,162845.613675766 2262,148103.867006579 2263,201016.171121215 2264,277936.12694354 2265,249768.279823405 2266,161596.052159825 2267,158011.114889899 2268,194089.683858004 2269,181733.336941451 2270,182852.32772198 2271,189893.003058465 2272,194650.210979875 2273,187904.461286262 2274,171774.925622692 2275,177998.685921479 2276,175648.484325498 2277,196918.071362067 2278,184299.838071218 2279,182379.855682734 2280,184050.725802482 2281,158296.975970284 2282,175053.355553278 2283,162293.376090644 2284,186328.880047186 2285,151422.116936538 2286,181969.358707768 2287,189122.67702416 2288,185645.475220346 2289,182829.898109257 2290,195848.788183328 2291,198785.059550672 2292,181676.126555428 2293,194131.012663328 2294,201416.004864508 2295,185096.577205616 2296,195158.972598372 2297,184795.783735112 2298,189168.263864671 2299,216855.260149095 2300,184946.642483576 2301,189317.51282069 2302,180803.277842406 2303,175061.18585763 2304,179074.839090732 2305,145708.764336107 2306,142398.022752011 2307,161474.534863641 2308,157025.945155458 2309,163424.037827357 2310,164692.778645345 2311,152163.2443541 2312,192383.215486656 2313,182520.230322476 2314,187254.507549722 2315,176489.659740359 2316,181520.466841293 2317,186414.978214721 2318,185197.764639705 2319,178657.794083741 2320,179731.198023759 2321,161748.271317074 2322,158608.749069322 2323,178807.370559878 2324,184187.158803897 2325,181686.10402108 2326,190311.050228337 2327,192252.496354076 2328,193954.849525775 2329,181044.201560887 2330,180258.131219792 2331,199641.657313834 2332,197530.775205517 2333,191777.196949138 2334,195779.543033588 2335,202112.046522999 2336,192343.34807661 2337,185191.359443218 2338,186760.207965688 2339,177733.78193528 2340,164430.391189608 2341,185299.601552401 2342,186414.012339254 2343,176401.921054593 2344,182381.322639642 2345,176334.184710805 2346,184901.735847457 2347,180085.766885029 2348,184901.735847457 2349,183967.561548763 2350,193046.301574659 2351,168538.969495849 2352,170157.842016969 2353,196559.709259637 2354,177133.709361852 2355,181553.279576244 2356,185770.606634739 2357,177017.595099274 2358,184123.358536806 2359,165970.357492196 2360,158151.985049452 2361,177086.476441481 2362,196373.896176551 2363,172465.707083115 2364,168590.782409896 2365,158820.474171061 2366,151611.37057651 2367,152125.028585543 2368,158404.073081048 2369,160692.078640755 2370,170175.22684199 2371,169854.436591138 2372,183410.785819008 2373,180347.194026928 2374,178930.528374292 2375,153346.220086301 2376,182675.204270589 2377,180770.649792036 2378,188714.148087543 2379,191393.608594076 2380,174016.157494425 2381,183189.685319552 2382,183621.508757866 2383,168991.29635758 2384,185306.650665866 2385,189030.680303208 2386,179208.665698449 2387,174901.452792889 2388,168337.406544343 2389,158234.96461859 2390,179562.453368834 2391,174176.391640607 2392,173931.531845427 2393,184111.729429665 2394,179374.482001188 2395,207348.811884535 2396,186983.419339031 2397,206779.094049527 2398,177472.074683935 2399,156727.948324862 2400,157090.568462479 2401,160387.032696693 2402,172410.28005086 2403,191603.365657467 2404,182152.207151253 2405,180161.697340702 2406,169652.235284283 2407,182503.520140218 2408,179714.630677039 2409,180282.570719908 2410,192600.338060371 2411,166115.491248565 2412,186379.553524443 2413,184361.992258449 2414,186220.965458121 2415,198176.47090687 2416,168437.776500131 2417,178003.582312015 2418,179180.469244588 2419,191930.561104806 2420,175590.266214964 2421,176713.19307219 2422,180159.090947005 2423,188090.100808026 2424,186184.717727913 2425,223055.588672278 2426,158270.753116401 2427,184733.12846644 2428,199926.378957429 2429,175075.785166001 2430,180917.925148076 2431,182067.760625207 2432,178238.60191545 2433,173454.944606532 2434,176821.936262814 2435,183642.191304235 2436,177254.582741058 2437,168715.950111702 2438,180096.931198144 2439,160620.728178758 2440,175286.544392273 2441,153494.783276297 2442,156407.65915545 2443,162162.525245786 2444,166809.886827197 2445,172929.156408918 2446,193514.330894137 2447,181612.141603756 2448,191745.386377068 2449,171369.325038261 2450,184425.470567051 2451,170563.252355189 2452,184522.369240168 2453,164968.947931153 2454,157939.621592364 2455,151520.381580069 2456,176129.508722531 2457,171112.978971478 2458,169762.081624282 2459,162246.828936295 2460,171339.303381589 2461,189034.753653813 2462,175758.873595981 2463,163351.721489893 2464,189806.546645026 2465,175370.990918319 2466,196895.599900301 2467,176905.917994834 2468,176866.557227858 2469,163590.677170026 2470,212693.502958393 2471,192686.931747717 2472,181578.684951827 2473,166475.457581812 2474,185998.255166219 2475,185527.714877908 2476,159027.118197683 2477,181169.654933769 2478,176732.915304722 2479,191619.294648838 2480,189114.303789324 2481,180934.635330334 2482,164573.372223048 2483,173902.011270196 2484,165625.127741229 2485,179555.219570787 2486,196899.720661579 2487,207566.12470446 2488,163899.981149274 2489,189179.428177786 2490,193892.880023125 2491,178980.874331431 2492,179749.876244365 2493,197999.674975598 2494,203717.470295797 2495,185249.261156892 2496,201691.208274848 2497,181956.548314794 2498,171895.936275806 2499,187245.168439419 2500,157816.77461318 2501,191702.912573325 2502,198599.420028908 2503,187193.313676329 2504,220514.993999535 2505,181814.527595192 2506,183750.755371907 2507,183000.431679579 2508,185830.971906573 2509,185497.872344187 2510,179613.437681321 2511,164454.967963631 2512,185127.237217638 2513,178750.613844623 2514,160927.61044889 2515,192562.808057836 2516,180990.24148554 2517,180064.941503122 2518,196070.997393789 2519,180352.919019023 2520,183367.953769362 2521,176734.841494027 2522,180848.220765939 2523,187806.059368823 2524,180521.52640004 2525,181502.754496154 2526,174525.87942676 2527,188927.984063168 2528,184728.870431253 2529,179857.975518011 2530,180962.868071609 2531,179194.066390078 2532,179591.789259484 2533,180638.463702549 2534,185846.215131922 2535,195174.031139141 2536,192474.56829063 2537,164200.595496827 2538,178403.094096818 2539,170774.84018302 2540,179879.945898337 2541,177668.192752792 2542,180174.328610725 2543,170643.303572141 2544,165448.004289838 2545,195531.754886222 2546,165314.177682121 2547,172532.757660882 2548,203310.218069877 2549,175090.062515883 2550,230841.338626282 2551,155225.19006632 2552,168322.342441945 2553,165956.259265265 2554,193956.817564124 2555,171070.367893827 2556,166285.243628001 2557,182875.801346628 2558,218108.536769738 2559,174378.777632042 2560,164731.316372391 2561,156969.695083273 2562,173388.854342604 2563,177559.628685119 2564,194297.789279905 2565,174894.588364005 2566,196544.144075798 2567,179036.158528149 2568,211423.986511149 2569,208156.398935188 2570,159233.941347257 2571,210820.115134931 2572,140196.10979821 2573,198678.469082978 2574,186818.610760803 2575,175044.797633861 2576,180031.162892704 2577,176889.171525162 2578,159638.856165666 2579,154287.264375509 2580,191885.618181273 2581,177503.378612934 2582,166548.31684976 2583,164475.14942856 2584,167484.744857879 2585,188683.160555403 2586,162243.399502668 2587,180807.213919103 2588,176279.079637039 2589,163438.959094218 2590,161495.5393685 2591,216032.303722443 2592,176632.181541401 2593,168743.001567144 2594,183810.11848086 2595,156794.36054728 2596,169136.43011395 2597,183203.318752456 2598,213252.926930889 2599,190550.327866959 2600,234707.209860273 2601,135751.318892816 2602,164228.45886894 2603,153219.437030419 2604,164210.746523801 2605,163883.229117973 2606,154892.776269956 2607,197092.08733832 2608,228148.376399122 2609,178680.587503997 2610,165643.341167808 2611,222406.642660249 2612,184021.843582599 2613,170871.094939159 2614,189562.873697309 2615,170591.884966356 2616,172934.351682851 2617,186425.069879189 2618,218648.131133006 2619,183035.606761141 2620,178378.906069427 2621,184516.716597846 2622,181419.5253183 2623,196858.923438425 2624,189228.701486278 2625,208973.380761028 2626,180269.86896412 2627,159488.713683953 2628,191490.299507521 2629,228684.245137946 2630,201842.998700429 2631,209242.82289186 2632,202357.62258493 2633,168238.61218265 2634,202524.12465369 2635,170588.771929588 2636,198375.31512987 2637,170636.827889889 2638,181991.079479377 2639,183994.54251844 2640,182951.482193584 2641,174126.297156192 2642,170575.496742588 2643,175332.239869971 2644,167522.061539111 2645,168095.583738538 2646,154406.415627461 2647,170996.973346087 2648,159056.890245639 2649,181373.6165193 2650,152272.560975937 2651,168664.346821336 2652,211007.008292301 2653,182909.515032911 2654,203926.829353303 2655,179082.825442944 2656,206260.099795032 2657,181732.443415757 2658,189698.740693148 2659,203074.34678979 2660,201670.634365666 2661,173756.812589691 2662,181387.076390881 2663,184859.155270535 2664,158313.615666777 2665,151951.955409666 2666,162537.52704471 2667,178998.337067854 2668,186732.584943041 2669,187323.318406165 2670,199437.232798284 2671,185546.680858653 2672,161595.015798593 2673,154672.422763036 2674,159355.710116165 2675,155919.014077746 2676,182424.87095604 2677,178100.589622319 2678,202577.900044456 2679,177862.778940605 2680,182056.024744887 2681,191403.199177104 2682,196264.754980043 2683,209375.003419718 2684,196691.81930173 2685,192458.431539585 2686,182242.80926507 2687,183259.503900506 2688,188108.243748841 2689,171418.640195797 2690,194698.882220432 2691,174841.84007522 2692,172965.476488899 2693,189386.323677132 2694,185682.618340257 2695,176412.012719061 2696,174976.489722867 2697,180718.581707643 2698,186131.188248242 2699,165220.786354033 2700,164115.893800435 2701,182125.729127024 2702,182285.140233276 2703,196325.442210366 2704,164865.215329881 2705,182694.492209823 2706,185425.485520958 2707,171414.7041191 2708,183433.472466085 2709,176844.981155794 2710,180568.187753206 2711,185948.625475832 2712,189388.291715481 2713,142754.489165865 2714,156106.800760811 2715,155895.397617561 2716,159851.977738548 2717,185157.832305524 2718,180716.291710805 2719,176901.093954071 2720,181017.222455218 2721,183269.159407668 2722,193550.830097069 2723,170625.842699726 2724,182012.405942725 2725,179162.507290733 2726,183269.159407668 2727,180589.836175042 2728,181465.935198741 2729,196053.029878304 2730,183421.020319014 2731,167926.839083612 2732,168027.530997889 2733,182164.26685407 2734,172469.071592608 2735,181059.374300472 2736,182997.570115536 2737,166140.504179894 2738,198515.546934075 2739,193789.648503294 2740,173550.025727531 2741,176487.943174734 2742,188813.302559147 2743,178531.911979192 2744,182145.731469001 2745,179196.465024103 2746,169618.349900686 2747,170010.168655046 2748,181739.671652174 2749,172846.934955574 2750,195560.8830172 2751,180358.114292956 2752,211817.702818093 2753,176170.128686742 2754,234492.248263699 2755,182450.956536015 2756,174902.068073146 2757,173684.174293738 2758,147196.673677562 2759,175231.189709791 2760,193417.64740633 2761,183313.601249761 2762,180882.250849082 2763,186735.697979808 2764,172922.865411247 2765,202551.677190573 2766,190485.634074173 2767,173439.49362151 2768,196613.598849219 2769,178152.259700828 2770,174519.904825949 2771,172627.796932837 2772,173732.689486435 2773,209219.844787023 2774,181059.374300472 2775,188515.443002459 2776,182164.26685407 2777,188137.901597981 2778,158893.54306269 2779,189579.65066771 2780,165229.803505847 2781,162186.071220207 2782,166374.879866351 2783,161665.184974757 2784,175079.328798445 2785,203840.874021305 2786,152129.078861057 2787,181012.141380101 2788,161305.53503837 2789,203326.392972343 2790,168385.571141831 2791,183564.365159986 2792,163784.619440861 2793,171989.192193993 2794,180839.95616829 2795,170895.923185907 2796,174071.054808518 2797,259423.859147546 2798,188000.824679588 2799,179171.703565498 2800,171022.241447762 2801,174126.297156192 2802,187625.573271948 2803,199567.946369234 2804,205328.078219268 2805,166231.535025379 2806,154743.91606057 2807,159714.537012622 2808,185563.069082422 2809,171500.796725006 2810,180983.443844799 2811,183141.236914997 2812,178498.634450214 2813,224323.710512388 2814,218200.642127877 2815,182283.177756557 2816,190054.639237419 2817,160192.453934518 2818,171289.393581756 2819,151131.098733642 2820,181721.458225594 2821,172725.053851858 2822,222438.699143414 2823,235419.373448928 2824,185150.926027596 2825,184772.239624699 2826,180658.216435809 2827,209673.316647174 2828,205939.810625621 2829,165633.573325837 2830,186030.317211014 2831,160312.319589212 2832,190702.440251029 2833,175122.810326699 2834,183783.13937519 2835,178290.666302221 2836,181605.343963015 2837,187992.451444752 2838,188885.11781517 2839,189959.344795118 2840,179258.619211334 2841,181518.750275669 2842,193008.659237315 2843,186313.89385619 2844,181499.39185067 2845,174126.297156192 2846,183918.612062767 2847,184114.270899227 2848,158540.947801398 2849,197034.759055859 2850,185170.284452595 2851,221134.533635148 2852,184306.637575967 2853,199792.302740996 2854,143237.803559736 2855,177294.838897736 2856,182368.620883855 2857,176487.943174734 2858,183849.408762071 2859,184964.141507413 2860,196395.969632434 2861,188374.936650438 2862,176261.296806135 2863,163628.142248426 2864,180618.032628904 2865,161647.329794081 2866,167129.598867773 2867,174750.988352687 2868,177560.202116333 2869,192577.796112839 2870,199202.898960871 2871,182818.156667308 2872,148217.262540651 2873,188997.797082492 2874,185807.928877601 2875,177030.477842021 2876,175942.474593632 2877,172912.518576433 2878,198359.248864591 2879,184379.133036383 2880,194255.566948886 2881,209449.651603064 2882,169979.323958443 2883,188206.281858748 2884,186412.438609167 2885,196761.386409959 2886,208353.269558209 2887,166548.067241044 2888,175942.474593632 2889,166790.457916434 2890,160515.850579067 2891,192167.621096362 2892,178751.551083369 2893,198678.894117024 2894,164553.120272354 2895,156887.932862327 2896,164185.777305524 2897,212992.120630876 2898,197468.550532521 2899,180106.84373966 2900,183972.071056674 2901,245283.198337927 2902,170351.963410756 2903,195596.307707478 2904,189369.756330412 2905,223667.404551664 2906,169335.310624364 2907,167411.02835165 2908,187709.555003968 2909,196526.002998991 2910,137402.569855589 2911,165086.775061735 2912,188506.431412274 2913,172917.456816012 2914,166274.325225982 2915,167081.220948984 2916,164788.778231138 2917,219222.423400059 2918,184924.279658997 2919,187741.866657478 ================================================ FILE: tests/test_commands.sh ================================================ #!/bin/bash # Verify all options are plumbed through to the MT. # Set envar KAGGLE_DEVELOPER to the Kaggle user name (probably already done). # When prompted to delete something, respond with "no". # Still need to adjust for assumptions about existing artifacts, like # the notebook "exercise-as-with" kaggle --version echo "kaggle competitions files" kaggle competitions files titanic --page-size=3 --page-token=abcd -v -q echo "kaggle competitions list" kaggle competitions list --group general --category featured --sort-by prize echo "kaggle competitions download" kaggle c download titanic -w -o -q kaggle c download titanic -f test.csv -p tost echo "kaggle competitions submit" kaggle c download house-prices-advanced-regression-techniques -f sample_submission.csv --force kaggle c submit house-prices-advanced-regression-techniques -f sample_submission.csv -m "Test message" echo "kaggle competitions submissions" kaggle c submissions house-prices-advanced-regression-techniques -v -q echo "kaggle competitions leaderboard" kaggle c leaderboard titanic -v -q -d -p leaders kaggle c leaderboard titanic -s > leaderboard.txt rm -r titanic.zip tost sample_submission.csv leaders leaderboard.txt echo "kaggle kernels list" kaggle k list -m -s Exercise --page-size 5 -p 2 -v --sort-by dateRun kaggle k list --parent $KAGGLE_DEVELOPER/exercise-lists kaggle k list --competition house-prices-advanced-regression-techniques --page-size 5 kaggle k list --dataset dansbecker/home-data-for-ml-course --page-size 5 kaggle k list --user $KAGGLE_DEVELOPER --language python --kernel-type notebook --output-type data echo "kaggle kernels files" kaggle kernels files kerneler/sqlite-global-default -v --page-size=1 echo "kaggle kernels init" kaggle k init -p tests/kernel echo "kaggle kernels get" kaggle k get -p tests/kernel $KAGGLE_DEVELOPER/exercise-as-with -m kaggle k get --wp $KAGGLE_DEVELOPER/exercise-as-with #sed -i s/exercise-as-with/exercise-delete/ tests/kernel/kernel-metadata.json #sed -i s/As \& With/Delete/ tests/kernel/kernel-metadata.json #mv tests/kernel/exercise-as-with.ipynb tests/kernel/exercise-delete.ipynb echo "kaggle kernels update" kaggle kernels update -p tests/kernel echo "kaggle kernels status" kaggle k status kerneler/sqlite-global-default echo "kaggle kernels output" kaggle k output kerneler/sqlite-global-default -o echo "kaggle kernels delete" #kaggle k delete $KAGGLE_DEVELOPER/exercise-delete #kaggle k delete $KAGGLE_DEVELOPER/exercise-delete --yes rm -f tests/kernel/exercise-as-with.ipynb tests/kernel/kernel-metadata.json exercise-as-with.ipynb echo "kaggle datasets list" kaggle d list --size 10 kaggle d list -m kaggle d list --user oktayrdeki --csv kaggle d list --file-type csv --page 2 --sort-by updated -s student --min-size 13000 --max-size 15000 kaggle d list --license odb --tags internet --search telco echo "kaggle datasets files" kaggle datasets files kerneler/brazilian-bird-observation-metadata-from-wikiaves --page-size=7 --page-token=abcd echo "kaggle datasets init" kaggle d init -p tests/dataset echo "kaggle datasets create" export SLUG=testing sed -i s/INSERT_TITLE_HERE/TitleHere/ tests/dataset/dataset-metadata.json sed -i s/INSERT_SLUG_HERE/$SLUG/ tests/dataset/dataset-metadata.json kaggle d create -p tests/dataset --public -q -t -r skip echo "kaggle datasets download" kaggle datasets download -d willianoliveiragibin/pixar-films kaggle d download goefft/public-datasets-with-file-types-and-columns -p tmp --unzip -o -q kaggle d download goefft/public-datasets-with-file-types-and-columns -f dataset_results.csv -w -q -o echo "kaggle datasets version" kaggle d version -m VersionNotesGoHere -p tests/dataset -q -t -r skip -d echo "kaggle datasets metadata" kaggle datasets metadata goefft/public-datasets-with-file-types-and-columns -p tests/dataset echo "kaggle datasets status" kaggle d status goefft/public-datasets-with-file-types-and-columns echo "kaggle datasets delete" kaggle d delete $KAGGLE_DEVELOPER/$SLUG kaggle d delete $KAGGLE_DEVELOPER/$SLUG --yes rm -rf tmp tests/dataset/dataset-metadata.json dataset_results.csv.zip dataset_results.csv pixar-films.zip echo "kaggle models init" mkdir tmp kaggle m init -p tmp echo "kaggle models list" kaggle m list --owner $KAGGLE_DEVELOPER --sort-by createTime -v kaggle m list -s gemini --page-size 5 echo "kaggle models create" sed -i s/INSERT_OWNER_SLUG_HERE/$KAGGLE_DEVELOPER/ tmp/model-metadata.json sed -i s/INSERT_TITLE_HERE/ModelTitle/ tmp/model-metadata.json sed -i s/INSERT_SLUG_HERE/test-model/ tmp/model-metadata.json kaggle m create -p tmp echo "kaggle models update" kaggle m update -p tmp sleep 10 echo "kaggle models get" kaggle m get -p tmp $KAGGLE_DEVELOPER/test-model echo "kaggle models instances init" kaggle m instances init -p tmp echo "kaggle models instances create" sed -i s/INSERT_OWNER_SLUG_HERE/$KAGGLE_DEVELOPER/ tmp/model-instance-metadata.json sed -i s/INSERT_EXISTING_MODEL_SLUG_HERE/test-model/ tmp/model-instance-metadata.json sed -i s/INSERT_INSTANCE_SLUG_HERE/main/ tmp/model-instance-metadata.json sed -i s/INSERT_FRAMEWORK_HERE/jax/ tmp/model-instance-metadata.json echo "a,b,c,d" > tmp/data.csv kaggle models instances create -p tmp -q -r skip sleep 10 echo "kaggle models instances update" kaggle models instances update -p tmp sleep 10 echo "kaggle models instances get" kaggle models instances get $KAGGLE_DEVELOPER/test-model/jax/main -p tmp echo "kaggle models instances files" kaggle models instances files $KAGGLE_DEVELOPER/test-model/jax/main -v --page-size 5 echo "kaggle models instances versions files" kaggle models instances versions files google/gemma/pytorch/7b/2 -v --page-size=3 --page-token=abcd echo "kaggle models instances versions create" kaggle models instances versions create -p tmp -q -r skip -n VersionNotes $KAGGLE_DEVELOPER/test-model/jax/main echo "kaggle models instances versions download" kaggle models instances versions download -p tmp -q -f --untar $KAGGLE_DEVELOPER/test-model/jax/main/1 rm -rf tmp rm -f results.csv echo "kaggle models instances versions delete" kaggle m instances versions delete $KAGGLE_DEVELOPER/test-model/jax/main/1 kaggle m instances versions delete $KAGGLE_DEVELOPER/test-model/jax/main/1 -y echo "kaggle models instances delete" kaggle m instances delete $KAGGLE_DEVELOPER/test-model/jax/main kaggle m instances delete $KAGGLE_DEVELOPER/test-model/jax/main -y echo "kaggle models delete" kaggle m delete $KAGGLE_DEVELOPER/test-model kaggle m delete $KAGGLE_DEVELOPER/test-model -y ================================================ FILE: tests/unit_tests.py ================================================ # coding=utf-8 import json import shutil import unittest import os import sys import time from requests import HTTPError # noinspection PyTypeChecker sys.path.insert(0, "..") from kaggle import api ApiException = IOError # Unit test names include a letter to sort them in run order. # That seemed easier and more obvious than defining a test suite. # Run these tests in the container with the server running. # In the case of unresolvable failure, run reset_database.sh. # To run from Rider, create a Python tests>Unittests run config. # Give it the module: unit_tests.TestKaggleApi # Set the working directory to: kaggle-cli/tests # Define some envars: # KAGGLE_API_ENVIRONMENT=LOCALHOST # KAGGLE_CONFIG_DIR=/home/kaggle/.config/kaggle/dev # KAGGLE_KEY=local_api_token # KAGGLE_USERNAME= # Add those envars to the Python Tests>Unittest template to # make running individual tests easier. test_user = api.config_values["username"] model_title = "testing" instance_name = "test" framework_name = "jax" kernel_name = "testing-x" dataset_name = "kaggleapi-dataset-x" up_file = "sample_submission.csv" description = "House prices submission message" competition = "house-prices-advanced-regression-techniques" dataset_directory = "dataset" kernel_directory = "kernel" model_directory = "model" model_inst_directory = os.path.join(model_directory, "instance") model_inst_vers_directory = os.path.join(model_inst_directory, "version") # Max retries to get kernel status max_status_tries = 10 def tearDownModule(): file = os.path.join(dataset_directory, api.DATASET_METADATA_FILE) if os.path.exists(file): os.remove(file) file = os.path.join(kernel_directory, api.KERNEL_METADATA_FILE) if os.path.exists(file): os.remove(file) file = os.path.join(model_directory, api.MODEL_METADATA_FILE) if os.path.exists(file): os.remove(file) file = os.path.join(model_inst_directory, api.MODEL_INSTANCE_METADATA_FILE) if os.path.exists(file): os.remove(file) def update_kernel_metadata_file(metadata_file, k_name): with open(metadata_file) as f: meta_data = json.load(f) meta_id = meta_data["id"] if "INSERT_KERNEL_SLUG_HERE" in meta_id: meta_id = meta_id.replace("INSERT_KERNEL_SLUG_HERE", k_name) meta_title = meta_data["title"] if "INSERT_TITLE_HERE" == meta_title: meta_title = k_name meta_path = meta_data["code_file"] if "INSERT_CODE_FILE_PATH_HERE" == meta_path: meta_path = f"{k_name}.ipynb" meta_data["id"] = meta_id meta_data["title"] = meta_title meta_data["code_file"] = meta_path meta_data["language"] = "python" meta_data["kernel_type"] = "notebook" with open(metadata_file, "w") as f: json.dump(meta_data, f, indent=2) return meta_data def initialize_dataset_metadata_file(dataset_dir): metadata_file = os.path.join(dataset_dir, api.DATASET_METADATA_FILE) try: with open(metadata_file) as f: original = json.load(f) if "versionNumber" in original: version_num = int(original["versionNumber"]) else: version_num = 0 except FileNotFoundError: version_num = 0 version_num += 1 return version_num, metadata_file def update_dataset_metadata_file(metadata_file, data_name, version_num): with open(metadata_file) as f: meta_data = json.load(f) meta_id = meta_data["id"] if "INSERT_SLUG_HERE" in meta_id: meta_id = meta_id.replace("INSERT_SLUG_HERE", data_name) meta_title = meta_data["title"] if "INSERT_TITLE_HERE" == meta_title: meta_title = data_name meta_data["id"] = meta_id meta_data["title"] = meta_title meta_data["versionNumber"] = version_num if not "resources" in meta_data: resource_list = [ { "path": "data.csv", "description": "Description", "schema": { "fields": [ {"name": "NumberField", "description": "id", "type": "number"}, {"name": "StringField", "description": "label", "type": "string"}, ] }, } ] meta_data.update({"resources": resource_list}) with open(metadata_file, "w") as f: json.dump(meta_data, f) def update_model_metadata(metadata_file, owner, title, slug): with open(metadata_file) as f: meta_data = json.load(f) meta_id = meta_data["ownerSlug"] if "INSERT_OWNER_SLUG_HERE" == meta_id: meta_id = owner meta_title = meta_data["title"] if "INSERT_TITLE_HERE" == meta_title: meta_title = title meta_path = meta_data["slug"] if "INSERT_SLUG_HERE" == meta_path: meta_path = slug meta_data["ownerSlug"] = meta_id meta_data["title"] = meta_title meta_data["slug"] = meta_path with open(metadata_file, "w") as f: json.dump(meta_data, f, indent=2) return meta_data def update_model_instance_metadata(metadata_file, owner, model_slug, instance_slug, framework): with open(metadata_file) as f: meta_data = json.load(f) meta_owner = meta_data["ownerSlug"] if "INSERT_OWNER_SLUG_HERE" == meta_owner: meta_owner = owner meta_framework = meta_data["framework"] if "INSERT_FRAMEWORK_HERE" == meta_framework: meta_framework = framework meta_instance = meta_data["instanceSlug"] if "INSERT_INSTANCE_SLUG_HERE" == meta_instance: meta_instance = instance_slug meta_model = meta_data["modelSlug"] if "INSERT_EXISTING_MODEL_SLUG_HERE" == meta_model: meta_model = model_slug meta_data["ownerSlug"] = meta_owner meta_data["modelSlug"] = meta_model meta_data["framework"] = meta_framework meta_data["instanceSlug"] = meta_instance with open(metadata_file, "w") as f: json.dump(meta_data, f, indent=2) return meta_data def print_fields(instance, fields): # For debugging. for f in fields: if not hasattr(instance, api.camel_to_snake(f)): print(f"Missing field: {f} named: {api.camel_to_snake(f)}") # noinspection PyTypeChecker class TestKaggleApi(unittest.TestCase): version_number, meta_file = initialize_dataset_metadata_file(dataset_directory) # Initialized from Response objects. competition_file = None kernel_slug = "" kernel_metadata_path = "" dataset = "" dataset_file = None model_instance = "" model_meta_data = None model_metadata_file = "" instance_metadata_file = "" # Inbox def test_files_upload(self): a = 2 # Change this value to run this test. if a - 1 == 1: return # Only run this test when needed because it uploads an inbox file. filename = "tmp_file.test" with open(filename, "w") as f: f.write("test") try: api.files_upload_cli([filename], "kaggle-cli-test", False, False) finally: if os.path.exists("tmp_file.test"): os.remove("tmp_file.test") # Kernels def test_kernels_a_list(self): try: kernels = api.kernels_list(sort_by="dateCreated", user="stevemessick", language="python") self.assertGreater(len(kernels), 0) api.kernels_list_cli(user="stevemessick", csv_display=True) except ApiException as e: self.fail(f"kernels_list failed: {e}") def test_kernels_b_initialize(self): try: self.kernel_metadata_path = api.kernels_initialize(kernel_directory) self.assertTrue(os.path.exists(self.kernel_metadata_path)) except ApiException as e: self.fail(f"kernels_initialize failed: {e}") def test_kernels_c_push(self): if self.kernel_metadata_path == "": self.test_kernels_b_initialize() try: md = update_kernel_metadata_file(self.kernel_metadata_path, kernel_name) push_result = api.kernels_push(kernel_directory) self.assertIsNotNone(push_result.ref) self.assertTrue(isinstance(push_result.version_number, int)) self.kernel_slug = md["id"] time.sleep(30) except ApiException as e: self.fail(f"kernels_push failed: {e}") def test_kernels_d_status(self): if self.kernel_slug == "": self.test_kernels_c_push() try: status_result = api.kernels_status(self.kernel_slug) start_time = time.time() # If this loop is stuck because the kernel stays queued, go to the Kaggle website # on localhost and cancel the active event. That will exit the loop, but you may # need to clean up other active kernels to get it to run again. count = 0 while status_result.status == "running" or status_result.status == "queued" or count >= max_status_tries: time.sleep(5) status_result = api.kernels_status(self.kernel_slug) print(status_result.status) if count >= max_status_tries: self.fail(f"Could not get kernel status in allowed trys. Status: {status_result.status}") end_time = time.time() print(f"kernels_status ready in {end_time-start_time}s") except ApiException as e: self.fail(f"kernels_status failed: {e}") def test_kernels_e_list_files(self): if self.kernel_slug == "": self.test_kernels_c_push() try: fs = api.kernels_list_files(self.kernel_slug) # TODO Make sure the test uses a kernel that has at least one file. self.assertGreaterEqual(len(fs.files), 0) except ApiException as e: self.fail(f"kernels_list_files failed: {e}") def test_kernels_f_output(self): fs = [] try: fs, token = api.kernels_output("kerneler/sqlite-global-default", "kernel/tmp") self.assertIsInstance(fs, list) if token: print(token) except ApiException as e: self.fail(f"kernels_output failed: {e}") finally: for file in fs: if os.path.exists(file): os.remove(file) if os.path.exists("kernel/tmp"): os.rmdir("kernel/tmp") def test_kernels_g_pull(self): if self.kernel_metadata_path == "": self.test_kernels_c_push() fs = "" try: fs = api.kernels_pull(f"{test_user}/{kernel_name}", "kernel/tmp", metadata=True) self.assertTrue(os.path.exists(fs)) with open(f'{fs}/{self.kernel_metadata_path.split("/")[1]}') as f: metadata = json.load(f) [ self.assertTrue(metadata.get(f)) for f in ["id", "id_no", "title", "code_file", "language", "kernel_type"] ] [ self.assertTrue(metadata.get(f) is not None) for f in [ "is_private", "enable_gpu", "enable_tpu", "enable_internet", "keywords", "dataset_sources", "kernel_sources", "competition_sources", "model_sources", ] ] except ApiException as e: self.fail(f"kernels_pull failed: {e}") finally: shutil.rmtree(fs) # Competitions def test_competition_a_list(self): try: competitions = api.competitions_list(group="general").competitions self.assertGreater(len(competitions), 0) self.assertLessEqual(len(competitions), 20) [self.assertTrue(hasattr(competitions[0], api.camel_to_snake(f))) for f in api.competition_fields] competitions = api.competitions_list(page=2, category="gettingStarted", sort_by="prize").competitions self.assertEqual(len(competitions), 1) except ApiException as e: self.fail(f"competitions_list failed: {e}") def test_competition_b_submit(self): try: self.skip_submissions = False result = api.competition_submit_cli(up_file, description, competition) if not result or not result.startswith("Successfully submitted"): self.fail(f"competition_submit failed: {result}") except HTTPError: # Handle submission limit reached gracefully (potentially skip the test) print("Competition submission limit reached for the day") self.skip_submissions = True pass except ApiException as e: self.fail(f"competition_submit failed: {e}") def test_competition_c_submissions(self): self.test_competition_b_submit() try: submissions = api.competition_submissions(competition) self.assertIsInstance(submissions, list) if not self.skip_submissions: self.assertGreater(len(submissions), 0) [self.assertTrue(hasattr(submissions[0], api.camel_to_snake(f))) for f in api.submission_fields] except ApiException as e: self.fail(f"competition_submissions failed: {e}") def test_competition_d_list_files(self): try: competition_files = api.competition_list_files(competition).files self.assertIsInstance(competition_files, list) self.assertGreater(len(competition_files), 0) self.competition_file = competition_files[0] [self.assertTrue(hasattr(competition_files[0], api.camel_to_snake(f))) for f in api.competition_file_fields] except ApiException as e: self.fail(f"competition_list_files failed: {e}") def test_competition_e_download_file(self): if self.competition_file is None: self.test_competition_d_list_files() try: api.competition_download_file(competition, self.competition_file.ref, force=True) self.assertTrue(os.path.exists(self.competition_file.ref)) api.competition_download_file(competition, self.competition_file.ref, force=False) except ApiException as e: self.fail(f"competition_download_file failed: {e}") finally: if os.path.exists(self.competition_file.ref): os.remove(self.competition_file.ref) def test_competition_f_download_files(self): try: api.competition_download_files(competition) self.assertTrue(os.path.exists(f"{competition}.zip")) self.assertTrue(os.path.getsize(f"{competition}.zip") > 0) except ApiException as e: self.fail(f"competition_download_files failed: {e}") finally: if os.path.exists(f"{competition}.zip"): os.remove(f"{competition}.zip") def test_competition_g_leaderboard_view(self): try: result = api.competition_leaderboard_view(competition) self.assertIsInstance(result, list) self.assertGreater(len(result), 0) [self.assertTrue(hasattr(result[0], api.camel_to_snake(f))) for f in api.competition_leaderboard_fields] except ApiException as e: self.fail(f"competition_leaderboard_view failed: {e}") def test_competition_h_leaderboard_download(self): try: api.competition_leaderboard_download(competition, "tmp") self.assertTrue(os.path.exists(f"tmp/{competition}.zip")) except ApiException as e: self.fail(f"competition_leaderboard_download failed: {e}") finally: if os.path.exists(f"tmp/{competition}.zip"): os.remove(f"tmp/{competition}.zip") if os.path.exists("tmp"): os.rmdir("tmp") # Datasets def test_dataset_a_list(self): try: datasets = api.dataset_list(sort_by="votes") self.assertGreater(len(datasets), 0) self.dataset = str(datasets[0].ref) [self.assertTrue(hasattr(datasets[0], api.camel_to_snake(f))) for f in api.dataset_fields] datasets = api.dataset_list(license_name="other", file_type="bigQuery") self.assertGreater(len(datasets), 10) except ApiException as e: self.fail(f"dataset_list failed: {e}") def test_dataset_b_metadata(self): if self.dataset == "": self.test_dataset_a_list() try: m = api.dataset_metadata(self.dataset, dataset_directory) self.assertTrue(os.path.exists(m)) except ApiException as e: self.fail(f"dataset_metadata failed: {e}") def test_dataset_c_metadata_update(self): if self.dataset == "": self.test_dataset_a_list() if not os.path.exists(os.path.join(dataset_directory, api.DATASET_METADATA_FILE)): self.test_dataset_b_metadata() try: api.dataset_metadata_update(self.dataset, dataset_directory) # TODO Make the API method return something, and not exit when it fails. except ApiException as e: self.fail(f"dataset_metadata_update failed: {e}") def test_dataset_d_list_files(self): if self.dataset == "": self.test_dataset_a_list() try: response = api.dataset_list_files(self.dataset) self.assertIsInstance(response.dataset_files, list) self.assertGreater(len(response.dataset_files), 0) self.dataset_file = response.dataset_files[0] [self.assertTrue(hasattr(self.dataset_file, api.camel_to_snake(f))) for f in api.dataset_file_fields] api.dataset_list_files_cli(self.dataset) except ApiException as e: self.fail(f"dataset_list_files failed: {e}") def test_dataset_e_status(self): if self.dataset == "": self.test_dataset_a_list() try: status = api.dataset_status(self.dataset) self.assertIn(status, ["ready", "pending", "error"]) except ApiException as e: self.fail(f"dataset_status failed: {e}") def test_dataset_f_download_file(self): if self.dataset_file is None: self.test_dataset_d_list_files() try: api.dataset_download_file(self.dataset, self.dataset_file.name, "tmp") self.assertTrue(os.path.exists(f"tmp/{self.dataset_file.name}")) except ApiException as e: self.fail(f"dataset_download_file failed: {e}") finally: if os.path.exists(f"tmp/{self.dataset_file.name}"): os.remove(f"tmp/{self.dataset_file.name}") if os.path.exists("tmp"): os.rmdir("tmp") def test_dataset_g_download_files(self): if self.dataset == "": self.test_dataset_a_list() ds = ["a", "b"] try: api.dataset_download_files(self.dataset + "/1") ds = self.dataset.split("/") self.assertTrue(os.path.exists(f"{ds[1]}.zip")) except ApiException as e: self.fail(f"dataset_download_files failed: {e}") finally: if os.path.exists(f"{ds[1]}.zip"): os.remove(f"{ds[1]}.zip") def test_dataset_h_initialize(self): try: api.dataset_initialize("dataset") self.assertTrue(os.path.exists(os.path.join(dataset_directory, api.DATASET_METADATA_FILE))) except ApiException as e: self.fail(f"dataset_initialize failed: {e}") def test_dataset_ia_create_new(self, fail_if_exists=True): if not os.path.exists(os.path.join(dataset_directory, api.DATASET_METADATA_FILE)): self.test_dataset_h_initialize() try: update_dataset_metadata_file(self.meta_file, dataset_name, self.version_number) new_dataset = api.dataset_create_new(dataset_directory) self.assertIsNotNone(new_dataset) if len(new_dataset.error) > 0: if fail_if_exists: self.fail(f"dataset_create_new failed: {new_dataset.error}") except ApiException as e: if fail_if_exists: self.fail(f"dataset_create_new failed: {e}") def test_dataset_ib_wait_after_create(self): # When running all tests sequentially, give the new dataset some time to stabilize. time.sleep(10) # TODO: Use a status check. def test_dataset_j_create_version(self): if not os.path.exists(os.path.join(dataset_directory, api.DATASET_METADATA_FILE)): self.test_dataset_ia_create_new(fail_if_exists=False) try: new_version = api.dataset_create_version(dataset_directory, "Notes") self.assertIsNotNone(new_version) self.assertTrue(new_version.error == "") self.assertFalse(new_version.ref == "") except ApiException as e: self.fail(f"dataset_create_version failed: {e}") def test_dataset_k_delete(self): self.test_dataset_ia_create_new(fail_if_exists=False) api.dataset_delete(None, dataset_name, True) # Models def test_model_a_list(self): try: ms = api.model_list() self.assertIsInstance(ms, list) self.assertGreater(len(ms), 0) [self.assertTrue(hasattr(ms[0], api.camel_to_snake(f))) for f in api.model_fields] except ApiException as e: self.fail(f"models_list failed: {e}") def test_model_b_initialize(self): try: self.model_metadata_file = api.model_initialize(model_directory) self.assertTrue(os.path.exists(self.model_metadata_file)) self.model_meta_data = update_model_metadata(self.model_metadata_file, test_user, model_title, model_title) self.model_instance = f'{test_user}/{self.model_meta_data["slug"]}/{framework_name}/{instance_name}' except ApiException as e: self.fail(f"model_initialize failed: {e}") def test_model_c_create_new(self): if self.model_metadata_file == "": self.test_model_b_initialize() try: model = api.model_create_new(model_directory) if model.error: if "already used" in model.error: delete_response = api.model_delete(f"{test_user}/{model_title}", True) if delete_response.error: self.fail(delete_response.error) else: model = api.model_create_new(model_directory) if model.error: self.fail(model.error) else: self.fail(model.error) self.assertIsNotNone(model.ref) self.assertGreater(len(model.ref), 0) [self.assertTrue(hasattr(model, api.camel_to_snake(f))) for f in ["id", "url"]] except ApiException as e: self.fail(f"model_create_new failed: {e}") def test_model_d_get(self): try: model_data = api.model_get(f"{test_user}/{model_title}") self.assertIsNotNone(model_data.ref) self.assertGreater(len(model_data.ref), 0) self.assertEqual(model_data.title, model_title) [self.assertTrue(hasattr(model_data, api.camel_to_snake(f))) for f in api.model_all_fields] except ApiException as e: self.fail(f"model_get failed: {e}") def test_model_e_update(self): if self.model_metadata_file == "": self.test_model_c_create_new() try: update_response = api.model_update(model_directory) self.assertEqual(len(update_response.error), 0) self.assertIsNotNone(update_response.ref) self.assertGreater(len(update_response.ref), 0) except ApiException as e: self.fail(f"model_update failed: {e}") # Model instances def test_model_instance_a_initialize(self): try: self.instance_metadata_file = api.model_instance_initialize(model_inst_directory) self.assertTrue(os.path.exists(self.instance_metadata_file)) except ApiException as e: self.fail(f"model_instance_initialize failed: {e}") def test_model_instance_b_create(self, check_result: bool = True): if self.model_meta_data is None: self.test_model_b_initialize() if self.instance_metadata_file == "": self.test_model_instance_a_initialize() try: update_model_instance_metadata( self.instance_metadata_file, test_user, self.model_meta_data["slug"], instance_name, framework_name ) inst_create_resp = api.model_instance_create(model_inst_directory) if check_result: if inst_create_resp.error: if "already exists" in inst_create_resp.error: delete_response = api.model_instance_delete(f"{test_user}/{model_title}", True) if delete_response.error: self.fail(delete_response.error) else: inst_create_resp = api.model_instance_create(model_inst_directory) if inst_create_resp.error: self.fail(inst_create_resp.error) else: self.fail(inst_create_resp.error) self.assertIsNotNone(inst_create_resp.ref) self.assertGreater(len(inst_create_resp.ref), 0) except ApiException as e: self.fail(f"model_instance_create failed: {e}") def test_model_instance_b_wait_after_create(self): # When running all tests sequentially, give the new model some time to stabilize. time.sleep(10) # TODO: Find a better way to detect model stability. def test_model_instance_c_get(self): if self.model_instance == "": self.test_model_b_initialize() self.test_model_c_create_new() self.test_model_instance_a_initialize() self.test_model_instance_b_create() self.test_model_instance_b_wait_after_create() self.test_model_instance_c_get() try: inst_get_resp = api.model_instance_get(self.model_instance) self.assertIsNotNone(inst_get_resp.url) self.assertGreater(len(inst_get_resp.url), 0) os.makedirs("model/tmp", exist_ok=True) api.model_instance_get_cli(self.model_instance, "model/tmp") except ApiException as e: self.fail(f"model_instance_get failed: {e}") finally: shutil.rmtree("model/tmp") def test_model_instance_d_files(self): if self.model_instance == "": self.test_model_b_initialize() try: inst_files_resp = api.model_instance_files(self.model_instance) self.assertIsInstance(inst_files_resp.files, list) self.assertGreater(len(inst_files_resp.files), 0) [self.assertTrue(hasattr(inst_files_resp.files[0], api.camel_to_snake(f))) for f in api.model_file_fields] except ApiException as e: self.fail(f"model_instance_files failed: {e}") def test_model_instance_e_update(self): if self.model_instance == "": self.test_model_b_initialize() self.test_model_instance_a_initialize() try: self.test_model_c_create_new() except AssertionError: pass try: update_model_instance_metadata( self.instance_metadata_file, test_user, self.model_meta_data["slug"], instance_name, framework_name ) except AssertionError: pass self.test_model_instance_b_create(check_result=False) self.test_model_instance_b_wait_after_create() try: inst_update_resp = api.model_instance_update(model_inst_directory) self.assertIsNotNone(inst_update_resp) self.assertIsNotNone(inst_update_resp.ref) self.assertGreater(len(inst_update_resp.ref), 0) [self.assertTrue(hasattr(inst_update_resp, api.camel_to_snake(f))) for f in ["error", "id", "ref", "url"]] except ApiException as e: self.fail(f"model_instance_update failed: {e}") def test_model_instance_f_list(self): if self.model_instance == "": self.test_model_instance_c_get() try: a, b, c, d = self.model_instance.split("/") inst_files_resp = api.model_instances_list(f"{a}/{b}") self.assertIsInstance(inst_files_resp.instances, list) self.assertGreater(len(inst_files_resp.instances), 0) [ self.assertTrue(hasattr(inst_files_resp.instances[0], api.camel_to_snake(f))) for f in api.model_instance_fields ] except ApiException as e: self.fail(f"model_instance_files failed: {e}") # Model instance versions def test_model_instance_version_a_create(self): if self.model_instance == "": self.test_model_b_initialize() try: version_metadata_resp = api.model_instance_version_create(self.model_instance, model_inst_vers_directory) self.assertIsNotNone(version_metadata_resp.ref) [self.assertTrue(hasattr(version_metadata_resp, api.camel_to_snake(f))) for f in ["id", "url", "error"]] except ApiException as e: self.fail(f"model_instance_version_create failed: {e}") def test_model_instance_version_b_files(self): if self.model_instance == "": self.test_model_b_initialize() try: r = api.model_instance_version_files(f"{self.model_instance}/1") self.assertIsInstance(r.files, list) self.assertGreater(len(r.files), 0) api.model_instance_version_files_cli(f"{self.model_instance}/1") except ApiException as e: self.fail(f"model_instance_version_files failed: {e}") def test_model_instance_version_c_download(self): for x in range(0, 5): if self.model_instance == "": self.test_model_b_initialize() version_file = "" try: version_file = api.model_instance_version_download(f"{self.model_instance}/1", "tmp", force=True) self.assertTrue(os.path.exists(version_file)) except KeyError: pass # TODO Create a version that has content. except ApiException as e: if e.strerror.startsWith("The archive is not ready yet"): continue self.fail(f"model_instance_version_download failed: {e}") finally: if os.path.exists(version_file): os.remove(version_file) if os.path.exists("tmp"): os.rmdir("tmp") return def test_model_instance_version_d_list(self): pass # Kernel deletion def test_kernels_h_delete(self): print("Running test_kernels_h_delete") if self.kernel_slug == "": self.test_kernels_c_push() try: api.kernels_delete(self.kernel_slug, no_confirm=True) # The kernels_delete method prints success, no specific return to assert. # If no exception is raised, we assume success. except ApiException as e: self.fail(f"kernels_delete failed: {e}") # Model deletion def test_model_instance_version_d_delete(self): if self.model_instance == "": self.test_model_b_initialize() try: version_delete_resp = api.model_instance_version_delete(f"{self.model_instance}/1", True) self.assertEqual(len(version_delete_resp.error), 0, msg=version_delete_resp.error) except ApiException as e: self.fail(f"model_instance_version_delete failed: {e}") def test_model_instance_x_delete(self): if self.model_instance == "": self.test_model_b_initialize() try: inst_update_resp = api.model_instance_delete(self.model_instance, True) self.assertIsNotNone(inst_update_resp) if len(inst_update_resp.error): print(inst_update_resp.error) self.assertEqual(len(inst_update_resp.error), 0) except ApiException as e: self.fail(f"model_instance_delete failed: {e}") def test_model_z_delete(self): try: delete_response = api.model_delete(f"{test_user}/{model_title}", True) if delete_response.error: self.fail(delete_response.error) else: pass except ApiException as e: self.fail(f"model_delete failed: {e}") if __name__ == "__main__": unittest.main() ================================================ FILE: tools/cicd/integration-tests.yaml ================================================ steps: # Access and store the secret in a file on a shared volume - name: 'gcr.io/cloud-builders/gcloud' id: 'download-secrets' script: | #!/usr/bin/env bash gcloud secrets versions access latest --secret=integration-tests --project=464139560241 > /root/secrets.sh volumes: - name: 'root' path: /root # Source the secrets and run integration tests using the built Python image - name: us-docker.pkg.dev/$PROJECT_ID/tools/hatch:$_PYTHON_VERSION id: integration-tests waitFor: ['download-secrets'] script: | #!/usr/bin/env bash export KAGGLE_USERNAME export KAGGLE_KEY source /root/secrets.sh hatch run test:integration volumes: - name: 'root' path: /root substitutions: _PYTHON_VERSION: '3.11' ================================================ FILE: tools/releases/Dockerfile ================================================ FROM debian:12.0 # Install some useful tools. RUN apt-get update -y && \ apt-get install -y sudo \ curl \ unzip \ default-jre \ python3-pip COPY requirements.txt requirements.txt RUN cat requirements.txt RUN pip install --require-hashes -r requirements.txt --break-system-packages ENTRYPOINT ["/bin/bash"] ================================================ FILE: tools/releases/cloudbuild.yaml ================================================ steps: # Import builder if exists. # Note: the reason to use bash is to be able to return an exit code 0 even if # the docker image doesn't exist yet. - name: "gcr.io/cloud-builders/docker" entrypoint: "bash" args: - "-c" - | docker pull ${_IMAGE_REPO_NAME}/${PROJECT_ID}/cli-releaser || exit 0 # Build a modified version of the python image, including tools to build and # release the Kaggle CLI. # Use the previous built image as cache. - name: "gcr.io/cloud-builders/docker" dir: "tools/releases" args: - build - -f - Dockerfile - -t - ${_IMAGE_REPO_NAME}/${PROJECT_ID}/cli-releaser - --cache-from - ${_IMAGE_REPO_NAME}/${PROJECT_ID}/cli-releaser - . - name: "${_IMAGE_REPO_NAME}/${PROJECT_ID}/cli-releaser" id: generate-cli entrypoint: bash args: - "-c" - | rm -f kaggle cp -r src/kaggle . python3 -m pip install build --break-system-packages python3 -m build # Move the built CLI to a volume that will survive to next steps. mv dist /root/ volumes: - name: "root" path: /root # Get the pypi token from Secret Manager, and create the ~/.pypirc file. - name: "gcr.io/cloud-builders/gcloud" id: create-credentials-pypirc entrypoint: "bash" args: - "-c" - | token_test=$(gcloud secrets versions access latest --secret=test-pypi-token) token=$(gcloud secrets versions access latest --secret=pypi-token) cat >~/.pypirc < $KAGGLE_CONFIG_FILE chmod 600 $KAGGLE_CONFIG_FILE echo "dev credentials created for username '$username'." echo "PLEASE VERIFY this matches your Kaggle username!" echo "If not, update the $KAGGLE_CONFIG_DIR/kaggle.json file manually to set your Kaggle username. You will only need to do this once." fi ================================================ FILE: tools/use-prod.sh ================================================ # Source this file to run Kaggle Api V1 against https://www.kaggle.com. if [[ "$0" == "$BASH_SOURCE" ]]; then echo -e "Source this file to run kaggle api cli against prod:\n" echo "$ source use-prod.sh" echo exit 1 fi unset KAGGLE_API_ENVIRONMENT unset KAGGLE_CONFIG_DIR if ! [[ -f "${XDG_CONFIG_HOME:-$HOME/.config}/kaggle/kaggle.json" ]]; then echo "Warning: Please download an API token at https://www.kaggle.com/settings and" echo "copy it to home directory to run the client against prod." echo else chmod 600 "${XDG_CONFIG_HOME:-$HOME/.config}/kaggle/kaggle.json" fi